Designing Twitter Part 1: Introduction
This will be the first article in a series where I will be designing a Twitter clone. By “design”, I don’t mean the actual interface, but the way such a system will be built, what micro-services will it have, how will they interact with one another, and more. In other words, the system architecture. These articles will explain the system design of such a service and how I think that it should be made.
Tools of trade
I don’t know the exact inner working of Twitter and how it is actually made. I can guess a few things and I can assume others based on best-practices, however, the exact system design is a mystery to me. Still, I believe that we can make something that is functional, scalable, and robust. Since this is the first article in the series, let’s look at the tools and frameworks that we will be using. The list may not be complete since I may discover new requirements as the series progresses.
Play Framework – For the backend, I will be using mostly Play framework. This, however, is not mandatory and can be switched to Spring, or something else that you are more comfortable with. Some may suggest Node.js or Django, however, I think that these frameworks are not suitable for such large-scale applications. Furthermore, I am a Java Developer, so I will be choosing a Java Framework
Twirl – Since we will be using Play, we will be using Twirl for the template engine. No point in redesigning the wheel.
Bootstrap – Frontend is NOT by strong point. I will be sticking with Bootstrap since I have a bit of experience with it. It should be also fairly easy to use it in Twirl
jQuery – Better than Vanilla JS.
Kafka – We will be needing a message broker. An alternative would be RabbitMQ. Both will do just fine
Key Features and Functionalities
Before we start designing the actual system, let us look at what we need the system to actually do. This will help us understand our end goal, will help us structure the service properly and will be used as a base for the other articles. As with any major platform, we can’t go straight into coding, but need to first plan. For both familiar with Agile methodology, consider these our main user stories or Epics that need to be done prior to the platform being available.
User Management – Users need to be able to register on the platform. We will also need administrative users that have higher degree of access and can delete content, ban other users or edit descriptions. Here let’s also include the public profile for the users as well as actions that the user can take related to his account.
Posting – Being a social network, users should be able to post messages on the platform. For simplicity, we won’t be dealing with pictures or videos, but only text messages. Posts can also be pinned on top
Socializing – Users can follow other users, comment on other peoples posts, like or re-post (re-tweet) a message.
Scalability – The platform should be properly distributed so that it can handle thousands of active users at the same time and from all around the globe with minimal delay. As a result, the system should have scalability in mind.
Resiliency – The system should be resilient to failure and should properly handle errors or downtime for one of the servers. Furthermore, data integrity is important.
Other Considerations
The articles in the series can and will evolve over time. As I progress in writing them I will discover features that are needed but were not initially included or problems in the initial design. As the design progresses, changes will be made to adjust for the new information and new requirements.
Furthermore, there are other ways of designing such a complex system and in some aspects there may even be better ones. This is how I would do it with my current knowledge and understanding.
