System Design 1

What is system Design?

System design is the process of defining a system’s architecture, components, modules, interfaces, and data to meet specific requirements, essentially creating a detailed blueprint for building software or hardware systems that are efficient, scalable, and reliable

It’s the crucial step between understanding user needs (requirements analysis) and writing code (implementation), focusing on high-level structure, interactions, and non-functional aspects like performance, security, and maintainability, rather than the code itself

How to approach system design?

System Design is extremely practical and there is a structured way to tackle the situations.

1. Understand the problem statement
2. Break it down into components/features (essential). e,g, Social media Feed, Notification
3. Dissect each component into sub component (if required). e.g. Aggregator and Recommendation generator in Feed.
4. Now for each sub component look into
   1. Database and Caching
   2. Scaling & Fault Tolerance
   3. Async processing (Delegation)
   4. Communication

How do you know that you have built a good system?

Every system is “infinitely” buildable and hence knowing when to stop the evolution is important.

1. You broke your system into components
2. Every component has a clear set of responsibilities (Mutually Exclusive)
   1. For each component, you’ve slight technical details figured out
   2. Database and Caching
   3. Scaling & Fault Tolerance
   4. Async processing (Delegation)
   5. Communication
3. Each component (in isolation) is
   1. scalable → horizontally scalable
   2. fault tolerant → plan for recovery in case of a failure
      1. Data and the server it self should auto recover
   3. available → component functions even when some component “fails”

Relational Databases

Databases are most critical component of any system. They make or break a system.

Data is stored & represented in rows and columns. You pick relational databases far relations and acid.

Properties A - Atomicity C - Consistency I - Isolation D - Durability

A - Atomicity

All statements within a transaction takes effect or none of them take effect.

C - Consistency

All data in the database should conform to the rules, no constraints are violated using constraints, cascades, triggers Foreign key checks do not allow you to delete parent if child exists (configurable though) e.g. Cascade

I -Isolation

Concurrent transactions should not interfere with each other. This is achieved through various levels of isolation (Read uncommitted, Read committed, Repeatable read, Serialisable) depending on the needs of the application.

D - Durability

when transaction commits, the changes outlives outages.

Database Isolation Levels

Database isolation levels refer to the degree to which a transaction must be isolated from the data modifications made by any other transactions.

Isolation levels dictate how much one transaction knows about the other

There are four isolation levels defined by the SQL standard. Each level defines the degree to which one transaction must be isolated from the resource modifications made by other transactions.

Read uncommitted

This is the lowest isolation level, and it allows a transaction to read data changes that have not yet been committed by other transactions. This can lead to “dirty reads,” where a transaction reads data that is later rolled back.

e.g. Transaction A reads data modified by transaction B before B commits.

Read committed

This level guarantees that a transaction will only read data that has been committed. It prevents dirty reads but allows for non-repeatable reads, where the same query can return different results at different times.

e.g. Transaction A reads data committed by transaction B.

Repeatable read (Default)

This level ensures that all reads within a transaction see the same data. However, it can still lead to phantom reads, where new rows are added or removed during a transaction.

e.g. Transaction A reads data that was visible at the start of transaction A, even if transaction B has committed changes in between.

What is Phantom read?

Phantom reads occur when one transaction reads a set of rows and then another transaction inserts or deletes rows that match the search criteria of the first transaction.

This causes the first transaction to see rows that did not exist when it started, leading to inconsistent results.

Serializable

This is the highest isolation level. It guarantees that transactions are isolated from each other. In this level, transactions are executed serially, preventing dirty reads, non-repeatable reads, and phantom reads.

This will reduce the throughput of the system and things can be a lot slower

e.g. Transaction A is processed as if no other transactions were running concurrently, Even if they are running it will wait until the rollback or commit of it.

Scaling Databases

1. These techniques are applicable to most databases out there relational and non- relational

Vertical Scaling

• add more CPU, RAM, Disk to the database
• requires downtime during reboot
• gives you ability to handle “scale”, more load

Horizontal Scaling

Read Replicas

• when read: write is 90:10
• You move reads to other database so that “master” is free to do writes
• API Servers should know which DB to connect to get things done. So that it will connect two connections. One with Primary for write and another with replica for read.

Replication

Changes on one database (Master) needs to be sent to Replica to maintain consistency

Synchronous Replication

The main database waits for changes to be written to the replica before acknowledging the commit. Offers higher consistency but can impact performance.

Asynchronous Replication

The main database does not wait for changes to be written to the replica before acknowledging the commit. Offers higher performance but can potentially lead to data inconsistency.

• In this Primary DB can send the data to be committed by replica
• or replica periodically polls the data from master both way it can be implemented.
• All the DB providers have this functionality just need to configure it.

Sharding

What if we get huge amount of write in that case sharding helps

• Partitioning data across multiple databases.
• Because one node cannot handle the data load we split it into multiple exclusive subsets writes on a particular row/document will go to one particular shard.
• API server needs to know whom to connect to, to get things done.
• Note: some databases has a proxy that takes care of routing
• Each shard can have its own replica (if needed)

Advantages of sharding

• Handle large Reads and Writes
• Increase overall storage capacity
• Higher availability

Disadvantages of sharding

• operationally complex
• cross-shard queries expensive

Sharding : Method of distributing data across multiple machines

Partioning: splitting a subset of data within the same instance or multiple.

Overall, a database is sharded while the data is partitioned (split across).

Diagram showing a 100 GB dataset split into 3 partitions distributed across 2 shards (40, 40, 20 GB):

Partioning

1. Horizontal Partitioning (most common)
   1. Some of the table rows put in one DB and other rows in other DB
2. Vertical Partitioning
   1. these 3 tables in DB1 and these 4 tables in another DB.2
   2. It is like moving from monolith to microservice

deciding which one to pick depends on load, use case, and access pattern.

Non - Relational DB

It is a very broad generalisation of databases that are non-relational (MysqL, PostgresqL, etc) But this does not mean all non-relational databases are similar.

Most non-relational databases shard out-of-the box.

Important DB types

Document DB

• MongoDB, Elasticsearch.
• Mostly JSON based
• Support complex queries
• Partial updates to document is possible.
  • Like no need to get whole record and edit and put it back

Key Value Stores

• (Redis, DynamoDB, Acrospike)
• Extremely simple databases | Limitied functionalities
  • GET (K)
  • PUT (K,v)
  • DEL (K)
• Does not support complex queries (aggregations)
• Can be heavily sharded and partitioned
• Use case: profile data, order data, auth data.

Note

You can use relational databases and document DBs as key value db as well. Just restrict yourself to the limited features

Graph DB

• (Neo4j, Neptune, Dgraph)
• What if our graph data structure had a database
• it stores data that are represented as nodes, edges, and relations

• Great for running complex graph algorithms
• Powerful to model Social Networks, Recommendations & Fraud Detection

How to pick right database?

Common Misconception: Picking Non-relational DB because relational Databases do not scale.

Why non-relational DBs scale?

• There are no relations and constraints
• Data is modelled to be sharded

if we relax the above on relational DB, we can scale

• do not use Foreign Key check
• do not use cross shard transaction
• do manual sharding

Does this mean, no OB is different? No!! every single database has some peculiar properties and guarantees and if you need those, you pick that DB

How does this help in designing system?

While designing any system, do not jump to a particular DB right away

1. understand what data you are storing
2. understand how much of data you will be storing
3. understand how you will be accessing the data
4. What kind of queries you will be firing
5. any special feature you expect eg: Expiration

Caching

Caches are anything that helps you avoid an expensive network I/o, disk I/o, or cpu computation

1. API call to get profile information
2. Reading a specific line from a file
3. doing multiple table joins

Store frequently accessed data in a temporary storage location to improve performance.

Caching is nothing but a glorified HashMap.

Caching is technique and caches are the places where we store the data.

e.g. Redis (v popular), Mem cached.

Use case

1. In my system something that is recently published, will be more accessed e.g. Tweet, YT video, Blog, Recent News, so let me cache it.
2. I notices my recommendation system started pushing videos 4 years ago for this channel let me cache it
3. Auth Tokens: Authentication are cached in “cache” to avoid load on database when tokens are checked on every requests.
4. Live Stream: Last 10 min of Live Stream is cached on CDN, as it will be accessed the most.

Populate Cache

• Mostly logically cache sits between API server and Database

Lazy Population

Populate the cache on demand when data is first requested, or when the cache is accessed, and the data is not present. This is also called “cache aside”.

If the data is requested and not found in the cache, the API server fetches the data from the database, caches it, and then returns it to the client. This approach is simple to implement. The main downside is that the first request for a piece of data will always be slow because it requires a database lookup.

e.g. Caching Blogs

Eager Population

Populate the cache proactively, either at the time of data creation or updates, or through a background process that periodically refreshes the cache.

It means pre-loading the caches with the data even before the clients requesting for it. This approach ensures that the data is always available in the cache, resulting in fast reads.

However, it requires more resources, and data in the cache can become stale if not regularly updated.

e.g.

1. Some popular person with 1M follower has tweeted, let’s cache all the tweets of this popular person.
2. People watching cricket, serve match score from cache

Scaling Technique of Cache

• All the things of System Design 1#Scaling Databases applies here as well.

Caching at Different Level

1. Client Side: Cache is the browser
   1. e.g. Browser caching - saving static assets like images, CSS, JS files
2. Application Side: Inside your code/API server itself
   1. e.g. Java, Python: using hash maps to cache the results of frequently called functions, or even the whole result of a API call.
3. CDN Side: CDN (Content Delivery Network). This is CDN caching.
   1. e.g. Caching static assets like images, videos, etc.
4. Database Side: Cache the result of a query, so that subsequent calls to the same queries does not hit the database.
   1. e.g. Query caching can be used in databases.
   2. e.g. Caching indexes, or the results of the queries themselves.
   3. Precomputed columns Instead of computing total posts by users everytime we store ‘total-posts’ as column and update it once in a while. (saves an expensive DB computation)

Select count (*) from posts where user-id = 123
this is an expensive query.

5. Network Side: Using a proxy cache
   1. e.g Reverse proxy caching to cache HTTP responses
   2. Caching on Load Balancer
6. Remote /Distributed Cache e.g. Redis
   1. API server uses it to store frequently served data

Async Processing

• Spinning a virtual machine (EC2) can be done asynchronusly. Cause if we wait for 5 mins till it spins up and http request respond me back after that is not good experience.
• Typically long running tasks will be done in async processing.

Message Queues / Brokers

Brokers help two services applications communicate through messages. We use message brokers whe we want to do something asynchronously.

1. Long- running task
2. Trigger dependent tasks across machines

Example: Video processing

SQS - Simple Queue Service by Amazon. RabbitMQ - is an open-source message broker.

Features

1. Brokers help us connect diferent sub-systems
2. Brokers act as a buffer for the messages/tasks
   1. i.e. Service can add as many things as they want to delegate to system, it will put it in queue and worker will pick it from there, when they are free.
3. Brokers can retain messages far ‘n’ days. It is configurable though.
   1. SQS offers 14 days of maximum days, after that it will expire
4. Brokers can re-queue the message if not deleted
   1. Let say you picked a email sending job from queue, while processing it crashed, and it failed to delete the job from queue. Now after some time queue we automatically reque it. - It’s Visibility Time out on SQS

SQS Core API Actions

These actions facilitate the core message lifecycle: sending, receiving, and deleting messages. 

• CreateQueue: This action is used to create a new message queue. You can specify parameters like QueueName, choose between a Standard or FIFO (First-In-First-Out) queue type
• SendMessage: This is how producer components introduce messages into the queue. A developer provides the queue’s URL and the message body. In FIFO queues, this action might also involve specifying a MessageDeduplicationId for exactly-once processing guarantees.
• ReceiveMessage: Consumer components call this action to retrieve messages from the queue. When a message is received, it isn’t immediately deleted; instead, it becomes temporarily invisible to other consumers for a VisibilityTimeout period, allowing the current consumer time to process it.
• DeleteMessage: After a consumer successfully processes a message, it must call this action using the unique ReceiptHandle provided during the ReceiveMessage call. This is the crucial step that prevents the message from being processed again when the visibility timeout expires.

RabbitMQ in 100 Seconds by Fireship

Message Streams

Apache kafka - Most popular AWS Kinesis is another example

• Similar to message queues, but designed for high-throughput, real-time data streaming to multiple different clients

Apache Kafka allows us to organize data into topics, with each topic having multiple partitions for parallel read/write operations. Consumers read data from these topics, and multiple consumer groups can independently process the same data streams.

The User Service publishes “User Created” events to a Kafka topic. Both the Email Service and Analytics Service subscribe to this topic, enabling real-time email campaigns and analytics updates.

Difference between Kafka vs SQS

• In SQS (Message Broker) - While multiple consumers can poll the queue, any given message is intended to be processed by only one receiver at a time i.e *Single-Message Processing once
• But in case of Apache Kafka - Multiple consumer groups can independently process the same data streams i.e single-message processing by multiple different services.
• SQS is more suited for task queue