System Design Interview Questions and Answers

Vertical scaling (scale up) means giving one machine more CPU, RAM or disk. Horizontal scaling (scale out) means adding more machines and spreading the load across them.

Practical approach:

• Start vertical for simplicity (one bigger database server goes a long way).
• Make the app servers stateless (keep sessions in Redis or in tokens). Add more servers behind a load balancer.
• Scale the database with read replicas first, then caching, then sharding only when needed.

Mention that horizontal scaling adds complexity: consistency, distributed debugging, and network failures.

What they are checking: Whether you know the pros and cons and a sensible order of scaling

A load balancer distributes incoming requests across multiple servers to improve throughput, availability and fault tolerance. It health-checks backends and stops sending traffic to failed ones.

Common algorithms:

• Round robin: rotate through servers in order.
• Weighted round robin: bigger servers get more requests.
• Least connections: send to the server with the fewest active connections (good for long requests).
• Least response time: pick the fastest responder.
• IP hash / consistent hashing: the same client or key goes to the same server (sticky behaviour, cache locality).

Types:

• Layer 4 (TCP/UDP): fast, routes by IP and port.
• Layer 7 (HTTP): routes by path, host or header, terminates TLS, can do redirects.

Examples: NGINX, HAProxy, AWS ALB/NLB. The load balancer must not be a single point of failure. Run a pair of them, or use a managed service.

What they are checking: Algorithm choice and L4 vs L7

Caching stores frequently used data in a fast layer (memory) to reduce latency and database load.

Where: browser, CDN, application memory, distributed cache (Redis, Memcached), database buffer.

Strategies:

• Cache-aside (lazy loading): the app checks the cache; on a miss it reads the database and fills the cache. Most common.
• Read-through: the cache itself loads data on a miss.
• Write-through: writes go to the cache and database together; consistent but slower writes.
• Write-back (write-behind): write to cache first, flush to the database later; fast but risks data loss.
• Write-around: write to the database only, cache fills on read.

Eviction: LRU, LFU, TTL expiry.

Challenges: old data, cache stampede (many misses at once, fix with locking or by merging requests), and removing old entries, which is famously hard. Always set a TTL as a safety net.

What they are checking: Strategy names and pros and cons

In a distributed data store, during a network partition you can guarantee only two of three properties:

• Consistency: every read sees the latest write.
• Availability: every request gets a (non-error) response.
• Partition tolerance: the system keeps working when messages between nodes are lost.

Network splits (partitions) happen in real networks. So the real choice is between C and A when a split happens:

• CP systems refuse or delay some requests to stay consistent (ZooKeeper, etcd, HBase, MongoDB with majority writes).
• AP systems keep answering but may return stale data and reconcile later (Cassandra, DynamoDB default, Couchbase).

Without a partition, you can have both; the PACELC extension adds that, even without a split, you still trade latency against consistency.

Example: a bank balance prefers consistency. A social feed like count prefers availability and eventual consistency.

What they are checking: The correct view: the choice is C vs A under partition

SQL (relational) databases use tables, a fixed schema, joins and ACID transactions. NoSQL includes document (MongoDB), key-value (Redis, DynamoDB), wide-column (Cassandra) and graph (Neo4j) stores. They usually have flexible schemas and are easier to spread over many servers.

Choose SQL when:

• Data is relational and integrity matters (payments, inventory).
• You need complex queries, joins and reporting.
• Strong consistency is required.

Choose NoSQL when:

• You need massive scale or write throughput across many nodes.
• Access is simple by key, or the data is nested and schema changes often.
• Eventual consistency is acceptable.

Often the answer is both (polyglot persistence): PostgreSQL for orders, Redis for sessions and cache, Elasticsearch for search, object storage for files. Do not pick NoSQL just because it sounds scalable; a single well-indexed PostgreSQL instance handles a large workload.

What they are checking: Reasoning from requirements

A message queue is a buffer between a producer and a consumer. The producer publishes a message and moves on; consumers process it asynchronously. Examples: RabbitMQ, Amazon SQS, Kafka (log-based streaming), Redis Streams.

Benefits:

• Decoupling: services do not need to be up at the same time.
• Load leveling: absorb traffic spikes and process at a steady pace.
• Reliability: messages persist and are retried after failures.
• Scalability: add more consumers to drain faster.
• Offload slow work (emails, image processing) from the request path.

Things to design for:

• Delivery guarantees: at-most-once, at-least-once (most common), exactly-once (hard). Make consumers idempotent.
• Ordering, which is only guaranteed per partition or queue.
• Dead-letter queue for poison messages.
• Backpressure and monitoring of queue depth.

Queue (one consumer gets each message) differs from pub/sub (every subscriber gets a copy).

What they are checking: Guarantees and idempotency

A monolith is one deployable application containing all features. Microservices split the system into small, independently deployable services, each owning its data and a business capability.

Microservices cost you network calls, eventual consistency, distributed tracing, versioning and a platform team. They pay off with large teams and independent scaling needs.

Good advice: start with a well-structured (modular) monolith, and extract services along clear boundaries when a real need appears. Many "microservice" migrations create a distributed monolith that is worse than before.

What they are checking: Judgement, not hype

Replication copies the same data to multiple nodes. Sharding splits different data across nodes.

A Content Delivery Network (CDN) is a group of edge servers in many places. They keep copies of content close to users.

Eventual consistency means that, if no new updates are made, all copies end up with the same value after some time. Reads in the meantime may return stale data.

Latency is the time one request takes from start to finish (milliseconds). Throughput is how many requests the system handles per unit of time (requests per second).

A stateless service keeps no per-client data between requests, so any instance can serve any request. A stateful service stores data in its own memory or disk that later requests depend on.

A forward proxy sits in front of clients. It sends requests to the internet for them (hides who the client is, filters content, caches). A reverse proxy sits in front of servers and receives requests on their behalf (hides backend servers).

Synchronous communication (REST, gRPC) means the caller waits for the response. It is simple and immediate, but it couples the services in time: if the called service is slow or down, the caller suffers. Long chains of calls add up delay and failures.

They overlap, and one product (NGINX, Envoy) can play all three roles, but the focus differs.

ACID (Atomicity, Consistency, Isolation, Durability) gives strict transactional guarantees, typical of relational databases.

A cache holds a temporary, disposable copy of data in fast memory to speed up reads. A database is the durable source of truth.

A process has its own memory space; a thread is a lightweight unit of execution inside a process sharing its memory.

• SLI (Service Level Indicator): a measured number for service quality, for example the percentage of requests that succeed or the p99 latency.
• SLO (Service Level Objective): the internal target for an SLI, for example 99.9 percent of requests succeed over 30 days.
• SLA (Service Level Agreement): the contract with customers that states consequences, such as credits, if the target is missed. It is usually looser than the SLO.

Sharding splits one dataset across many database servers (shards). Each shard holds some of the rows. So storage and traffic can grow by adding servers.

Strategies:

• Range-based: by id or date range. Simple, but hot spots (latest range gets all writes).
• Hash-based: hash(user_id) mod N. Even spread, but resharding moves much data; consistent hashing reduces that.
• Directory-based: a lookup service maps keys to shards. Flexible, extra component.

Challenges:

• Choosing a shard key with high cardinality and even distribution; a poor key causes hot shards.
• Cross-shard joins and transactions become hard or impossible.
• Resharding when adding nodes.
• Extra work to run it: backups, schema changes, monitoring per shard.
• Global unique ids need a generator (Snowflake style, UUIDs).

Before sharding, exhaust cheaper options: indexes, caching, read replicas, partitioning, archiving old data.

What they are checking: Shard key reasoning

Assume any network call can be slow or fail, and prevent one failure from cascading.

Availability is the share of time the system works. 99.9 percent allows about 8.8 hours of downtime a year; 99.99 percent allows about 52 minutes; 99.999 percent about 5 minutes.

Consistent hashing maps both keys and servers onto a circular hash space (ring). A key belongs to the first server found clockwise from its hash.

Options and pros and cons:

The three pillars of observability answer different questions.

A read replica is a copy of the primary database. It gets changes through replication and answers read-only queries. You use it to handle many reads, move reports and analytics off the primary, and improve availability or speed in other regions.

Inside one database a join is cheap and consistent. Across services or shards a join means network calls, partial failures and no shared transaction.

Match the mechanism to the interaction.

Primary-replica (single leader): all writes go to one primary, which streams changes to replicas that serve reads. Simple and consistent for writes. Limits: write throughput is capped by one node, and failover needs promotion (brief downtime or lag).

For each failure, decide if it is temporary or permanent. Make sure no failure blocks the partition forever or loses a message silently.

Requirements: make a short link for a long URL, redirect fast, many more reads than writes (about 100 to 1), optional expiry, and click analytics.

Capacity example: 100 million new URLs per month, about 40 writes/s and 4,000 reads/s; 7 characters of base62 give 62^7, about 3.5 trillion codes.

API:

• POST /urls with {long_url} returns {short_url}.
• GET /{code} returns 301 or 302 to the long URL. Use 302 if you want every click tracked. Use 301 when browsers may cache it.

Key generation:

• Counter or ID generator encoded in base62: unique, no collisions, but predictable.
• Random 7 characters plus a uniqueness check (retry on collision).
• Pre-generated key service handing out unused keys in batches to app servers.

Storage: a key-value table code -> long_url, created_at, expiry, user_id. NoSQL (DynamoDB, Cassandra) or sharded SQL by code works.

Scaling reads: cache popular codes in Redis (LRU), put a CDN or edge in front, and run stateless app servers behind a load balancer.

Extras: rate limits, checks for abuse and malware, click analytics done later through a queue, custom aliases, and removing expired links.

What they are checking: Structured approach: requirements, API, storage, scale

Requirements: 1:1 and group chat, real-time delivery, message history, online status, read receipts, media sharing, and millions of users at the same time.

Core design:

• Connection: clients keep a WebSocket connection to a chat gateway. Gateways are stateless apart from the connection, kept behind a load balancer.
• Routing: a session service (Redis) maps user_id -> gateway. To send, the sender's gateway looks up the recipient's gateway and forwards the message.
• Offline users: store the message and send a push notification (APNs/FCM); deliver on reconnect.
• Persistence: save messages in a store split by conversation (Cassandra or DynamoDB). Key it by (conversation_id, message_id) with ids ordered by time, for ordering and paging.
• Ordering and delivery: per-conversation sequence numbers; at-least-once delivery with client acks and deduplication by message id.
• Groups: fan-out on write for small groups, via a queue (Kafka) for large ones.
• Media: upload to object storage (S3) with pre-signed URLs, send only the link in the message, and serve it through a CDN.
• Presence: heartbeats stored in Redis with TTL.
• Security: TLS, optionally end-to-end encryption.

Scale by sharding on conversation or user id and adding gateway instances.

What they are checking: Real-time architecture and delivery guarantees

Goal: limit each client to N requests per time window, with low cost and counters shared across servers.

Where: at the API gateway or middleware, before expensive work.

Requirements: multiple channels, millions of notifications, user preferences, reliable delivery, no duplicates, rate limits per provider.

Work from measurement to action.

Avoid two-phase commit across services (slow, fragile). Use the Saga pattern: a sequence of local transactions where each step has a compensating action.

Protect the critical path and shed the rest.

The key decision is fan-out on write versus fan-out on read.

Split by capability, keep the checkout path correct and the browsing path fast.

Low-level design questions test modelling, not scale. Follow a repeatable path.

Do not run LIKE '%term%' over a large table. Use a search index.

Requirements: upload, download, sync across devices, sharing with permissions, large files, versioning.

Goals: sub-millisecond reads, high availability, horizontal scale, bounded memory.

There are three common models, and the choice depends on isolation needs, number of tenants and cost.

Put authentication in one identity service (or a managed provider such as Keycloak, Auth0 or Cognito). Every other service trusts the tokens it issues. Services do not keep their own passwords.