Design a Chat System

Step 1 of 4: Understand the problem and establish design scope #

Goal: lock in requirements before designing.

Requirements:

1. Both 1-on-1 and group chat
2. Mobile + web
3. 50 million DAU
4. Group max: 100 members
5. Features: 1-on-1 chat, group chat, online presence indicator
6. Text only, max 100,000 chars per message
7. No end-to-end encryption (discuss if time allows)
8. Chat history stored forever
9. Multiple device support (same account logged into multiple devices)

Step 2 of 4: Propose high-level design and get buy-in #

Goal: pick the protocol, then sketch the architecture.

Communication protocols #

Sender side: HTTP with keep-alive (persistent connections, fewer TCP handshakes).

Receiver side: three approaches:

Final choice: WebSocket for both sending and receiving. Simplifies design. HTTP used for everything else (signup, login, profile).

High-level architecture #

Three categories:

1. Stateless services (behind load balancer): login, signup, user profile. Can be monolithic or microservices. Service discovery recommends the best chat server for a client (based on geo location, server capacity)
2. Stateful service: Chat service: each client maintains a persistent WebSocket connection to a chat server. Client stays connected to the same server while available
3. Third-party integration: push notifications for offline messages

Scalability note #

1M concurrent users × 10KB per connection ≈ 10GB memory: could fit on one server in theory, but a single point of failure is unacceptable.

Adjusted high-level design #

1. Chat servers: message sending/receiving via WebSocket
2. Presence servers: online/offline status
3. API servers: login, signup, profile
4. Notification servers: push notifications
5. Key-value store: chat history persistence

Storage #

Generic data (user profile, settings, friends list) → relational database with replication/sharding.

Chat history data: characteristics:

1. Enormous volume: Facebook Messenger + WhatsApp process 60 billion messages/day
2. Mostly recent chats accessed; old chat random access (search, mentions) must still work
3. Read:write ratio ≈ 1:1 for 1-on-1 chat

Why key-value store for messages:

1. Easy horizontal scaling
2. Very low latency
3. Relational DBs struggle with long-tail data; indexes grow large, random access expensive
4. Proven: Facebook Messenger uses HBase, Discord uses Cassandra

Data models #

1-on-1 message table: primary key = message_id (determines sequence; created_at insufficient: two messages can have the same timestamp).

Group chat message table: composite primary key = (channel_id, message_id). channel_id is the partition key.

Message ID generation:

1. Must be unique and sortable by time (newer = higher ID)
2. Options: Snowflake (global 64-bit), or local sequence number generator (unique within a channel, easier to implement, sufficient for message ordering)

Step 3 of 4: Design deep dive #

Goal: discovery, message flows, multi-device sync, presence.

Service discovery #

Apache ZooKeeper registers available chat servers and picks the best one for a client.

1. User A logs in → 2. Load balancer → API servers → 3. Authenticate → service discovery picks best chat server → 4. User A connects via WebSocket

Message flows #

1-on-1 chat flow:

1. User A sends message to Chat server 1
2. Chat server 1 obtains a message ID from the ID generator
3. Message sent to message sync queue
4. Stored in KV store 5a. User B online → forwarded to Chat server 2 → delivered 5b. User B offline → push notification sent

Multi-device sync:

Each device maintains cur_max_message_id. New messages: recipient_id == current_user_id AND message_id > cur_max_message_id. Each device syncs independently.

Small group chat flow:

Message from User A is copied to each group member's message sync queue (each recipient has an inbox). Good for small groups (WeChat caps at 500). For large groups, storing a copy per member is too expensive.

Online presence #

Login: WebSocket connected → status saved as "online" + last_active_at in KV store.

Logout: status changed to "offline" in KV store.

Disconnection: heartbeat mechanism: client sends a heartbeat every 5s. If no heartbeat for x seconds (e.g. 30s), mark offline. Avoids flickering from brief disconnections.

Online status fanout: pub/sub model: each friend pair has a channel. Status change published to all friend channels. Friends subscribed. Works well for small groups (WeChat caps at 500). For large groups: fetch status on-demand (enter group, manual refresh).

Step 4 of 4: Wrap up #

Additional talking points:

1. Media files: compression, cloud storage, thumbnails
2. End-to-end encryption: only sender and recipient can read messages (WhatsApp model)
3. Client-side caching: reduce data transfer
4. Geo-distributed network: cache user data near users for fast load times (Slack's Flannel)
5. Error handling: chat server failure → ZooKeeper provides a new server; message retry via queues
6. Message resent mechanism: retry + queueing