System Design · HLD Interview Questions — Hard

Answer: Design a distributed file system managing petabytes of data across thousands of commodity storage servers with high sequential read/write throughput.

• Original Research Paper: https://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf
• Key Design Decisions:
  1. Architecture Topology: Single Master / NameNode managing metadata in memory, paired with thousands of ChunkServers / DataNodes storing raw data chunks.
  2. Large Chunk Size (64MB): Reduces metadata overhead and eliminates frequent master communication.
  3. Replication & Durability: Chunks replicated $3\times$ across different racks with background heartbeat rebalancing.

---

Answer: Design a cloud storage and file synchronization platform supporting desktop/mobile sync, conflict resolution, bandwidth-efficient delta syncing, and offline edits.

• Solution Guide (Text): https://www.geeksforgeeks.org/system-design/design-dropbox-a-system-design-interview-question/
• Deep-Dive Walkthrough (Video): https://www.youtube.com/watch?v=U0xTu6E2CT8
• Key Design Decisions:
  1. Chunking & Content-Addressable Storage (CAS): Split files into 4MB chunks and hash each chunk (SHA-256). Store chunks by their hash in S3 to achieve cross-user deduplication.
  2. Delta Sync: When a 500MB file changes, calculate the binary delta (rsync algorithm) and upload only the modified 4MB chunks rather than re-uploading the entire file.
  3. Sync Engine: Desktop client watches file system changes, maintains a local SQLite database, and pushes metadata events via WebSockets.

---

Answer: Design a distributed web crawler fetching billions of web pages monthly with strict politeness policies, duplicate detection, and robust error handling.

• Solution Walkthrough (Video): https://www.youtube.com/watch?v=BKZxZwUgL3Y
• Key Design Decisions:
  1. URL Frontier: Priority queues for page importance + Politeness queues partitioned by host domain with rate-limiting delays.
  2. Duplicate URL & Content Detection: Bloom Filter for visited URLs ($0.1%$ false positive rate); 64-bit SimHash algorithm for duplicate HTML content detection.
  3. DNS Caching: Custom in-memory DNS caching fleet to eliminate external DNS lookup latency.

---

Answer: Design an omni-channel notification platform sending billions of transactional and marketing alerts across iOS (APNS), Android (FCM), SMS (Twilio), and Email (SendGrid).

• Solution Link: https://algomaster.io/learn/system-design-interviews/design-notification-service
• Key Design Decisions:
  1. Priority Queues: Separate Kafka topics for high-priority transactional alerts (OTP / Payment receipts) vs low-priority marketing blasts.
  2. Deduplication & Rate Limiting: Deduplicate notifications within a 5-minute sliding window using Redis keys (SETNX user:notif:template_id) to prevent spamming users on network retries.
  3. Integration Adapters: Decoupled vendor adapter microservices with circuit breakers and fallback providers.

---

Answer: Design a global distributed rate limiter protecting API backends at scale with sub-millisecond decision latency and atomic synchronization.

• Solution Walkthrough (Video): https://www.youtube.com/watch?v=mhUQe4BKZXs
• Algorithm Deep-Dive: https://blog.algomaster.io/p/rate-limiting-algorithms-explained-with-code
• Key Design Decisions:
  1. Redis + Lua Scripts: Execute token bucket evaluations inside atomic Redis Lua scripts to prevent read-modify-write race conditions.
  2. Sliding Window Counter: Combine current and previous window weights to achieve $O(1)$ memory usage per user while smoothing boundary traffic bursts.
  3. Local Memory + Centralized Sync: Maintain local in-memory token counters at each API Gateway, asynchronously batching sync deltas to Redis every 100ms.

---