• Refresh the IRCTC page

• Hold their breath

• Spam the “Book Now” button

10:00:01 AM →
Website slows down.
Payments fail.
Twitter explodes.

Why does this happen every single time?
It’s not bad luck.
It’s not always poor engineering.
It’s a classic distributed systems issue called the Thundering Herd Problem.

Let’s break it down.

🧠 What is the Thundering Herd Problem?

The Thundering Herd Problem occurs when:

Many clients/processes wake up at the same time and try to access or regenerate the same resource simultaneously.

This overwhelms the system.

Think of it like a buffalo herd suddenly charging in one direction.

🔍 Why IRCTC Tatkal is a Perfect Example

At 9:59:59 AM:

• Users are refreshing continuously.

At 10:00:00 AM:

• Booking opens.

• Lakhs of requests hit the same booking API.

• All want limited seats.

• Payment gateways get flooded.

The system isn't slow because it's weak.

It’s slow because:
Everyone synchronized their action.

And synchronized traffic is dangerous.

💻 Where This Happens in Real Systems

1. Cache Expiry Disaster

Imagine:

• 50,000 users request /popular-trains

• Data is cached for 1 hour

• Cache expires at 12:00 PM

At 12:00:01:
All 50,000 requests try to fetch fresh data from the database.

Instead of:
✔ 1 database query

You get:
❌ 50,000 database queries

That’s a database meltdown.

Before Expiry:
Users → Cache → Response

After Expiry:
Users → Database → 💥 Overload
Users → Database → 💥 Overload
Users → Database → 💥 Overload

One expired key.
Thousands of simultaneous refreshes.

1. Retry Storm

A service becomes temporarily unavailable.

Clients retry immediately.

Instead of helping recovery,
they increase pressure.

It’s like:

A person faints in a crowd and instead of giving space, everyone gathers around.

1. Flash Sales (Big Billion Days, etc.)

At exactly 12:00:

• “Only 100 units available!”

• 3 lakh users click “Buy Now”

• Inventory service gets slammed

• Payment APIs spike

• Many users see “Out of stock”

Classic herd behavior.

How does It Happen ? (Step by step breakdown)

Let's visualize a typical cache stampede scenario (very common in e-commerce/booking systems).

1. Normal state — Many users request popular data (e.g., train availability). Cache serves most hits quickly.

2. Cache expires — TTL ends for a hot key (e.g., Tatkal quota for a route).

3. The herd charges — Thousands of incoming requests see a cache miss at once.

4. Stampede to backend — All requests query the slow/expensive database simultaneously → DB gets overloaded, latency spikes.

5. Cascading pain — Slower responses → more timeouts/retries → even bigger surge. Servers crash or queue up.

Here's a simple diagram showing the flow:

Why It's So Dangerous

• Performance cliff — System handles 10k req/s normally, but 10k simultaneous req/s kills it.

• Wasted resources — Redundant work (multiple processes recompute the same thing).

• Cascading failures — One slow service drags down the entire chain.

• Poor user experience — Errors, lost sales, frustrated users (hello, endless IRCTC memes).

💥 Impact on System Components

CPU

Sudden thread wake-ups → high context switching → CPU spikes.

Database

Connection pools exhaust → queries queue → timeouts.

Cache

Cache stampede → mass regeneration → eviction pressure.

Latency

P99 latency increases drastically → user experience degrades.

📊 Normal Traffic Spike vs Thundering Herd

Not all traffic spikes are bad.

🟢 Normal Spike

• Gradual increase in users

• System auto-scales

• Load is distributed over time

• Predictable

🔴 Thundering Herd

• Sudden synchronized burst

• All requests hit at the same second

• Resource contention

• High probability of failure

Visual Timeline

Normal Spike:
    /
   /
  /
 /

Thundering Herd:
    |
    |
    |
    |

The difference is synchronization, not just volume.

🛠️How Engineers Solve It

This is where system design gets interesting.

• ✅ 1. Exponential Backoff with Jitter

  Instead of retrying instantly:

  Retry after:

  • 1 second

  • 2 seconds

  • 4 seconds

  • Add randomness

  Randomness prevents synchronization.

  Without randomness:
  Everyone retries together again.

  With randomness:
  Requests spread out.

  ---

  ✅ 2. Single Flight / Cache Locking

  When cache expires:

  • First request regenerates data

  • Others wait

  • Only one DB call happens

  Problem solved.

  ---

  ✅ 3. Staggered Cache Expiry

  Instead of:

  All keys expire at 12:00 PM
  

  Use:

  Expiry = 1 hour ± random(0–5 minutes)
  

  Now refreshes are distributed over time.

  ---

  ✅ 4. Rate Limiting

  Control how many requests per second are allowed.

  This prevents stampedes.

  ---

  🧩 Visualizing the Difference

  Without Protection

  10,000 Users
        ↓
  Database
        ↓
  Crash
  

  With Protection

  10,000 Users
        ↓
  Cache Layer
        ↓
  Single DB Query
        ↓
  Stable System
  

  ---

  🎯 Simple Definition

  The Thundering Herd Problem happens when many clients attempt to access or regenerate the same resource simultaneously, overwhelming the system.

  Or in very simple words:

  Everyone remembered they were hungry at the same second.

  ---

  🚀 Final Thoughts

  Distributed systems don’t just fail because of bugs.

  They fail because:

  • Humans synchronize.

  • Timers synchronize.

  • Retries synchronize.

  And when everything wakes up together…

  You don’t get traffic.

  You get a herd.