CHYP.ai · ORBIT // Cloudy_Day Security Suite

OUTAGE DISSECTOR · CASE FILE · Meltdown Index · vendor post-incident report

Kinesis front-end fleet exceeded OS thread limit

Amazon Web Services resolved M7.2 Severe storm 2020-11-25

Amazon Web Services: Kinesis front-end fleet exceeded OS thread limit. Scope: Regional.

M7.2

Severe storm

How CHYP Magnitude works

  • 17h 8mDurationstart → resolved
  • 5Systems namedfrom the source
  • 1Places named1 on the map
  • 136Dependentscatalogued, inferred
  • 2/4Evidence kindsindependent families
  • 2Updatesofficial revisions

1Scrub the outage

full window
Severityofficial state Nov 25, 5 AMNov 25, 9 AMNov 25, 12 PMNov 25, 3 PMNov 25, 6 PMNov 25, 9 PM Official statusNov 25, 5:15 AM PT · Impact begins (vendor-stated)Nov 25, 10:23 PM PT · Mitigated (vendor-stated)Network & routingnothing in this windowEarly signalsnothing in this windowNews & chatternothing in this windowCoincidence (KEV)nothing in this window ▼ startresolved ▼

Drag or press Play to replay what was public at each moment. Each dot links to its evidence. Lanes are kept separate on purpose.

What happened

Amazon Web Services: Kinesis front-end fleet exceeded OS thread limit. Scope: Regional.

What broke

Amazon Kinesis, Amazon Cognito, Amazon CloudWatch, AWS Auto Scaling and 1 more.

Who was hit

136 catalogued services run on Amazon Web Services (inferred). Hit: Regional.

How it recovered

Resolved after 17h 8m.

4Failure-mode fingerprint

RetriesControl planeConfig pushCascadeCapacity
  • Capacity exhaustionvendor post-incident report (Meltdown Index)
  • Control-plane couplingvendor post-incident report (Meltdown Index)
  • Dependency cascadevendor post-incident report (Meltdown Index)
  • Retry amplificationno sign
  • Config propagationno sign

Failure modes as stated in the vendor's own post-incident report (Meltdown Index).

5Evidence matrix

Official statusOfficial status · Nov 24, 11:15 PM PT: 0 itemsOfficial status · Nov 25, 12:12 AM PT: 0 itemsOfficial status · Nov 25, 1:10 AM PT: 0 itemsOfficial status · Nov 25, 2:08 AM PT: 0 itemsOfficial status · Nov 25, 3:06 AM PT: 0 itemsOfficial status · Nov 25, 4:04 AM PT: 0 itemsOfficial status · Nov 25, 5:02 AM PT: 1 itemOfficial status · Nov 25, 5:59 AM PT: 0 itemsOfficial status · Nov 25, 6:57 AM PT: 0 itemsOfficial status · Nov 25, 7:55 AM PT: 0 itemsOfficial status · Nov 25, 8:53 AM PT: 0 itemsOfficial status · Nov 25, 9:51 AM PT: 0 itemsOfficial status · Nov 25, 10:49 AM PT: 0 itemsOfficial status · Nov 25, 11:46 AM PT: 0 itemsOfficial status · Nov 25, 12:44 PM PT: 0 itemsOfficial status · Nov 25, 1:42 PM PT: 0 itemsOfficial status · Nov 25, 2:40 PM PT: 0 itemsOfficial status · Nov 25, 3:38 PM PT: 0 itemsOfficial status · Nov 25, 4:36 PM PT: 0 itemsOfficial status · Nov 25, 5:33 PM PT: 0 itemsOfficial status · Nov 25, 6:31 PM PT: 0 itemsOfficial status · Nov 25, 7:29 PM PT: 0 itemsOfficial status · Nov 25, 8:27 PM PT: 0 itemsOfficial status · Nov 25, 9:25 PM PT: 0 itemsNetwork & routingNetwork & routing · Nov 24, 11:15 PM PT: 0 itemsNetwork & routing · Nov 25, 12:12 AM PT: 0 itemsNetwork & routing · Nov 25, 1:10 AM PT: 0 itemsNetwork & routing · Nov 25, 2:08 AM PT: 0 itemsNetwork & routing · Nov 25, 3:06 AM PT: 0 itemsNetwork & routing · Nov 25, 4:04 AM PT: 0 itemsNetwork & routing · Nov 25, 5:02 AM PT: 0 itemsNetwork & routing · Nov 25, 5:59 AM PT: 0 itemsNetwork & routing · Nov 25, 6:57 AM PT: 0 itemsNetwork & routing · Nov 25, 7:55 AM PT: 0 itemsNetwork & routing · Nov 25, 8:53 AM PT: 0 itemsNetwork & routing · Nov 25, 9:51 AM PT: 0 itemsNetwork & routing · Nov 25, 10:49 AM PT: 0 itemsNetwork & routing · Nov 25, 11:46 AM PT: 0 itemsNetwork & routing · Nov 25, 12:44 PM PT: 0 itemsNetwork & routing · Nov 25, 1:42 PM PT: 0 itemsNetwork & routing · Nov 25, 2:40 PM PT: 0 itemsNetwork & routing · Nov 25, 3:38 PM PT: 0 itemsNetwork & routing · Nov 25, 4:36 PM PT: 0 itemsNetwork & routing · Nov 25, 5:33 PM PT: 0 itemsNetwork & routing · Nov 25, 6:31 PM PT: 0 itemsNetwork & routing · Nov 25, 7:29 PM PT: 0 itemsNetwork & routing · Nov 25, 8:27 PM PT: 0 itemsNetwork & routing · Nov 25, 9:25 PM PT: 0 itemsEarly signalsEarly signals · Nov 24, 11:15 PM PT: 0 itemsEarly signals · Nov 25, 12:12 AM PT: 0 itemsEarly signals · Nov 25, 1:10 AM PT: 0 itemsEarly signals · Nov 25, 2:08 AM PT: 0 itemsEarly signals · Nov 25, 3:06 AM PT: 0 itemsEarly signals · Nov 25, 4:04 AM PT: 0 itemsEarly signals · Nov 25, 5:02 AM PT: 0 itemsEarly signals · Nov 25, 5:59 AM PT: 0 itemsEarly signals · Nov 25, 6:57 AM PT: 0 itemsEarly signals · Nov 25, 7:55 AM PT: 0 itemsEarly signals · Nov 25, 8:53 AM PT: 0 itemsEarly signals · Nov 25, 9:51 AM PT: 0 itemsEarly signals · Nov 25, 10:49 AM PT: 0 itemsEarly signals · Nov 25, 11:46 AM PT: 0 itemsEarly signals · Nov 25, 12:44 PM PT: 0 itemsEarly signals · Nov 25, 1:42 PM PT: 0 itemsEarly signals · Nov 25, 2:40 PM PT: 0 itemsEarly signals · Nov 25, 3:38 PM PT: 0 itemsEarly signals · Nov 25, 4:36 PM PT: 0 itemsEarly signals · Nov 25, 5:33 PM PT: 0 itemsEarly signals · Nov 25, 6:31 PM PT: 0 itemsEarly signals · Nov 25, 7:29 PM PT: 0 itemsEarly signals · Nov 25, 8:27 PM PT: 0 itemsEarly signals · Nov 25, 9:25 PM PT: 0 itemsNews & chatterNews & chatter · Nov 24, 11:15 PM PT: 0 itemsNews & chatter · Nov 25, 12:12 AM PT: 0 itemsNews & chatter · Nov 25, 1:10 AM PT: 0 itemsNews & chatter · Nov 25, 2:08 AM PT: 0 itemsNews & chatter · Nov 25, 3:06 AM PT: 0 itemsNews & chatter · Nov 25, 4:04 AM PT: 0 itemsNews & chatter · Nov 25, 5:02 AM PT: 0 itemsNews & chatter · Nov 25, 5:59 AM PT: 0 itemsNews & chatter · Nov 25, 6:57 AM PT: 0 itemsNews & chatter · Nov 25, 7:55 AM PT: 0 itemsNews & chatter · Nov 25, 8:53 AM PT: 0 itemsNews & chatter · Nov 25, 9:51 AM PT: 0 itemsNews & chatter · Nov 25, 10:49 AM PT: 0 itemsNews & chatter · Nov 25, 11:46 AM PT: 0 itemsNews & chatter · Nov 25, 12:44 PM PT: 0 itemsNews & chatter · Nov 25, 1:42 PM PT: 0 itemsNews & chatter · Nov 25, 2:40 PM PT: 0 itemsNews & chatter · Nov 25, 3:38 PM PT: 0 itemsNews & chatter · Nov 25, 4:36 PM PT: 0 itemsNews & chatter · Nov 25, 5:33 PM PT: 0 itemsNews & chatter · Nov 25, 6:31 PM PT: 0 itemsNews & chatter · Nov 25, 7:29 PM PT: 0 itemsNews & chatter · Nov 25, 8:27 PM PT: 0 itemsNews & chatter · Nov 25, 9:25 PM PT: 0 itemsCoincidence (KEV)Coincidence (KEV) · Nov 24, 11:15 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 12:12 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 1:10 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 2:08 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 3:06 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 4:04 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 5:02 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 5:59 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 6:57 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 7:55 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 8:53 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 9:51 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 10:49 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 11:46 AM PT: 0 itemsCoincidence (KEV) · Nov 25, 12:44 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 1:42 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 2:40 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 3:38 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 4:36 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 5:33 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 6:31 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 7:29 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 8:27 PM PT: 0 itemsCoincidence (KEV) · Nov 25, 9:25 PM PT: 0 itemsNov 24, 11:15 PM PTNov 25, 10:23 PM PT

Correlation is not causation

Five independent lanes across the window. Brighter = stronger item. Items landing in the same column are coincidence unless the provider says otherwise.

6In the vendor's words

The new capacity had caused all of the servers in the fleet to exceed the maximum number of threads allowed by an operating system configuration. As this limit was being exceeded, cache construction was failing to complete and front-end servers were ending up with useless shard-maps that left them unable to route requests to back-end clusters.

Duration is from first alarms at 5:15 AM PST to “Kinesis fully returned to normal at 10:23 PM PST,” per the PES. Capacity addition that triggered the event began at 2:44 AM PST.

Verbatim excerpt from Summary of the Amazon Kinesis Event in the Northern Virginia (US-EAST-1) Region.

7Historical twins

Nearest matches in the Control-Plane Meltdown Index (1.0.0, vendor-sourced, tracked since 2024 with earlier landmark cases). Similarity = provider + failure modes + scope.

Cloudy_Day take

  • 136 catalogued services lean on one provider. That's concentration risk, and it's the real headline.
  • Longer than 18 of 19 indexed control-plane meltdowns with a stated duration.

Automated take: rules written by Cloudy_Day, applied to public data. Not a human verdict on this incident, not a root-cause finding, not affiliated with Amazon Web Services.

8Dissection capsule

Share card for this dissection

Everything above as one shareable, citable capsule with a content hash.

JSON capsuleMarkdownShare card (SVG)

Open Amazon Web Services in the War Room →Status page →

Embed this card
<a href="https://www.chyp.ai/dissect/m/aws-kinesis-2020-11-25/"><img src="https://www.chyp.ai/dissect/m/aws-kinesis-2020-11-25/card.svg" alt="Kinesis front-end fleet exceeded OS thread limit · ORBIT Outage Dissector" width="600"></a>

Sources & limits

  • Taken from the vendor's own post-incident report; quotes are verbatim excerpts.
  • Dependents are today's inferred catalogue, not who was affected on the day.
  • Correlation is not causation. Lanes are kept apart so overlap never reads as cause.
  • ORBIT is passive: it never contacts the provider or probes its systems. Independent researcher, not affiliated.