Will our platform survive launch day?
Conceptual Case study. This is an editorial illustration of how RAZOOM is designed to approach the problem, written by the RAZOOM team. It was not produced by a working system, and the figures in it are illustrative.
The Case outcome at a glance
The queue saturates at roughly four times peak under current retry settings. Whether to re-architect depends on how likely traffic is to exceed that multiple.
- Next action
- Cap client retries with backoff, then re-run the model; re-architect only if the breaking point stays below five times peak.
- What could change it
- Traffic arriving in sharper bursts than forecast
Original problem
A platform team expects traffic to reach three to five times normal peak during a product launch. They need to decide whether to scale, re-architect the job queue, or accept the risk.
Why the question is hard
- Expected traffic is a range, not a number.
- Bottlenecks interact: a slow queue backs up the API, which drives retries.
- Load tests cover the scenarios someone thought of, not the ones that break the system.
How RAZOOM would reframe it
At what traffic multiple does p99 latency or queue depth breach our service objectives, and what is the cheapest change that moves that point beyond five times peak?
"Will it survive?" becomes a threshold and a cost — both of which can be calculated.
Problem graph
Decision
Scale, re-architect or accept?
Method: Decision theory
Subproblems
Breaking point
Method: Queueing theory
Failure modes
Method: FMEA
Cost of options
Method: Capacity planning
Drivers
Traffic distribution
Method: Forecasting
Feeds: Breaking point
Retry behaviour
Method: Discrete-event simulation
Feeds: Breaking point, Failure modes
Selected methods
- Queueing theoryBreaking point
- Latency under load follows from arrival and service rates.
- Discrete-event simulationRetry behaviour
- Retries create feedback loops that formulas miss.
- FMEAFailure modes
- Ranks what to fix by severity, likelihood and detectability.
- Capacity planningCost of options
- Compares scaling and re-architecture on cost per unit of headroom.
Evidence needed
- Service-time distributions for the job workers
- Historical peak traffic and launch-day comparables
- Client retry and timeout configuration
- Current service objectives
What could change the decision
- Traffic arriving in sharper bursts than forecast
- Retry storms amplifying load beyond the model
- A third-party dependency with a lower limit than assumed
