The Rs 14 Lakh Production Outage We Prevented with Blast Radius Analysis

The One-Line Change That Was Not Simple

Vikram Rao had been at the company for three years. He knew the codebase well, or thought he did. The change was simple: rename a parameter in formatCurrencyAmount() to align with their new style guide. The function looked self-contained. A quick grep showed it was called in about 10 places. He reviewed those 10 callsites. They all looked fine.

Before pushing, almost as an afterthought, he ran ArchSetu blast radius analysis on formatCurrencyAmount.

What the Blast Radius Showed

The grep had found direct calls. Blast radius found the full chain. formatCurrencyAmount was called by 11 functions. Those 11 functions were called by 34 more. Those 34 were called by 49 more. Total affected functions across 3 hops: 94 functions in 31 files.

Three of the 94 affected functions were in the PDF generation service. Two were in the GST calculation module. One was in the reconciliation cron job that ran every night at 2 AM and touched every customer's invoice records.

The Change in Engineering Culture

After Vikram's near-miss, the company made blast radius analysis mandatory for any change to a function with more than 5 callers. They added it to their PR template: paste the ArchSetu blast radius output for any utility function change.

In the three months since, they have caught 7 more high-risk changes before they reached production. None of them would have been caught by code review alone, because the risk was in the indirect dependency chain, not in the code being changed.

The Economics of Blast Radius Checks

A blast radius check takes 5 seconds. A production outage for a B2B invoice platform takes 2-4 days to fully recover from: customer communication, root cause analysis, fixes, post-mortem, and the business relationship damage. The math on mandatory blast radius checks is straightforward.