Change Failure Analyst
Finds which kind of change keeps breaking production
Teams Payments and others fail changes at a rate well above the 11.1% overall; 6 failed changes are flagged.
72 changes, 8 failed: an overall change failure rate of 11.1%. 2 team(s) sit well above it.
CH-010 is the worst: it took 52 hours to restore service.
A failure needs a clear definition (rollback, hotfix, incident): this uses whatever you marked as failed.
Why rows were flagged
- Team failure rate high67%
- Restore over 24 hours33%
Value flagged by Team
- Payments4
- Data2
Failed changes to review, worst first
| Item | Team | Value | Hours to restore | Why |
|---|---|---|---|---|
| CH-010 | Payments | 1 | 52 | Team failure rate high, Restore over 24 hours |
| CH-005 | Payments | 1 | 30 | Team failure rate high, Restore over 24 hours |
| CH-053 | Data | 1 | 29 | Team failure rate high, Restore over 24 hours |
| CH-013 | Payments | 1 | 6 | Team failure rate high |
| CH-055 | Data | 1 | 4 | Team failure rate high |
| CH-002 | Payments | 1 | 3 | Team failure rate high |
What to do this week
- Review the failed changes on Payments together: the same cause may repeat.
- Start with CH-010: its restore took 52 hours.
- Add a pre-release check for the team with the most failures.
Do
- Count a change as failed when it needs a rollback, hotfix or incident.
Don't
- Don't compare teams without looking at change size and risk.
How to improve
- Add change size and type so the next report can separate big risky changes from routine ones.
- Add the failure cause so the next report can group repeats.
Demo
Overview
Every engineering organisation runs post-incident reviews, and almost none runs the analysis one level up: across many incidents, what kind of change keeps causing them? One postmortem produces a fix for one bug. Fifty read together produce the shape of the changes that break this particular system.
That shape is always specific and usually surprising. Not "we need more testing". Rather: changes touching one service fail four times more often than anything else. Deploys after 4pm on Thursday account for a third of incidents. Every change bundling a schema migration with application code has needed a rollback. The changes that fail are the ones with no linked ticket, because those were the urgent ones.
The industry has settled on four delivery metrics and teams report them without asking what is behind their own numbers. A change failure rate of 18% is a fact; which changes make it 18% is something you can act on.
This agent reads incidents against changes and reports the pattern, with the specific control that would have caught each class — a test, a gate, a migration policy, a deploy window. It is deliberately not a blame tool: findings are about classes of change, never about who wrote them. A per-engineer failure table stops being used honestly within a month, because people start managing the metric instead of the risk.
Benefits
A failure rate you can move
Because you know which changes make it what it is.
Controls that target the real risk
Instead of process added evenly across everything.
Fewer incidents from the same cause
Which is the only measure that matters.
What it does
Reads across incidents, not one at a time
A postmortem fixes one bug. Fifty read together show the shape of what breaks this system.
Finds the change class, not the culprit
Schema migrations bundled with code. Deploys with no linked ticket. Specific, and fixable with a rule.
Names the control that would have caught it
A test, a gate, a migration policy, a deploy window. Not "more testing".
Prices the pattern in downtime
Incidents, minutes and rollbacks per class, so the fix has a case behind it.
Finds the changes with no ticket
The urgent ones. Usually the ones that fail, and the pattern nobody wants to look at.
Never ranks engineers
A per-person failure table gets gamed within a month and stops the honest reporting the analysis depends on.
What this changes
What this changes about your week
- It takes over
- Reading many incidents together to find which changes keep causing them.
- Done today by
- Nobody. Postmortems look at one incident at a time.
- How often
- Quarterly at best.
- Typical manual effort
- Correlating every incident against change metadata across a whole period.
Why Software & Engineering in particular. Teams report a change failure rate without ever asking what is behind their own number. The answer is never 'more testing' — it is a specific class of change, fixable with one rule.
Runs unattended
Started by you or by an event, and it finishes on its own. Nothing waits for someone to be at a desk.
The same standard every time
The two-hundredth item is held to the bar the first one was. Consistency is the part people cannot sustain.
It cannot act on its own
Change Failure Analyst has no path to sending, spending or committing. That limit is why its output is safe to act on.
How it runs
This agent runs server-side through the PROMIVO runtime. Each run is logged step by step and every tool call is permission-checked before it executes.
- 1InputValidated against the agent input schema
- 2Modelclaude-opus-5 with automatic fallback
- 3Toolsextract_structured_data
- 4PermissionsRuns READ actions automatically; anything higher waits for approval
- 5OutputStructured result matching the output schema
- 6LoggingTokens, duration and every step recorded
Requirements
Integrations
No integrations required.
Before you start
- Incident records with timing, severity and the change that caused each, where known
- Change and deploy history with service, type, size and timing
- Requires the All-Access plan or higher
Use cases
Quarterly reliability review
The pattern behind the numbers you already report.
After a bad month
Whether the incidents share a cause.
Before adding process
Where a gate would actually pay for itself.
Platform investment case
Which service earns the rewrite, with evidence.
Pricing
All-Access
Billed monthly through your PROMIVO subscription. Cancel at any time.
Runs consume your plan allowance for agent executions and tokens. See plan limits
FAQ
Does it identify who caused incidents?
No, deliberately. A per-engineer failure table gets gamed within a month and destroys the honest incident reporting this depends on. Findings are about classes of change.
Does it need our source code?
No. It works from incident records and change metadata — service, size, timing, type, rollback status. The pattern is in the metadata, not the diff.
Will it change our pipeline?
No. It has no access to your pipeline or repositories and recommends controls rather than implementing them.
Reviews
Reviews open once customers have run this agent.
Need this agent to work differently?
Tell us what to change and our team will scope a customised version for your business.
Customize this agent