Metric Definition Auditor
Finds where one metric name means three different things
5 of 8 metric definitions need attention.
8 metric definitions checked. 5 flagged. The largest gap is 606 days.
Start with Conversion rate. Growth has the most flagged.
Whether a formula is the right one for the business question is a judgement, not in the data.
Why rows were flagged
- Not reviewed in 12 months33%
- No owner22%
- Formula not documented22%
- Same name defined twice22%
Value flagged by Team
- Growth2
- Finance3
Items to act on, worst first
| Item | Team | Value | Days since the review | Why |
|---|---|---|---|---|
| Conversion rate | Growth | 11 | 0 | No owner, Formula not documented, Not reviewed in 12 months |
| Churn rate | Growth | 9 | 536 | No owner, Not reviewed in 12 months, Same name defined twice |
| Revenue per user | Finance | 7 | 0 | Formula not documented |
| Gross margin | Finance | 6 | 606 | Not reviewed in 12 months |
| Churn rate | Finance | 5 | 0 | Same name defined twice |
What to do this week
- Resolve Conversion rate first: it is the most urgent on the list.
- Fix the data or the status on every flagged row, then re-run.
- Re-run this check every time the list changes, before it reaches a decision.
Do
- Fix the highest-risk item first, then run the check again on fresh data.
Don't
- Don't act on one row alone: open the source record first.
How to improve
- Add more detail per row (owner, date, source) so the next report can trace each item.
- Add a second period of data so the next report can show whether things are improving.
Demo
Overview
The most damaging failure in business intelligence is not a broken pipeline — those get noticed and fixed. It is that "active customer" means one thing in the finance dashboard, another in the product dashboard and a third in the board pack, all labelled the same way.
This happens without anyone doing anything wrong. Each dashboard was built by a different person for a different question, and each definition was reasonable for that question. Nobody wrote it down because at the time it was obvious. Two years later the person has gone and the definition lives only in the SQL.
The cost lands in meetings. Two teams argue for an hour about whose number is right when both are correct and measuring different things. The more expensive version is when nobody notices, and a decision is taken on a number that does not mean what the room thinks it means.
Governance tools sell catalogues you have to populate, which is why they end up half-populated. This does the opposite: it reads the queries that already exist, extracts what each metric actually computes, and reports where the same name means different things and where different names mean the same thing.
It changes no query and publishes no definition. Which definition is correct is a business decision, and it belongs to the people who own the metric.
Benefits
Meetings stop arguing about numbers
And start arguing about the decision.
Decisions on numbers that mean what you think
Which is the expensive version of this problem.
A definition register built from reality
Rather than one nobody populates.
What it does
Reads the definitions that actually run
Not a catalogue somebody was supposed to fill in. The real definition lives in the query, and only there.
Finds one name, several meanings
Each definition was reasonable for the question it was built for. None of them was written down.
And several names, one meaning
Duplicate work maintained separately, drifting apart quietly.
Names the differences that matter
A filter, a date basis, an inclusion rule. It says exactly which clause makes two numbers disagree.
Predicts which reports will clash
So the meeting is prepared rather than derailed.
Publishes no definition
Which one is correct is a business decision owned by the people who own the metric.
What this changes
What this changes about your week
- It takes over
- Establishing what each metric actually computes, across every dashboard.
- Done today by
- A data catalogue somebody was supposed to populate.
- How often
- Never, which is why the meeting argues.
- Typical manual effort
- The real definition lives in the query, and only there.
Why Data & Business Intelligence in particular. 'Active customer' means three different things in three dashboards, all labelled the same. Each was reasonable for its own question and nobody wrote it down, so the definition left when its author did.
Runs unattended
Started by you or by an event, and it finishes on its own. Nothing waits for someone to be at a desk.
The same standard every time
The two-hundredth item is held to the bar the first one was. Consistency is the part people cannot sustain.
It cannot act on its own
Metric Definition Auditor has no path to sending, spending or committing. That limit is why its output is safe to act on.
How it runs
This agent runs server-side through the PROMIVO runtime. Each run is logged step by step and every tool call is permission-checked before it executes.
- 1InputValidated against the agent input schema
- 2Modelclaude-opus-5 with automatic fallback
- 3Toolsextract_structured_data
- 4PermissionsRuns READ actions automatically; anything higher waits for approval
- 5OutputStructured result matching the output schema
- 6LoggingTokens, duration and every step recorded
Requirements
Integrations
No integrations required.
Before you start
- The SQL or model definitions behind your metrics
- Which dashboard or report each feeds
- Metric names as displayed to readers
- Requires the All-Access plan or higher
Use cases
Before a board pack
Whether the numbers agree with the operational dashboards.
Building a semantic layer
What the definitions currently are, before deciding what they should be.
After a data team change
The definitions that left with the person who wrote them.
A disputed number
Exactly which clause makes the two disagree.
Pricing
All-Access
Billed monthly through your PROMIVO subscription. Cancel at any time.
Runs consume your plan allowance for agent executions and tokens. See plan limits
FAQ
Does it decide which definition is right?
No. Both are usually right for their own question. Choosing the canonical one is a business decision, and a tool making it silently is how the wrong one wins.
Does it change our queries or dashboards?
No. It reads them. It has no write access to any warehouse, model or dashboard.
Do we need a data catalogue first?
No — the absence of one is the reason this exists. It works from the query definitions themselves, which is the only place the truth is stored.
Reviews
Reviews open once customers have run this agent.
Need this agent to work differently?
Tell us what to change and our team will scope a customised version for your business.
Customize this agent