See what a policy would have done, before it can block anything

The report

Two weeks of advisory running produce a report addressed to the people who have to decide whether to enforce. It contains, in order:

• Coverage — how much of what the agents did the policy could judge at all. This comes first because every other number in the report applies only to the covered part. A pilot that reports 94% coverage is telling the truth and can have a useful conversation about the remaining 6%.
• What the policy would have refused, each entry with the rule that produced it.
• The cost in human attention, counted as distinct questions rather than raw holds. The same question asked twenty times is one item in an approver's queue, and staffing decisions depend on that difference.
• A worksheet of every refusal for your own people to rule on. Whether a refusal was right is a question about your work, and until those rulings come back the report states no false-positive rate at all.

The report does not claim prevention. Nothing was prevented, and it says so at the top.

How it runs

The agent's credentials move behind the checkpoint and its calls route through it for real, the same as in an enforcing deployment. Each request is resolved, authorized, scoped and gated in the ordinary way, producing a verdict; the gateway records the verdict and lets the action through. Advisory and enforcing deployments run the same code and reach the same verdict for the same input — without that property, "it would have held 27 payments" is not a measurement of anything.

The routing has to be real even while the verdicts are advisory. A setup that only watches an agent still holding its own credentials measures a path nobody will actually use, and its numbers say little about what enforcement would do.

That routing is also what a pilot costs, so it is worth naming: the agent's tools have to be pointed at the checkpoint instead of straight at your systems, and its credentials have to move behind it. How much work that is depends on how the agent holds its tools today — a configuration change where they are declared in one place, honest integration work where credentials are scattered through its code. Nothing else about the agent changes.

One exception is disclosed: the operator's kill switch keeps working. A watch-only pilot whose operator cannot stop the agent is not safe to run. If the switch is ever used, the report says so on its own line.

Turning it on takes two keys

The policy document declares enforcement: advisory, and the deployment configuration must separately permit advisory for that agent. If the two disagree, the policy does not load at all, rather than warning or silently downgrading in either direction. This is deliberate: a single YAML line that could quietly turn enforcement off would make the whole control irrelevant, and an operator who is wrong about which mode they are in makes exactly the wrong decision about what to route.

What the numbers cannot say

• They measure attempts under no feedback. Under enforcement the agent would have seen each refusal and changed its next move. The counts are the truth about those two weeks, not a simulation of an enforced fortnight.
• A refusal is not an averted loss. Some of what the policy would have stopped, your existing systems would have stopped too. Until your outcome data is joined to the record, every count is an upper bound on what the policy adds, and the report says so wherever the claim would otherwise be made.
• Two weeks covers one month-end and one payment run, and it measures the specific policy written for the pilot, not the product in general.
• Actions that bypass the checkpoint are invisible, exactly as they would be under enforcement. The coverage section exists to surface how much that is.

From advisory to enforcement

The report ends with a ranked conversion path: rules ready to enforce, rules that need tuning first, rules with no evidence either way yet, and the coverage work to do alongside. Because the record is complete and the decision is deterministic, an amended rule can be re-run against the same fortnight before anything is switched on. Advisory keeps running throughout, so every change is measured against live traffic the same way the first report was.