If you cannot say how well an agent is doing, you do not know whether it is paying for itself or slowly getting worse. The five numbers to track monthly for any Business Central agent, where to get them, what good looks like, and what to do when they move.
An agent goes live, the first few weeks look good, everyone moves on. Six months later nobody can say whether it's still accurate, whether it's saving time or costing money. It needs measuring, monthly, with the same five numbers every time.
How many items the agent processed: invoices, orders, claims. Compare with total volume to get the share it's handling. Rising share means it's taking on more; falling share means more is being routed around it.
Of the items it processed, how many were right first time: no correction needed to supplier, lines, amounts, accounts or dimensions. Measure by sampling if you can't measure everything: twenty random items a month, checked properly by the owner.
How often a reviewer changed what the agent prepared. Different from accuracy: a reviewer might override for business reasons, not because the agent was wrong. A rising override rate is the early warning of drift: new supplier layouts, new items, changed processes.
Copilot Credits consumed divided by volume, in pounds. Watch it against the business case (see the business case). Longer invoices and more lines cost more; a change in your mix changes the cost.
How long a person spends checking each item. The whole benefit depends on review being quick for routine items. If it creeps up, either the agent is getting less reliable or reviewers are losing trust in it.
There's no universal target, but for a mature, well-scoped invoice agent: a stable or rising share of volume, accuracy in the high 90s for routine items, a low and stable override rate, cost per item in line with the case, and review time measured in seconds to a minute for routine items. Anything trending the wrong way for two months running needs looking at.
Report the five numbers to the agent's owner and to finance every month (see who owns AI). They're also exactly what an auditor will ask for (auditing AI).
