Your Financial Agent Passed Every Eval. Can You Defend Its Decision?

In audit, the opinion is the product, and an opinion is only as good as its basis. Being right isn’t...

In audit, the opinion is the product, and an opinion is only as good as its basis. Being right isn’t enough; you have to show the decision-making process behind it. This raises the question: what does agentic decision making look like?

Let’s Build an Agent

We’ll take a financial audit use case and focus on a specific procedure: testing a company’s bank reconciliation. Companies compare the cash balance on their books to what their bank reports, and any difference should be explained by things like deposits in transit or uncleared checks. Testing this is tedious, high-volume work, which makes it a good fit for an agent.

How might we do this? We’ll probably feed the agent the client’s books, reconciliation, and bank evidence. We can even include the audit methodology in the prompt and use prior-year audits as examples. Finally, we build an eval from previously reviewed audits and ensure a human signs off before anything is finalized. The metrics look great, users are happy, and we ship it.

Then, the audit gets selected for review, and someone asks: why was that sufficient? They don’t care what the evals say, and they don’t even care what the users say. For this specific audit, in this specific instance, why was this decision made?

Four Controls for Defensibility

In light of this, what does it mean to arrive at a “defensible” answer? Audit standards have a good definition for inspiration, paraphrased as “an experienced auditor, new to the work, can follow it.” We can translate this into “can follow the agent” and enforce this through four controls: authorization, execution control, evidence provenance, and evaluation integrity. Additionally, there is always the line where an agent stops and a human makes the final call.

Authorization. An auditor flags a $62,500 check to a payee missing from the vendor master, and a reviewer signs off. A few days later, new statements arrive, which triggers the agent, and it rewrites that row to “no exception.” The signature now vouches for an answer nobody reviewed. Every write was permitted for the agent’s identity, but it should not have been allowed on a signed row. Authorization isn’t just “can this identity call this tool”; it’s whether this action is allowed on this object, in this state, right now.

Execution Control. The agent says a $96,400 check cleared. An auditor reruns the job, and today’s run reads $69,400. Logs tell you the job ran, but they don’t let you reexamine one decision. Instead, break the run into steps (fetch, extract, match, conclude), and record each step’s inputs, outputs, and model version under a stable ID. Then rerun only the contested step on its exact inputs. It isn’t about whether the old agent or the new agent is correct; it’s about knowing why the result is what it is.

Evidence Provenance. The agent cites “January bank statement” for a $61,900 deposit. This ended up being wrong, but where did the citation come from? Many documents could share the same name, or the same document could have been modified over time. Bind every citation to a stable identity (document ID, version, hash), not the model’s text. Then enforce that the evidence attributes the claim depends on, such as relevance, actually match.

Evaluation Integrity. Can we trust the eval? It’s built on last year’s conclusions, not ground truth. There could be misses where the agent was actually right, but the label was wrong. There could also be hits where the agent was actually wrong, but the eval wasn’t updated to reflect changes in methodology. Tag every gold label with the methodology and period it’s valid for, audit both the misses and the hits, and report results by slice.

Human Review. Each failure above had a moment when the agent should have handed off to a human. The agent should stop when it’s about to change work a human signed, when a rerun disagrees with the record, when evidence doesn’t match the item, or when a required rule fails. Additionally, escalations should be easy to verify, not necessarily easy to approve. When one of these fires, ask a specific question, show the conflicting evidence side by side, and record the human’s answer.

Going Beyond Audit

None of this is specific to bank reconciliations or even financial audit. A declined loan, a closed AML alert, a price determination: the same four questions apply every time. Was the action allowed? Can you show what ran? Is each claim bound to the right source? Can you trust the benchmark?

Wherever an agent’s financial decision can be challenged, defensibility, not just correctness, is the bar.

Anirudh is Lead Security Engineer and the founding security hire at Fieldguide, the leading transformation partner for agentic audit and advisory.