Governing AI-Built Analytics: In the Loop or On It?

Teams adopted AI building far faster than they decided how to check it. The light, practical gates that keep AI-built reporting trustworthy.

Most teams have rushed to let AI build their analytics and far fewer have decided how to govern it. These are the checkpoints that keep AI-built reporting trustworthy.

We have written more on this via our AI work, and Consolidating multiple Business Central companies in one model takes a closer look at a related part of the picture.

There is a gap opening up in a lot of organisations, and it is a predictable one. Adopting AI to build dashboards, write DAX and accelerate analysis is easy and immediately rewarding, so teams do it fast. Deciding how to check that work is slower and less exciting, so it lags. The result is a growing pile of AI-assisted output and very little agreement on who validates it or how. That gap is where the trouble lives.

The fix is not to slow the building down. It is to be deliberate about where a human steps in.

In the loop, or on the loop.

There are two honest ways to work with an agent. In the loop means you define points where a person has to approve before things move on. On the loop means you set the boundaries up front and then let it run, stepping in only when it goes wrong. Both are legitimate. The mistake is drifting into the second by accident, while believing you are doing the first.

For analytics that feeds real decisions, our default is in the loop, with the gates sized to the risk. A throwaway exploration needs almost none. A measure that will sit in a board pack needs several. The skill is matching the ceremony to the stakes rather than applying the same heavy process to everything or, worse, applying none.

The gates that matter for BI.

A few checkpoints do most of the work. They are not exotic. They are the things good teams already do, made explicit and non-negotiable for anything that matters.

Validate the DAX. When an agent writes a query, run it and read it. Code-first reporting helps here, because the DAX sits in a file you can open, test and debug rather than something hidden and generated. Use that. An unread query is an unverified one.

Check the numbers against something you trust. Tie at least one figure on every new dashboard back to a known source: a finance report, a system of record, last period's signed-off number. AI-built output is confident whether or not it is correct, so confidence is not evidence.

An agent can produce the answer. It cannot decide that the answer is allowed to be wrong. That is still a person's job.

Hold the line on consistency. Generated visuals drift. Definitions wander. Decide your standards once, write them down, and check new work against them, because the agent will not remember what it did last time.

Name the owner and the sign-off. Every dashboard that informs a decision has a person who owns whether it should exist and a person who signs off that it is right. Sometimes that is the same person. It is never nobody.

Light, not heavy.

None of this should turn into a committee. The point of governance here is not to slow good work down. It is to make sure the speed you gained on the build does not get handed straight back as rework, distrust or a wrong number in front of leadership. A short, clear set of gates does that. A thick process that everyone routes around does not.

Right-sizing the gates.

The fear with governance is that it becomes a tax on everything. It should not. The same set of gates applied uniformly is its own failure: it slows the trivial work and breeds the habit of routing around the process, which leaves the important work no safer than before. Match the gates to the stakes instead.

A quick exploration for your own use needs no gate at all. Let the agent run, glance at the result, move on. A recurring operational dashboard that a team will act on daily needs the DAX read and the numbers tied back to a trusted source. A figure that lands in a board pack or goes to a regulator needs all of that plus a named sign-off and a written definition of what the number means. The judgement is not whether to govern. It is how much, given what happens if the number is wrong.

Make the gates easy to pass.

One last point. Gates that are hard to follow get skipped, quietly, by good people under deadline pressure. So make them easy. A shared checklist, a standard place definitions live, a habit of pasting the validated DAX into the review: small things that lower the effort of doing it right. Governance that is easier to follow than to dodge is the only kind that survives contact with a busy quarter.

What we would do this quarter.

Pick the one gate you are most obviously missing and add it. For most teams that is either reading the DAX before it ships or tying a new dashboard back to a trusted figure. One gate, applied consistently, moves you from "the agent built it and it looked fine" to "we checked it and we stand behind it". That is the whole difference.

If any of this sounds familiar, talk to us about your data.

In the loop or on the loop: governing analytics that AI helped build