Forward Pass #3: Garbage In, Confidence Out
Garbage In, Confidence Out
Last issue, the agent took the worst job you own and did it well. Merged cells gone, types fixed, one canonical spelling of Texas. The file came back clean. Here is the uncomfortable part, and the reason this issue exists: clean and correct are two different words, and the distance between them is where careers are made or quietly ended.
Clean means well formed. Correct means true. A column of perfectly formatted revenue that still includes intercompany sales is spotless and wrong at the same time. The agent cannot tell the two apart, because the difference doesn’t live in the data. It lives in the business.
‘Garbage in, garbage out’ is a forty-year-old cliche. What kept it survivable was human throughput. An analyst working a broken table by hand eventually smells it: the total that is off by an order of magnitude, the region that should not exist, the month with impossible numbers. You slow down. You go check. That pause was the immune system of the whole profession.
An agent has no nose. Point it at a flawed table and it returns a fluent, formatted, confident answer at machine speed, across every surface it feeds, all at once. It never hesitates, because it has nothing to hesitate with. The garbage did not disappear. It just travels faster now and arrives wearing a suit.
Here is the part only this newsletter will keep saying. The danger was never that AI produces wrong answers. People do that too. The danger is that it produces them with no tell: no hedge, no “this looks off,” no slowing down. Confidence used to correlate with correctness, because a confident human had usually checked. Agentic AI breaks that correlation. The answer sounds exactly as sure when it is wrong as when it is right.
In the Loop
So the smell test has to move. It stops being something that happens on its own, downstream, in a tired analyst’s gut, and becomes something you build in on purpose, upstream.
Three things stay yours.
1. You decide what correct means.
Not clean, correct. Does revenue include intercompany? Is a churned customer who came back one logo or two? The agent will compute whatever you point it at. Only you know which computation is the true one.
2. You supply the suspicion the agent lacks.
You know the number that is too good. You know sales do not triple in a quarter. Encode it: the valid ranges, the sanity checks, the “flag anything outside this.” The machine will never generate doubt on its own. You have to install it.
3. You own the confidence.
When the agent hands you an assured, well-formatted answer, the assurance is a formatting choice, not a truth claim. Someone has to supply the missing doubt, and that someone is now doing one of the most valuable jobs in the building.
Data quality, it turns out, is not a technical property you can automate away. It is a business judgment, and it enters the system only through a person who knows the business.
Try This
Take a table you know has one real flaw: a duplicated customer, a mislabeled region, a figure that includes something it should not. Do not fix it. Point a data agent or Copilot at it and ask a business question whose answer depends on that flaw.
Watch what comes back. A clean, confident, well-formatted answer, with no asterisk, no “note that,” no flag. The flaw rides straight through the output and out to whoever reads it next.
That silence is the entire lesson. The agent is not lying to you. It simply has no mechanism for doubt. For now, you are the mechanism.
Prompt & Circumstance
One prompt worth stealing: make it show its assumptions before it answers.
The default agent answers the question. Make it answer a harder one first.
Before you answer, list every assumption you are making about this data: what each column means, how you are handling duplicates and blank values, and any figures that look inconsistent or out of range. Flag anything you would want a human to confirm. Then answer, and tell me how confident you are and why.
This does not make the data correct. Nothing you type does. But it turns a confident monologue into a checkable one. You are forcing the pause the agent will not take on its own, and the flags it raises are the exact spots where your judgment goes to work.
Signal
The industry knows this is the problem. Microsoft’s answer is Fabric IQ and its Ontology layer, in preview, a semantic foundation that grounds agents in your business’s real definitions and relationships instead of raw tables, so an agent reasons about a Customer, not a column.
It is a real step. But read the fine print, because the fine print is the whole point. An ontology generated from a vague semantic model inherits every ambiguity in it: ontology quality determines agent quality. The tooling does not manufacture correct. It propagates the correctness you already defined, or the confusion you did not resolve. One analyst put it more bluntly: ontologies don’t build themselves.
Before You Trust the Answer
Which is the useful, uncomfortable truth under all of it. Every agentic capability arriving this year makes your data faster to act on. Not one of them makes it true. That is still a human job, and it sits upstream of every demo.
So before you let an agent answer from your data at machine speed, it is worth knowing what it would confidently get wrong. That is what we check. Our free Data Health Check hunts for the flaws that survive cleanup: the undefined metrics, the duplicate entities, the “revenue” that means three things in three reports. That is the exact material an agent will render into a confident wrong answer. If you would rather find it before your agents do, let’s talk.
The distance between a question and a built answer is collapsing fast. Next issue, we watch it disappear.
Until the next pass,
Isaac Truong | Founder, Allston Yale | Power BI and Microsoft Fabric Consulting
Sources:
- What is Fabric IQ? (Microsoft Learn).
- Fabric IQ and Ontology: when your data speaks the language of the business (Microsoft Fabric Community).
- Microsoft Fabric IQ adds ‘semantic intelligence’ layer to Fabric (InfoWorld).