The phrase - Garbage in, Garbage out pops up naturally in most AI conversations. Everyone nods. It’s self explainatory. But when it comes to talking about the fix, most draw a blank. Hence ELI5.
Before we jump into GIGO - it’s importnat to understand why was the phrase was coined, what it meant and why it swung back to relevance.
Origin story of Garbage In Garbage Out
Obviously - that problem got solved. Now, both code quality and data quality are well understood terms and for that matter - mature disciplines. There are vendors categories built around this. Data being correct, formatted, structured - is a thing and it’s done very well. These systems deal with structured data inputs. Not knowledge.
AI activates knowledge - in all forms and net new problems start to surface. For eg - How a specific clause in an amendment contradicts the master agreement. This obviously is not data quality. This is a knowledge quality problem. And systems haven’t built for this yet. (because until recently, nobody needed one)
This problem gets pushed downstream when agents execute basis poor knowledge Agents that read every protocol, every policy, every guideline, and now apply them across thousands of decisions. The problem is when enterprises jump straight from "we have documents" to "we have an AI agent" - skipping the step in between entirely. The model did exactly what it was built to do. It processed what it was given, fluently, confidently, at scale.
How does one knowing if they have a garbage in problem?
You start with garbage out first. Let’s construct a scenario here, but it's close to a few conversations I've had this quarter.
A clinician pulls up a patient summary from an AI bot. It's coherent. Well structured. Sourced, from the same clinical protocols the hospital spent months documenting and loading into the system. She trusts it. Why wouldn't she? Clinician’s workflow boxes them not knowing what they don’t know. If two of those protocols disagree on the dosing threshold for her patient's condition. One was updated 3 months ago. The other wasn't. Both pushed to the same knowledge base. The model blends them into a single answer.
In 1957, bad input produced broken output - physical, visible, traceable, and fixable. Programmers physically saw the damage. In 2026, messy knowledge sources produces a beautifully formatted wrong answer that’s near impossible to identify.
Let’s say you spotted garbage out. What Next? What’s the prognosis?
What typically gets spotted is - “this doesn’t seem correct. The model hallucinated. Seems like knowledge is incomplete, add more. If the answer is wrong, give it more context. More documents. More sources. More coverage.” Really smart teams do this. It feels right. The knowledge base grows. The problem, unfortunately compounds. More documents means more versions of the same facts. More versions means more surface area for contradiction.
Context windows don't filter for authority. All the model does is retrieve. The model blends across all of it, and the blend gets smoother and more confident the more material it has to work with.
This is what makes this extremely hard to fix: the garbage doesn't look like garbage. Because you are not dealing with typos or missing fields or incorrect dates. Across our experience of working and deploying agents to production, here’s what we have learnt about problems that exist in messy knowledge sources.
You are dealing with 3 Problems In Messy Knowledge Sources
Why Audit Trails are critical for correct Diagnosis - breaking down the Garbage Out problem
The hardest thing to accept about this garbage-in garbage-out, and it took me a while to get comfortable saying it out loud: LLMs will always generate things outside your knowledege. That's what they do. You cannot engineer your way to zero hallucination.
A simple decision tree to decipher garbage out
You can spot the hallucination in the output but the model will do what it will do. What one can do - and this is the part I wish I'd understood earlier - is tell the two kinds of failure apart. Because without an audit trail, everything looks the same.
The first is a knowledge failure. The source was wrong, conflicted, or outdated. The model did exactly what it was supposed to do. The fix belongs upstream. A neuro-symbolic approach to knowledge management gives you exactly that feedback loop - traceable to the source document, resolvable at the root. Fix the knowledge. The output quality is bound to improve.
The second is a generation failure. Clean, coherent knowledge system and the model still erred. Different problem. Different system. Different feedback loop entirely - one that runs through guardrails, not through your knowledge base. [I've written about guardrails here]
When both failures look the same, the fix is pushed downstream - on to a human reviewer, correcting outputs, one at a time or, with no way to tell whether they're fixing a symptom or a cause. And that quickly turns into a phantom human-in-the-loop. It's a system that handed them a diagnostic problem disguised as a correction task. An audit trail separates these two.
How do you fix Garbage In?
The instinct is to reach for a technical solution. Better embeddings. ReRanking. Smarter retrieval. A popular one - a larger context window. None of that even touches the core problem. Here's what we’ve zeroed on, and I'm still refining this - but the core hasn't changed in months of conversations.
AI cannot resolve truth. It can surface conflict - flag that two documents disagree, identify which version is newer, show you where the gap is. But the decision about which version is actually true? That belongs to a human. A domain expert. A clinical lead who understands what the policy was trying to say. That's not a process gap you can automate around.
Truth in a regulated industry is a judgment call, and it always needs a name attached to it. This means you need audit trails. Audit trails for the knowledge itself - who resolved a conflict, when, what the previous version said, why it changed. Without that, you don't have a knowledge base. You have a document pile and model giving a confidence score on top.
Same thing applies when external contracts and policies change - can be regulatory or org wide. Think FDA revising a guideline, a payer modifies their prior auth criteria - someone inside your organization needs to explicitly map that change to your internal SOPs and sign off. Not implicitly adopt it. Not hope the AI notices.
Then there is the operational reality. Businesses don't run just on SharePoint. They run on Slack. On WhatsApp groups. On email chains where a compliance officer confirms that yes, the new policy supersedes the old one, effective immediately. That knowledge is real. Your agents don't have it. And until there's a curation layer that captures conversational sources - not just formal documents - your knowledge base will always be incomplete in ways you can't see.
This is what I mean by production-ready knowledge. Not more documents. Not better retrieval. A governed, versioned source of truth where every update has an owner, a timestamp, and a record of what came before. [This is what a neuro-symbolic approach to knowledge management allows for]
I built a self-diagnostic around this. Seven questions - Answering these takes lesser time than making instant coffee. Most organizations I've shown it to get stuck by question two.
But hey - that’s a not blame game. It's a starting point.
If this landed, let me know - do reply. And if you know someone who is dealing with 1000s of documents and is still blaiming their AI vendor, send them the knowledge audit.
Vivek K
ps - This was a much delayed edition of ELI5. Still building the weekly rhythm of writing. Hopefully, the next edition will hit your inbox on weekend.