Chapter 8: Redesigning a Truth-Seeking Team
For a group of agents, a mix of humans and agents, or even an agent system with almost no humans in day-to-day operation, the biggest risk is fooling itself.
Much of a company's structure exists to serve information compression, blame-spreading, power maintenance, emotional reassurance, and the appearance of order — with facts pushed to the back. Copy those structures into an agent system and you get a machine organization that generates reports, confirms itself, and approves in layers, without necessarily getting any closer to the truth.
Agents change the ground conditions of organization. They can digest much larger bodies of material, spin up isolated judgment roles on demand, leave complete traces, and recombine capabilities at far lower cost. Every layer, process, and role from the old organization needs a fresh audit: is it helping facts flow, or replicating the corporate shell? Is it reducing risk, or manufacturing responsibility theater?
Some mechanisms must stay. Role boundaries, handoff contracts, delegation paths, work records, a final point of accountability — these come from collaboration itself. Without them, a team falls apart.
Some mechanisms must go. Reports substituting for work records, approvals diluting responsibility, hierarchy overriding facts, theater passing for performance — these come from the old problems of human organizations. Agents don't carry the human baggage of self-interest, face, and fear, and shouldn't inherit these patches.
But avoiding the old problems is not enough. Agents have their own failure modes: sycophancy, same-source repetition, mutual endorsement, fluent phrasing passing itself off as sound judgment. The people using agents hallucinate too: they mistake answers they prompted into existence for the system's independent judgment. A multi-agent system has to guard against agents being wrong — and against humans using agents to prove themselves right.
A truth-seeking team is designed through mechanisms: independent contexts, paths back to raw material, real critical authority, fact-checking, traceable records, revocable permissions, reversible outcomes, and a final party that can bear the consequences.