Every few weeks, I look at another AI governance product and notice the same architecture underneath.

When an AI agent requests permission to take an action, the current industry approach across control planes, execution boundaries, deterministic wrappers, and decision layers is roughly as follows. The request comes in. It passes through analysis, gates, and policy checks. Then, at some point in that pipeline, a language model evaluates whether the agent should proceed.

An LLM deciding whether an LLM should act.

There are variations. Some systems do more analysis before that final evaluation. Some add rule layers on top. Some call it something different. A scoring engine. A policy evaluator. A confidence ranker. The underlying mechanism stays the same. The decision about whether an AI action is appropriate is itself a probabilistic AI decision. The system being governed and the system doing the governing are operating on the same architectural foundation.

It’s the same conflict with placing deterministic constraints on agentic-based systems, which, in my opinion, negate the entire premise of having a probabilistic LLM in the stack.

What Decision Theory Actually Says

Decision theory is the study of how decisions should be made under uncertainty. Its concern is prescriptive rather than descriptive. The rational framework for evaluating options when outcomes lack guarantees.

At its core, formal decision theory asks a specific question. Given what we know, and given what we value, what action minimizes expected regret or maximizes expected utility? The answer requires quantifying uncertainty, rather than merely acknowledging it. It requires being explicit about what outcomes matter and how much. It requires treating the decision itself as an object of analysis rather than a reflex.

This is fundamentally different from pattern matching. Pattern matching looks at what the input resembles and responds accordingly. Decision theory asks what the consequences of different responses are, how likely those consequences are, and which response produces the best expected outcome.

The distinction matters enormously for governance. An AI system that pattern-matches to governance rules will behave correctly on cases that resemble its training distribution and unpredictably on cases that diverge from it. A governance system built on decision-theoretic principles evaluates each situation against explicit criteria: scope, consequence, reversibility, authority, and behavioral history. It produces a verdict that is explainable in terms of those criteria, regardless of whether the situation was anticipated.

The Structural Alternative

Another school of thought is emerging in AI governance discourse.

The argument, roughly: if admissibility can be defined structurally, then invariance is entailed rather than enforced. Invalid paths become impossible. Drift becomes non-expressible. The system holds together because violation was never available as a state in the first place, rather than because something is watching it.

This is an elegant move. It borrows from physics, specifically from the way conservation laws follow from symmetry principles. Energy holds through time-translation symmetry rather than enforcement. Noether’s theorem, applied to governance.

The problem is that conservation of energy is true whether anyone is watching. Admissibility is true because someone decided it should be. No symmetry principle tells you whether an AI should approve a loan, recommend a drug, or escalate a political post. Those are heteronomous impositions by definition. A human authored the policy. Another human approved it. A third is accountable when it misfires.

“Defined structurally” sounds as if the definition falls out of the universe. The reality is different. It still requires a definer. The structural claim just moves the policy decision upstream and stops talking about it.

This matters because the mechanism you choose carries epistemic commitments. A framework that pretends admissibility is a structural property is a framework that has hidden its authorship. A governance system with hidden authorship produces no audit trail, supports no accountability, and has no way to answer the one question that actually matters when something goes wrong: whose decision was this?

What a Different Approach Looks Like

I will say something here and then stop, because the details are protected for good reason.

The approach I took at Nomotic starts from a different premise. Rather than asking an AI system whether an AI action is appropriate, the question is: what does formal decision theory say about how a governance verdict should be constructed, given everything we know about this agent, this action, and this context?

That means being explicit about what factors matter and how much. It means treating reversibility, consequence magnitude, authority chain integrity, and behavioral history as measurable inputs to a formal evaluation rather than heuristics that a language model applies based on training. It means producing a verdict that is reconstructable. You can show exactly why a given action received the score it did, with the reasoning visible alongside the score itself.

It means governing AI the same way consequential choices are made under uncertainty. Through a structured analytical process rather than another LLM. With explicit criteria, weights, and rules for combining information into a verdict.

I will skip the specifics of how that works at Nomotic. There is pending IP that covers the decision framework, and frankly, it is the part of the system that makes Nomotic structurally different from the variations on the same theme the market keeps producing.

Why Accountability Needs a Mechanism

Here is the part that usually gets skipped in the governance conversation, and I think it is the most important.

Decision theory produces more than better verdicts. It produces verdicts traceable to explicit criteria authored by identifiable people.

When a bank declines a loan, regulations require it to explain its decision. The criteria are on record. The weights are on record. The human who set the policy is on record. The audit trail is the point, rather than an afterthought. It is what makes the decision defensible when challenged and correctable when wrong.

AI governance needs to work the same way. Less because regulators will eventually demand it (though they will), and more because there is no other way to answer the question that always arises when an AI action causes harm: who was responsible?

“The system drifted” fails as a defense. Somebody configured the agent. Somebody approved the policy. Somebody selected the model. Somebody signed off on the deployment. Every step has a human on record in every other consequential technology we operate. Bank transactions. Building access logs. Database commits. Email sends. File shares. The audit trail pattern is hardly exotic. It is how accountability has worked for decades.

A governance mechanism built on opaque probabilistic judgment fails to produce this kind of audit trail. You can log what the model said. You have no way to reconstruct why it said it, or show which criteria mattered most, or trace the verdict back to a human-authored policy. The reasoning remains unavailable because it was never structured in the first place.

A governance mechanism built on explicit decision-theoretic evaluation produces the audit trail natively. The criteria are authored. The weights are declared. The verdict can be reconstructed from the inputs. When something goes wrong, you can answer who decided, and on what grounds, and whether the grounds were appropriate.

That is what governance actually is. The main event, rather than a side benefit.

Why This Is the Right Question

The AI governance conversation is dominated by what? What tools are being deployed? What policies are being enforced? What verdicts are being produced?

Decision theory asks a different question. How should a rational agent make a decision under uncertainty, given explicit criteria and a desire to minimize expected regret?

That question, applied to governance, changes the architecture. It pushes toward explainability as a structural requirement rather than a compliance feature, because verification of a decision-theoretic process requires reconstructable reasoning. It pushes toward formal criteria because informal heuristics resist composition into a decision framework with predictable properties. It pushes toward separating the mechanism of governance from the probabilistic system being governed, because a reliable governor must have different reliability properties than the system it governs.

The vendors building LLM-over-LLM governance make a fair point. Their approach works in many cases. They are building something real and, in many deployment contexts, sufficient.

What they have yet to build is a governance system whose decision-making process is as rigorous as the decisions it makes are consequential. And what they have yet to build is a governance system that can answer, when it fails, whose decision this actually was.

Both questions matter. Decision theory addresses the first. Heteronomous design addresses the second. You need both, and you need them connected.

That is the problem decision theory was invented to solve. And it is the problem that the AI governance industry, for the most part, has yet to take seriously.

---

If you find this content valuable, please share it with your network.

Follow me for daily insights.

Book me to speak at your next event.

Chris Hood is an AI strategist and author of the #1 Amazon Best Seller Infailible and Customer Transformation, and has been recognized as one of the Top 30 Global Gurus for Customer Experience. His latest book, Unmapping Customer Journeys, will be published in 2026.