Agentic Vision Without Deterministic Rails Is Just a Faster Way to Do the Wrong Thing

Give the pattern its due, because it has earned real ground. A single model that can reason over a scene — decide to zoom in, re-read a smudged field, cross-check against a second detector, call a specialist tool and only then conclude — is a genuine advance over a monolithic detector that returns one number and stops. Expert-guided labeling and rapid training put working vision models within reach of teams that could never have trained one a few years ago. The observability stacks wrapped around these agents — spans, tool logs, token accounting, deterministic replay — are real engineering that make an otherwise opaque system debuggable. And role-based access is real access control: it decides who, and what, is allowed to invoke the agent at all.

None of that is marketing vapor. But stand back and notice what every one of those capabilities governs. The model's competence — accuracy, grounding, drift. The perimeter — who may call it. The record — what happened, after it happened. Three real forms of governance, and a conspicuous fourth that none of them touches: the action the agent takes at the moment it commits. That is the seam this whole category quietly steps over, and in vision it opens directly onto the factory floor, the claims queue, and the security desk.

Here is the load-bearing confusion in the phrase "governance for agents" as the frameworks tend to use it. Observability answers a question about the past: what did the agent do? Governance answers a question about the future: what is this agent permitted to do, and did this specific action pass a gate before it committed? A trace is descriptive and it arrives after the fact. A control is prescriptive and it stands in the path before the fact. Those are different objects, and no quantity of the first ever adds up to the second.

The trace will faithfully record that the agent halted line three at 02:14:07. It will not have stopped the agent from halting line three when the defect was on line four. By the moment the span is written, the line is already down, the claim is already denied, the guard is already walking toward a person. Tracing turns an incident into a well-documented incident — invaluable for debugging, worthless as a safeguard, because a rail placed behind the action is not a rail. It is a better-lit account of the accident. When a vendor offers you an eval dashboard and a replay view as the answer to "how is this governed," you are being handed instruments on the glass. You asked for a hand on the wheel.

The other half of the "it's governed" story is the IAM role — and a role is a genuinely useful thing that is being asked to do a job it was never shaped for. A role is coarse, static, identity-scoped, and binary. It grants a capability class: this agent may call the halt-line tool; this agent may call the deny-claim endpoint. What it cannot express is the policy that actually governs a consequential act:

• Not the confidence condition — auto-act only above a set likelihood, route everything below it to a human.
• Not the threshold — an action above a certain cost, quantity, or severity requires a second approval.
• Not segregation of duties — the identity that reads the frame may not be the identity that commits the write.
• Not the human-in-the-loop condition — an irreversible or safety-critical action pauses for an authorized person before it fires.

Once the agent holds the role, every invocation inside that role is permitted, every time, at machine speed. The blast radius is the entire capability, always. And a non-deterministic planner will, given enough frames, eventually exercise a corner of that capability no one rehearsed — perfectly within its role, and exactly wrong.

This is where the vision case is sharper than the general agent case. The planner is probabilistic by design — that is the point of using one. The same frame and the same prompt can yield a different sequence of tool calls from one run to the next. Which means the set of actions the system can reach is not enumerable in advance. And an action space you cannot enumerate is one you cannot review, sign off, or certify before it runs. In a physical or regulated setting, "we can't tell you everything it might do, but we log everything it did" is not a control narrative. It is a description of the exposure.

Governance-by-prompt does not close this. "Only halt the line if you are highly confident and it is safe to do so" is an instruction to a stochastic system, not an enforcement boundary — a soft constraint the model usually honors and occasionally doesn't, with nothing underneath to catch the exception. In a consequential setting, usually honors is the entire problem restated as a feature. And the consequences here are not a mislabeled row in a table; they are a stopped production line, a wrongly denied claim, a guard dispatched to a human being — immediate, sometimes irreversible, with a real person on the other end.

Some tools sidestep this honestly, and it is worth naming. They reserve the word "agentic" for the reading — plan the extraction, verify it, re-look at the page — and hand the consequential action to whatever consumes the output. That is candid about not governing the act. But candor is not a solution; it relocates the ungoverned last mile to a downstream system. So the pattern collapses to two shapes, and both leave the point of consequence exposed: an agent that acts with only a role and a trace behind it, or an "agent" that doesn't act and throws an ungoverned detection over the wall to a bot or an ERP that does. Neither governs the moment the detection becomes a decision.

The answer is not to ban the probabilistic agent. A stochastic reasoner turned loose on a messy visual scene is exactly the right instrument for perception and triage — that is precisely where you want flexibility. The answer is to bound it, and to put the boundary at the one place non-determinism is intolerable: the moment an action commits.

In Entroid the vision model runs as a governed Function, and an Atomic Agent, orchestrated through Intelligence Orchestration, may reason over the detection and propose an action — but it proposes inside a Deterministic Workflow. Every consequential action is a workflow step that passes an inline policy gate before a Connector — the only primitive permitted to touch the PLC, the ERP, or the dispatch system — commits it. That gate is where the confidence condition, the threshold, the segregation of duties, and the human-in-the-loop check actually live, enforced, not suggested. The agent proposes; the workflow disposes. The non-determinism stays contained to the reasoning, where it belongs. The action surface is deterministic and enumerable, so it can be reviewed and signed off before a single frame flows through it. And the immutable per-action audit — detection, confidence, policy, decision, actor, outcome — is emitted by the runtime as the action commits, not reconstructed afterward from spans that may or may not have captured it. Detect, decide, act, and audit become one accountable transaction on one runtime.

Be precise about what this is and is not. It is not zero-integration; that would be the same over-claim the frameworks make. ES runs over your existing estate — the cameras, the controllers, the claims system — through governed Connectors. The difference is architectural, not aspirational: the integration is the governed part, executed as a step inside the process, rather than a signal thrown over a wall into a system that acts with no policy on the wire and a split-brain record of what the camera saw versus what the second system did.

When a vendor shows you agentic vision, watch where the proof of governance points. If it points at a trace view, an eval dashboard, and a role assignment, you have been shown telemetry and a door key — real, useful, and silent on the act. Ask the different question, and hold the room until it is answered: show me the gate the action passes through before it commits — the policy, the threshold, the approval, the human — and show me that the record of it is written by the thing that took the action, not by an observer watching from outside. If the answer is "you implement that in your own application," then the governance is your unsolved problem, the vendor governed the model and the read, and the agent's speed has quietly become your liability.