Data Quality That Flags After the Write Is a Cleanup Crew

Start with what the data-quality and observability tools got right, because it is substantial. Profiling an estate at scale is real work: distributions, null rates, referential orphans, schema drift, volume and freshness anomalies learned from months of history rather than hand-written thresholds. Rolling those signals up into quality scores — by table, by data product, by domain — gave executives a view of rot that simply did not exist a decade ago. And the issue workflows that follow the alert — triage, route to the owning team, track to closure — imposed order on what used to be a tribal thread between whoever noticed and whoever remembered how the pipeline worked.

The posture is also honest for the terrain it was built on. These tools sit beside an estate they do not control: hundreds of pipelines written by teams that have since reorganized, ingestion from sources you do not run, warehouses loaded by scripts nobody will claim. When you do not own the write, watching everything and flagging deviation quickly is the only move available, and the category's own case for itself — fewer incidents going unnoticed, faster time-to-detection, quality visible to the business — is directionally credible. Detection is a real discipline, practiced well.

The question a CIO should ask is narrower and more uncomfortable: why is the write something we can only watch?

The cost of a bad record is not a constant. It compounds with every hop the record survives.

At the moment of the write, it is one wrong row, and the cost of refusing it is nearly zero. Then replication copies it into the lake and the read replicas. A join folds it into aggregates where it no longer looks wrong — it just makes the total wrong. A dashboard refresh turns it into a decision input. A feature pipeline feeds it to a model, where it stops being a record at all and becomes a weight. Somewhere in that window, an agent retrieves it as context and acts on it. Each hop multiplies the remediation bill: the fix is no longer correct one row but find every table it touched, re-run every join, restate the aggregate, and explain the decision that was made on Tuesday.

Now place the detection loop on that timeline. The monitor fires on the next profiling pass or the next anomaly evaluation — after landing, usually after replication, often after the dashboard. That is not a criticism of any tool's latency; it is the definition of the pattern. A detector, by construction, observes what has already been committed. The flag arrives after the blast radius. Notice the asymmetry in what gets measured: the category optimizes time-to-detection, but the enterprise pays for time-in-circulation — and the second clock starts before the first one does.

It is worth being precise about why this pattern cannot simply be tightened into prevention, because roadmaps promise exactly that. A tool that stands beside the platforms holding the data can only read what those platforms have committed. It has no standing in the write path. The pipeline does not ask its permission; the loading script does not know it exists. That is a structural property of sitting beside the estate, and it holds no matter how fast the monitors get.

The partial exception proves the boundary. The stronger tools in the category do push checks upstream: validation embedded as pipeline tasks, tests a pipeline author adds before the load step. That is real, and better. But the check is volunteered per pipeline, by the team that writes the pipeline. The quality platform can verify the check exists; it cannot make the write conditional on it. Every other writer in the estate — the older pipeline, the manual load, the vendor feed, the one-off script — writes anyway. The rule and the write live in different systems, so the rule can be perfectly correct while the write proceeds.

Even the genuine enforcement this category has produced sits on the other side of the data. The most enforcement-minded architectures in the space apply policy at query time — attribute-based access control, row- and column-level security, dynamic masking, applied inside the data platforms they front, with every query authorized and logged. That is enforcement, not documentation, and it deserves the respect it gets. But it governs the read: the door where data leaves the platform. Quality is a property of the write — the door where data enters. A perfectly masked read of a wrong number is still a wrong number, delivered securely.

On Entroid's Composable Process Fabric, the geometry changes — and the change is architectural, not aspirational. The Semantic Ontology is not a catalog describing tables that live somewhere else; it is the governed model the runtime executes on. Business processes run as Deterministic Workflows against ontology entities, and a quality rule attached to an entity — schema conformance, referential validity, range constraints, required dimensions — is a gate inside the workflow performing the write. Validation is not a monitor beside the process. It is a step of the process, and the runtime does not execute the write without it. There is no second door for the record to slip through, because on the fabric there is no write path that bypasses the workflow.

Take an illustrative flow — hypothetical by design, not a case study. A supplier invoice arrives for posting. One line carries a cost center that fails referential validity against the ontology, and a unit price outside the tolerance the rule defines. The workflow refuses the write at the gate. There is no bad record to replicate, because there is no record: the replicas, the joins, the dashboards, the feature pipelines, the agent's context window — none of them ever meet it. The refusal routes through an Atomic Agent to a human reviewer — human-in-the-loop is a first-class primitive, not an escalation bolt-on — who corrects the cost center or approves an explicit, attributable exception. And the refusal, exactly like an acceptance, lands in the immutable per-action audit.

That last property is the one that quietly retires the scanner. On the fabric, quality history is not a score computed by a profiler visiting your tables; it is the execution record itself — every write that passed, every write that was refused, which rule fired, which actor resolved it. Lineage is emitted per action as a byproduct of execution, not crawled and stitched afterward. And because the read, the write, and the business action share one gate and one audit, the question changes shape. Detection asks: what is the state of the data? The fabric can answer the harder one: what has every process been allowed to do to it?

None of this argues for turning the monitors off. Entroid runs over the existing estate through governed Connectors — the only primitive that touches external systems — and data that originates in platforms the fabric does not execute is still data you should observe. A detector is the honest instrument for writes you do not own. The indefensible position is accepting detection for writes you do own.

So the CIO question is an inventory, not an ideology. Take last year's most expensive quality incidents and ask, for each one: who performed the write? Where the answer is a process your own teams run — a posting, a master-data update, an order intake — the monitor was a choice, not a necessity; that write could have been gated. Where the answer is a system genuinely outside your control, detection remains the right posture and the observability investment stands. As processes move onto the fabric, the second list shrinks toward the true edges of the estate — and quality stops being a score you watch and becomes a property the runtime guarantees, one refused record at a time.