ORLANDO, Fla. — Every recommendation a viewer sees and every dashboard an executive trusts rests on a record that originated in a device, application or external feed before entering a data pipeline.
That failure is common enough to measure. 63% of organizations either lack the data management practices their AI work demands or cannot say whether they have them. The companies moving the most data often feel the consequences first.
Riazullah Khan, a senior data engineer who has worked across media, banking and health care, has spent 17 years building high-volume streaming and analytics pipelines that carry raw events from millions of devices into the models behind personalization and recommendations.
The cost of treating governance as paperwork
A data platform rarely fails all at once. It fails one quiet compromise at a time.
A validation check gets skipped to hit a deadline. A table loses its owner after a team change. Two departments use different definitions for the same metric.
Khan’s response is to make those compromises harder to take. When he designs a pipeline, governance rules are part of the architecture from the start: what a valid record looks like, where malformed events go and who is allowed to access or modify a shared table.
In his streaming work, an event that fails validation does not reach long-term storage. It is set aside for inspection instead of being silently dropped or allowed through.
“A policy document is a promise,” Khan said. “A pipeline is a fact. The first one gets broken the week a deadline lands. The second one holds, because it has to run before anything downstream can.”
Quality starts where the data enters
The cheapest defect to fix is the one caught at the front door. Many organizations, however, still lack a structured way to measure whether their data is reliable.
59% of organizations do not measure data quality in any structured way, which means they cannot easily say what a bad record costs them or whether the problem is growing.
In a data engineering project, that blind spot becomes expensive. A single malformed field accepted without challenge can flow downstream into tables, joins, dashboards and models, gaining authority simply because it survived.
That is why Khan treats ingestion as the moment quality is won or lost. His pipelines check incoming events against an expected structure before they are allowed to persist. Records that fail are quarantined where an engineer can inspect them.
“Every bad record you let in at ingestion, you meet again later, usually attached to a decision someone already made,” Khan said. “Catch it at the door and you solve the problem once. Miss it there and you solve it forever.”
Someone has to own the table
Speed of ingestion is only part of the problem. The harder challenge is structure: how data is named, organized and stored so a person can find it, trust it and know who is responsible for it.
A platform that begins with a handful of clean tables can become thousands of tables scattered across databases and data lakes. A table with no clear owner is a table no one is eager to fix.
Khan designs against that by treating the downstream consumer as the customer of every schema he writes. Models are only as reliable as the data beneath them. A recommendation system fed inconsistent or poorly structured inputs returns inconsistent output, regardless of how advanced the model may be.
That means data must be shaped and documented before the machine learning team touches it. What they receive should already be predictable, usable and traceable.
“Everyone wants to talk about the model,” Khan said. “The model is the easy part. The hard part is handing it data organized well enough to trust, and that work happens in the pipeline, not the notebook.”
The failures that do not announce themselves
The failures that hurt most are often the ones that never trigger an alarm.
A pipeline that crashes gets attention quickly because everyone can see it is broken. The more dangerous failure keeps running while producing wrong answers: a duplicated event, a dropped batch or a schema change that slipped through without a flag.
Khan builds systems to make quiet failures visible. Durable buffers, serverless processing and partitioned storage can help keep event data intact through traffic spikes and partial outages, while traceability shows where each record entered and where it landed.
Alerting sits over the top, so a delay or gap can surface within minutes instead of appearing in a report days later.
“The pipeline that falls over is a good day,” Khan said. “It tells you the truth immediately. The one that keeps running and lies to you is the one that ends up in a board deck. Governance is how you tell an honest pipeline from a merely confident one.”
The layer the next model stands on
The pressure on data teams is not easing. As organizations race to build AI systems on top of their data, weak foundations are often the first to fail.
A model cannot clean the data it was handed, and no amount of tuning can fully rescue a system trained on records no one governed. Projects that survive in production are often the ones whose data was trustworthy before the model existed.
The visible work may be the model. The durable work is the pipeline underneath it, designed early and deliberately so the data remains worth trusting when more ambitious systems arrive.
“Everyone remembers the model that shipped,” Khan said. “Nobody remembers the validation rule that kept it honest. But the teams still standing in a few years are the ones that made governance quiet and automatic, so every model and every report after it inherits data worth trusting. It is the least glamorous work in data engineering, and it is what separates the platforms that scale from the ones that quietly come apart.”