By Kajal Singh

For the past two decades, enterprise security teams have gotten good at one thing: keeping sensitive data where it belongs.

Related: Leaked secrets no. 1 exposure

Production data stays in production. Test environments get masked or synthetic data. Access is controlled. Ownership is defined. The system, while imperfect, largely works.

Then AI arrived — and that discipline quietly broke.

Not because teams forgot what to do, but because the workflows changed faster than the controls did.

Today, production data routinely flows into AI pipelines with far less scrutiny than it ever faced in traditional software development. It moves through data lakes, feature stores, training pipelines, evaluation sets, and sometimes third-party platforms. Along the way, the original boundary — the question of whether that data should have left production at all — often disappears.

Ownership without owners

No one is explicitly responsible for asking it.

This is not a tooling failure. It is a breakdown in ownership.

In one large financial institution, a team built a machine learning model using what they believed were low-risk, derived features. The raw data had already been processed, transformed, and abstracted. On paper, it looked safe.

But as those features accumulated and recombined inside the feature store, they began to reconstruct something much closer to the original sensitive data than anyone intended. Individually, each feature passed review. Together, they created a high-sensitivity exposure that no single control had been designed to catch.

The issue wasn’t a lack of encryption or access controls. It was that no one was tasked with evaluating how data changed meaning as it moved through the pipeline.

That’s the gap. Traditional data security models assume relatively stable environments. Data is classified, protected, and monitored within defined boundaries. Even when it moves, those movements are predictable and governed. AI workflows are different.

Who owns the pipeline?

Data is continuously transformed, combined, and repurposed. Context shifts at every stage. A dataset that appears benign at ingestion can become sensitive after transformation. A feature that looks harmless in isolation can contribute to meaningful reconstruction when combined with others.

And yet, most security programs still treat data protection in AI pipelines as an extension of existing controls, rather than a fundamentally new problem.

The result is a blind spot. Security teams often assume that if data was approved for use upstream, it remains safe downstream. Data teams assume that if they are working with derived features, the original sensitivity no longer applies. AI teams focus on model performance, not data lineage or exposure risk.

Each group is acting reasonably — within its own frame of reference. But no one owns the full path.

Asking the hard question

That is where the breakdown occurs. The question that needs to be asked is simple, but it is rarely formalized: at every stage of the pipeline, does this data still belong here?

Not just from a technical standpoint, but from a risk and compliance perspective. Answering that requires more than better tools. It requires a shift in how organizations assign responsibility. Someone — whether it sits in data security, AI governance, or a cross-functional review process — has to take ownership of how data is evaluated as it moves, transforms, and recombines inside AI systems.

Singh

That means tracking how derived features relate back to source data, paying attention to how combinations of those features can introduce new exposure, and making explicit decisions about whether data should continue downstream at all. It also means reintroducing the idea of boundaries, even in workflows that are designed to be fluid. Without that, organizations are effectively trusting that sensitivity doesn’t re-emerge once data has been transformed.

Evolve or expose

That assumption does not hold.

AI doesn’t just use data. It reshapes it. And in doing so, it can quietly undo the controls that were designed for a different era.

For security leaders, the takeaway is not to slow down AI adoption. It’s to recognize that the control model has to evolve with it.

If your team cannot clearly answer who owns data once it enters an AI pipeline, how sensitivity is evaluated after transformation, and where the decision is made about whether data should continue downstream, then the system has already failed — even if nothing has gone wrong yet.

That’s the moment to intervene.

Because by the time a breach or regulatory issue surfaces, the problem won’t be that the model was misused.

It will be that the data should never have been there in the first place.

About the essayist:  Kajal Singh leads enterprise data security strategy at Oracle. She focuses on data protection for LLM and GenAI pipelines and holds an M.S. from Dartmouth.

May 26th, 2026 | Guest Blog Post | Top Stories