Confluent: Stale, unreliable data could impair enterprise AI

Australian enterprises looking to scale artificial intelligence need to focus less on the capabilities of AI models and more on whether the data feeding those systems can be trusted, according to Confluent, an IBM Company.

The data streaming company says trust is emerging as a major barrier to enterprise AI adoption as organisations move beyond experimentation and begin putting generative and agentic AI applications into production.

The focus of the AI conversation has changed significantly over the past 12 months, said Andrew Foo, VP Customer Solutions APAC, Confluent, an IBM Company.

"In 2026, the promise of AI and AI's agency to reason, make decisions, and take action has very much taken over a lot of the data discussions," he said.

While much attention had initially centred on what AI models could do, organisations are now confronting the practical challenges of deploying those systems safely and reliably, he said.

"The challenge now is increasingly shifting from what the models themselves can do, to whether the businesses can actually trust the data that's underpinning it," Foo said.

Confluent's 2026 data streaming report, based on a survey of about 4600 IT leaders, found that 72 per cent cited challenges around areas including data lineage, timeliness and data quality assurance.

Sixty-nine per cent said insufficient real-time data infrastructure was holding back their ability to scale AI.

These findings highlight a fundamental issue for businesses seeking to move AI from pilot projects into business-critical applications.

They need confidence that the information supplied to generative AI systems is accurate, current and sourced appropriately, whilst ensuring the right controls are in place over who can access that data.

That challenge becomes even more significant as companies adopt agentic AI systems capable of taking actions rather than simply generating answers.

As a result, organisations need to expand their approach to AI governance beyond the models themselves and towards the data continuously feeding them.

Foo warned that organisations should not view governance and speed as competing priorities, particularly as they face pressure to keep pace with the rapid development of AI.

Instead, security and governance should be incorporated into data architectures from the beginning rather than being added after an AI application has already been built.

"What they should be thinking about is embedding this into their data architecture from the outset," he said.

Confluent advocates a 'shift-left' approach to data governance, moving controls closer to the source of the data instead of attempting to introduce them further downstream.

It has introduced capabilities designed to support that approach, including automated personally identifiable information detection and redaction within data streams.

This allows organisations to protect sensitive information before it reaches downstream AI applications.

Confluent has also introduced private connectivity capabilities intended to allow enterprises to connect AI workloads to external models without sending sensitive traffic across the public internet.

Such capabilities are particularly relevant in highly regulated sectors including financial services, healthcare and insurance.

Foo is adamant that governance should become an embedded part of how organisations build applications rather than being treated as a separate compliance exercise.

"You don't do a data governance project. You do data governance in every project," Foo said.

"The same applies for AI. You don't just do an AI governance project. You do AI governance in every project."

Real-time data becomes critical

Another major challenge for organisations adopting AI is the freshness of data available to those systems.

Traditional approaches based on batch and micro-batch processing are becoming inadequate for AI applications that need to respond to rapidly changing business conditions.

"AI is only as useful as the data and the context that's available to it," he said.

"In many enterprise scenarios, yesterday's information simply isn't enough."

Businesses continuously generate new events, from financial transactions and customer interactions to inventory movements, security incidents and operational updates.

Connecting AI systems to those live streams allows applications to make decisions based on current conditions rather than historical snapshots.

Foo cited fraud detection in the banking industry as an example.

A bank assessing a new credit card application may have information indicating that a customer previously represented a relatively low risk. But in the hours since the bank's last batch update, that same customer may have made unusual transactions, logged in from another country or used an unfamiliar device.

An AI system relying on stale information could therefore approve an application that should have been flagged.

With real-time data, however, the system can combine information about transactions, devices, geography and other risk indicators to make a decision based on what is happening at that moment.

"AI is doing the right thing. It simply didn't have access to the real-time information it needed to make the right decision," Foo said.

For Confluent, the role of data streaming is to turn those continuously changing business events into governed and trusted context that AI applications and agents can use.

Where technology leaders should start

For companies trying to strengthen their data foundations, the first step should be understanding the information already flowing through the business.

Technology leaders need to understand where data is coming from, how fresh it is, who can access it and whether it can be trusted.

Foo cautioned against treating AI as a standalone project that requires an entirely separate data environment.

Instead, organisations should look to securely connect AI applications to the operational data already powering the business.

Leaders should also identify security and governance requirements early, particularly for sensitive and regulated information, rather than attempting to retrofit controls once an AI application reaches production.

Focus then shifts to creating governed, real-time data products around the core entities of the business, including customers, products, orders and inventory.

"If you have this as a strong foundation from the outset, it will ultimately make it easier to innovate without introducing unnecessary complexity or risk when it comes to advancing your AI ambitions," Foo said.