AIBUILD launches AI verifier to fight scam messages

AIBUILD has launched an AI Content Verifier designed to identify AI-generated and suspicious communications. The Melbourne company says the system is already being used in government operations.

The launch comes as scam losses in Australia reached AUD $2.18 billion over the past year, based on figures from the Australian Competition and Consumer Commission cited by AIBUILD. The data also showed losses rose 7.8 per cent, while investment scams accounted for AUD $837.7 million.

The new tool is designed to detect manipulated or fabricated text, images, audio and video that may appear legitimate to users. AIBUILD says the rise of AI voice-cloning and impersonation scams has created a new challenge for businesses and consumers, with an estimated AUD $25.8 million lost in Australia in the first half of 2025 to scams using generative AI to clone voices.

Rather than scanning text alone, the system reviews multiple forms of content together, looking for inconsistencies in language, structure, provenance and content patterns. AIBUILD says the model was trained on thousands of corporate and government communications to help distinguish official messaging from material created or altered by AI tools.

Internal testing cited by the company covered 600 images from government communications, including material generated by leading AI models. In those tests, AIBUILD says its image detection correctly identified AI-modified images 98 to 100 per cent of the time and recorded a false positive rate of 1 per cent or lower on authentic images.

Growing threat

The launch reflects broader concern in Australia about the changing nature of fraud. Scam operators are increasingly using cloned voices, realistic branding and compromised business systems to make false communications convincing enough to pass basic checks.

Yifei Wang, chief executive officer and co-founder of AIBUILD, linked the new product to that shift.

"The multitude and sophistication of scams now burdening everyday Australians has never been as concerning as it is right now. Not only has there been a significant increase in scam losses, up 7.8 per cent in the past year, but scammers are getting better at fooling people," Wang said.

"They now have bigger teams and far more resources, and because of this, a new front has opened: AI voice-cloning and impersonation scams," he said.

"Once just a far-fetched threat, these cloning scams cost Australians an estimated $25.8 million in the first half of 2025 alone, from scammers using generative AI to clone the voices of family members, bosses and public figures, sometimes from just a few seconds of audio lifted off social media," Wang said.

Wang argued that existing fraud checks are often too narrow to counter newer attacks that blend several types of media in a single approach.

"Most existing detection tools weren't built for this. They scan only text in isolation, missing the image, the context, or the structural fingerprints that separate a genuine communication from a fabricated one," he said.

"What we've built responds directly to this issue: the AI Content Verifier, a multi-modal system that assesses whether text, images, audio and video are authentic, manipulated or AI-generated," Wang said.

Business case

AIBUILD pointed to a case involving Michael Wilczynski, who runs National Accounts Group, to show how impersonation attacks can go beyond simple phishing emails. His account suggests that even established controls such as multi-factor authentication may not stop a determined fraud attempt if attackers gain access to trusted systems and exploit internal processes.

"Earlier this year our firm was hit by an incredibly sophisticated attack posing as a trusted associate of the business," said Michael Wilczynski, business owner of National Accounts Group.

"The scam email was sent through our accounting software provider's own servers, so it passed every authentication check. The attackers then defeated multi-factor authentication, took over an account, posed as a trusted supplier and quietly changed bank details so a legitimate payment went to a fraudulent account," he said.

"The way they leveraged AI to do this is what worries me most," Wilczynski said.

He said one of the standard protections used by finance teams may now be under pressure from voice synthesis tools.

"The standard advice after every payment scam is 'pick up the phone and verify'. AI voice cloning is now attacking exactly that safeguard. A finance officer who calls to check new bank details, or takes a call from 'the director' approving an urgent payment, can no longer trust their ears," he said.

"It's a concerning time we are living in, but the more systems we have at our disposal to 'outsmart' these scammers, the better, especially ones that are adapting to how quickly these scams are evolving," Wilczynski said.

Wang said the company built the verifier around the idea that modern scams often look and sound convincing enough to pass informal checks inside households and businesses.

"The scams costing Australians the most money today aren't the obvious ones, they're the ones that pass every gut check: the right logo, the right voice, the right urgency at the right moment," he said.

"We built the AI Content Verifier because by the time something feels wrong, it's often too late to check. This gives people a way to verify before they act, not after," Wang said.