Microsoft details AI-powered defences against novel SVG phishing

Microsoft has detailed the mechanisms behind an AI-driven phishing campaign that was detected and blocked by its Threat Intelligence team, highlighting both attacker use of synthetic code as well as the role of AI-enabled defences.

The phishing operation, observed in August, involved attackers disguising malicious SVG files as PDFs. These files appeared visually similar to genuine business dashboards, leveraging specific terminology and structured layouts. According to Microsoft, this heightened level of detail and complexity was one indication used by Security Copilot, the company's AI-driven analysis tool, to flag the content as likely AI-generated.

Microsoft Defender for Office 365 ultimately blocked the campaign, relying on AI-powered protections that analyse infrastructure, behavioural cues, and message context to stop threats even when they are designed to mimic legitimate communications.

Campaign analysis

The campaign was primarily aimed at organisations in the United States and was discovered when Microsoft Threat Intelligence identified suspicious activity from a compromised small business email account. Attackers employed a self-addressed email tactic-in which sender and recipient matched while actual targets were included in the BCC field-to attempt bypassing standard detection heuristics.

The attack featured an attached file labelled "23mb - PDF- 6 pages.svg." Although it appeared to be a standard PDF document, the actual file type was SVG (Scalable Vector Graphics), a text-based and scriptable format that allowed attackers to embed JavaScript and dynamic content. SVG files in this context are attractive to cybercriminals because of their capacity for obfuscation, such as through invisible elements, encoded attributes and delayed script execution, making them challenging for traditional security tools to identify.

On activation, the SVG file redirected the recipient to a webpage requesting CAPTCHA completion-a common tactic designed to create trust and delay suspicion. Microsoft's analysis indicates the campaign would likely have then presented a counterfeit sign-in page in order to harvest user credentials.

Inspection of the SVG code demonstrated unique obfuscation techniques. The initial SVG content was structured as a facsimile of a business analytics dashboard, with elements rendered invisible via opacity and transparency settings. Microsoft reported that, "Within the file, the attackers encoded the malicious payload using a long sequence of business-related terms. Words like revenue, operations, risk, or shares were concatenated into a hidden data-analytics attribute of an invisible <text> element within the SVG."

This payload was subsequently decoded by embedded JavaScript, reconstructing malicious functions by mapping pairs or sequences of these business-related terms to specific instructions. The methodologies employed included user redirection, browser fingerprinting, and session tracking.

Determining AI involvement

Microsoft hypothesised that AI tools had been used to generate the obfuscated SVG code, and Security Copilot was deployed to assess this. In its analysis, Security Copilot noted:

Security Copilot's analysis indicated that it was highly likely that the code was synthetic and likely generated by an LLM or a tool using one. Security Copilot determined that the code exhibited a level of complexity and verbosity rarely seen in manually written scripts, suggesting it was produced by an AI model rather than crafted by a human.

Security Copilot highlighted several distinguishing factors, including the use of overly descriptive and redundant naming conventions incorporating random hexadecimal strings, highly modular and systematic code structure, verbose documentation in generic business language, formulaic obfuscation methods, and inclusion of unnecessary technical elements such as XML declarations and CDATA sections-patterns considered typical of large language model (LLM) outputs.

Defensive use of AI

According to Microsoft, while AI allows for increased sophistication in attacker methodology, it does not fundamentally alter the core indicators used by security systems to identify phishing. Microsoft Defender for Office 365 employs machine learning and AI to identify multi-dimensional patterns such as suspicious domain behaviour, use of redirects, impersonation strategies, and anomalous message delivery patterns, including self-addressed emails and use of the BCC field.

Microsoft also noted that AI-generated code, while potentially more complex, "still operates within the same behavioral and infrastructural boundaries as human-crafted attacks." The presence of verbose naming, redundant logic and synthetic encoding schemes resulting from AI-led obfuscation can become additional detection signals for AI-powered defences.

The specific signals that flagged the detected campaign included the use of self-addressed emails with BCC hiding targeted recipients, suspicious file types and naming conventions (SVG files masquerading as PDFs), redirection to recognised malicious domains, employment of novel code obfuscation, and observed suspicious network activity such as browser fingerprinting and session tracking.

A spokesperson from Microsoft Threat Intelligence commented, "By sharing our analysis, we aim to help the security community recognize similar tactics being used by threat actors and reinforce that AI-enhanced threats, while evolving, are not undetectable. As we discuss in this post, an attacker's use of AI often introduces new artifacts that can be leveraged for detection. By applying these insights and our recommended best practices, organizations can strengthen their own defenses against similar emerging, AI-aided phishing campaigns."

Recommendations and ongoing advice

Microsoft has recommended organisations review security settings for Exchange Online Protection and Defender for Office 365, enable rechecking of links through Safe Links, use Zero-hour auto purge capability, and ensure cloud-delivered protections are activated. Adoption of phishing-resistant authentication methods and enhanced browser security via Defender SmartScreen are also endorsed.

The company has stated that, "While this campaign was limited in scope and effectively blocked, similar techniques are increasingly being leveraged by a range of threat actors. Sharing our findings equips organizations to identify and mitigate these emerging threats, regardless of the specific threat actor behind them."

In addition, Microsoft encourages the security community to stay alert for evolving AI-driven attack methodologies, noting that the ongoing arms race between attackers and defenders will continue to centre on the respective application of artificial intelligence.