#

Compromised Email + Copilot: AI-assisted BEC Accelerating Internal Reconnaissance, Writing Style Imitation, AiTM, and Wire Fraud

#

1. Basic Information

Article Title: AI-enabled email accounts: The next insider threat #

Publisher: Barracuda Networks #

Publication Date: August 4, 2026 #

Original Source: Barracuda Networks #

Related Information Source: SecurityWeek #

Related Products: Microsoft 365 Copilot, Exchange Online-equivalent mailboxes, AiTM phishing proxies #

Attack Status: Researcher lab simulation. Not a real-world damage case yet. #

Severity: Medium

#

2. One-Sentence Summary

This is a PoC where an attacker uses Copilot within an already compromised regular employee's mailbox as a "well-informed accomplice" to quickly learn about organizational relationships, confidential threads, and writing styles, steals a CEO session via AiTM, and instructs a change of bank account details from the legitimate CEO mailbox.

#

3. Attack Flow

Chain: Escalating from a Regular Employee's Mailbox to the CEO and Finance

• The attacker compromises a regular employee's mailbox session (initial access is out of scope for the study).
• The attacker asks Copilot to create an inbox rule that moves sign-in notification emails to the Deleted Items folder.
• The attacker has Copilot summarize the organizational structure and important/confidential conversations.
• The attacker identifies real email threads and business contexts involving the CEO.
• The attacker drafts an invoice confirmation email using the victim employee's writing pattern.
• The attacker sends the email to the CEO from the legitimate internal mailbox.
• The CEO clicks the link and connects to the AiTM proxy.
• The attacker steals the credentials and the MFA-authenticated session token.
• The attacker logs into the CEO's mailbox and hides traces using a similar mailbox rule.
• The attacker has Copilot summarize invoices, amounts, and scheduled transfers.
• The attacker identifies an ongoing wire transfer of $247,500 USD.
• The attacker sends a bank account change email to the finance department using the CEO's writing style.
• The attacker forwards replies from finance to an external attacker email and hides them from the CEO.
• The attacker uses Copilot to search for and delete related emails before exiting.

#

4. Attacker Position and Execution Location

Attacker: Operates legitimate cloud email and Copilot from a compromised mailbox/session. #

AI Execution Location: Inside the Microsoft 365 tenant, searching and generating content within the scope of the target user's permissions. #

AiTM: Attacker-controlled reverse proxy. #

Subsequent Execution: CEO mailbox, finance workflow. #

Endpoint Malware: Not required.

#

5. Visibility for Victims and Administrators

To the CEO and finance department, the email appears legitimate because it comes from a valid internal address, uses the correct conversational context, matches the sender's writing style, and references an actual transaction name. SPF, DKIM, and DMARC checks pass normally, making traditional email gateway detection difficult. However, activities such as the creation of mailbox rules, Copilot's rapid and broad summarization, AiTM sessions, forwarding rules, and deletion operations can be observed through identity and mailbox audits.

#

6. Success and Failure Conditions

Success Conditions

• Compromising a regular employee's mailbox/session first.
• Copilot having broad access to mailbox and organization information.
• The attacker being able to create rules, send emails, and delete items.
• The CEO clicking the AiTM link and handing over the session token.
• The finance department processing the bank change without out-of-band verification.

Failure Conditions

• Phishing-resistant MFA, token binding, and conditional access.
• Stopping the initial account due to impossible travel, device, or session anomalies.
• Immediate alerts and approvals for inbox/forwarding rules.
• Auditing Copilot prompts/tool calls and detecting bulk sensitive queries.
• Verifying bank detail changes via phone, dual approval, or known contacts.

#

7. What Happens Upon Success

The CEO's session, financial information within the mailbox, and organizational relationships are compromised. Business email compromise (BEC) and wire fraud are then executed from the legitimate mailbox. The AI does not create new permissions, but it accelerates exploration within existing permissions, writing style imitation, target selection, and cleanup.

#

8. Observable Logs

Email

• Inbox rules moving sign-in

alerts and similar emails to Deleted Items.

• Internal emails addressed to the CEO/finance requesting bank account changes.
• Rules forwarding finance sender emails to external attacker addresses.
• Bulk search and deletion of related emails.

Proxy / SWG / DNS

• Access from invoice links to AiTM domains.
• Proxy redirects of CEO devices/browsers.

Endpoint / EDR

• No malware processes required.
• Browser history, session cookies, AiTM access.

Identity / IdP

• New devices, IPs, or sessions; token replays.
• Use of different locations/sessions immediately after successful MFA.
• Abnormal logins to CEO accounts and mailbox operations.

SaaS / Cloud

• Copilot prompt/activity logs (if available).
• Consecutive queries for organizational charts, sensitive threads, and financial emails.
• Mailbox rule creation/updates, message moves/deletes, and forwarding.

Network

• Network-only detection is limited due to reliance on legitimate M365 traffic.
• Connections to AiTM infrastructure serve as the primary external signal.

#

9. Attack Success Determination

Contact Only: Delivering internal phishing emails to the CEO. #

User Action: The CEO clicking a link and authenticating via AiTM. #

Initial Execution: Copilot queries/rule creation in a regular employee's mailbox (cloud action). #

Authentication Success: Logging in using the CEO's session token. #

Data Theft / Session Compromise: Summarizing financial emails, replaying the CEO token. #

Subsequent Compromise Confirmation: Bank change emails, forwarding rules, executing wire transfers, deleting evidence.

#

10. Investigation Playbook

Trigger

• Abnormal sessions on AI-enabled accounts.
• Inbox rules or internal phishing created immediately after Copilot usage.
• Requests for bank detail changes.
• Forwarding rules or automatic deletion of sign-in emails.

Initial Verification

• Preserve emails, headers, rules, Copilot audits, and sign-in/token logs.
• Identify the initially compromised account and the first abnormal session.
• Fully export threads related to the CEO and finance.

Devices

• Browsers/devices that opened AiTM links, cookies, and history.
• Cross-reference access times to phishing pages with IdP logs.

Authentication & Cloud

• Revoke sessions, invalidate tokens/cookies, and reset passwords.
• Check consent, MFA methods, device registrations, mail rules, and delegates.
• Investigate the scope of Copilot prompt/tool actions.

Subsequent Operations

• Emails sent to executives other than the CEO, finance, or vendors.
• Bank account changes, invoices, and deleted/recoverable items.
• Cross-search for external forwarding destinations using the same domain.

Containment

• Suspend affected accounts/sessions.

• Delete malicious rules/forwarding after preserving evidence.

• Contact banks to stop and recover wire transfers.

• Notify business partners out-of-band.

Determination Categories

• Mailbox Compromise Suspected
• AI-assisted Recon Confirmed
• Internal Phish Delivered
• Executive Session Hijacked
• BEC/Fraud Attempted
• Financial Loss Confirmed

#

11. Defense and Detection Ideas

Single Events

• Rules deleting sign-in notifications.
• External forwarding of finance emails.
• Large volumes of sensitive-mail queries from Copilot in a short time.

Timeline Correlation

New session → Copilot org/thread query → Stealth rule → Internal phish → AiTM → CEO token replay → Financial query → Bank change → Forward/delete.

Hunting Perspectives

• Mailbox rules created shortly after starting to use the AI assistant.
• Deep organization/finance queries by users who do not normally use Copilot.
• Invoice links coming from internal senders pointing to new domains.
• New sessions in the CEO's mailbox combined with finance forwarding.

Log Gaps

• Copilot prompt/tool-call audits.
• Semantic categorization of mailbox searches/queries.
• Session token binding and device posture.
• Rule changes before and after events.

Priority Countermeasures

• Prevent initial account takeovers and implement phishing-resistant MFA.

• Real-time alerts for mailbox rules and forwarding.

• Integrate Copilot access and operation logs into the SIEM.

• Continuous risk assessment of AI-enabled privileged accounts.

• Out-of-band two-person verification for bank detail changes.

#

12. Facts / Inference / Hypothesis

Facts

• Barracuda conducted a PoC in a lab setting using a compromised mailbox and Copilot.
• Copilot was used to extract organizational relationships, real threads, writing styles, and financial emails.
• Assuming the theft of a CEO session via AiTM, bank account changes were executed from the legitimate CEO mailbox.
• This was a simulation, not an observed real-world attack.

Inference

• AI assistants significantly shorten reconnaissance and BEC preparation times after account compromise, even without granting new permissions.
• Effective defense requires correlating identity, mailbox actions, and AI activities, rather than relying solely on email gateway content filtering.

Hypothesis

• Correlating the semantic classification of Copilot queries with mailbox rule changes over a short timeframe can help distinguish between normal business operations and post-compromise reconnaissance.

#

13. MITRE ATT&CK Mapping

High Confidence

T1114.002 Remote Email Collection #

T1098.002 Additional Email Delegate Permissions / Rule abuse (conceptually similar) #

T1564.008 Email Hiding Rules #

T1539 Steal Web Session Cookie #

T1656 Impersonation #

T1566.002 Spearphishing Link #

T1078.004 Cloud Accounts

Medium Confidence

T1589 Gather Victim Identity Information (Post-compromise internal recon) #

T1530 Data from Cloud Storage Object (Approximation of mailbox/Copilot access) #

T1070.008 Clear Mailbox Data

• Direct abuse of AI assistants is difficult to represent directly in ATT&CK.

#

14. Unknowns and Additional Investigation

• Granularity of Copilot audits available in production environments.
• Retention periods for prompts and mailbox actions.
• The extent to which token binding can halt AiTM chains.
• Differences in tenant policies regarding AI assistants creating rules.
• Adoption examples in real-world attacks.
• Accuracy of writing style imitation and its detectability.

#

15. Impact on SOCs

Organizations face significant risks from internal BEC targeting executives and finance departments, alongside partner account modifications. Evaluating Copilot deployments should go beyond data leakage prevention; organizations need designs that can track—across identity, mail, and SaaS—how a compromised account uses AI to target individuals and manipulate operations.

#

16. Summary

For SOC Teams

• Correlate Copilot usage, mailbox rules, internal phishing, and token replayed events chronologically.
• Treat bank changes and AiTM links as high priority, even when sent via legitimate internal emails.
• Distinguish between the simulation nature of the scenario and the specific behaviors that actually succeeded.

For Administrators

• Set up Copilot audits, mailbox rule alerts, and phishing-resistant MFA.
• Restrict AI access and forwarding on privileged mailboxes.
• Mandate out-of-band verification for bank account changes.

For End Users

• Always verify links, authentication requests, and bank account changes through a separate channel, even if emails appear to come from company executives or colleagues.

source & further reading

dev.to — original article I Audited My AI's To-Do List. A Quarter of It Was Already Done. Prompt engineering couldn't fix this LLM bug. 20 lines of binary parsing did. Beyond RAG: Building an AI Coding Agent with Planning, Tool Execution, and ReAct Reasoning