AI changed the economics of attacking. Convincing, personalised, multilingual social engineering used to take effort; now it's cheap and automated — and the 2025 numbers show it.
It helps to be clear about what actually changed, because it isn't some new class of unstoppable attack. The techniques are the same ones defenders have always faced: phishing, pretexting, impersonation. What changed is the cost curve. Crafting a flawless, personalised lure in perfect English — or any of a dozen languages — used to take a skilled human real time. Now it's a cheap, instant, infinitely repeatable API call. When the marginal cost of a convincing attack drops to near zero, attackers simply send far more of them, far better targeted.
The new threat landscape
- An estimated 80%+ of phishing now uses AI in some form, and AI-generated phishing achieves a 54% click-through rate versus 12% for traditional campaigns.
- Deepfake incidents rose ~680% year over year; deepfake-driven phishing climbed over 310% between 2023 and 2025.
- 87% of organisations report experiencing an AI-driven cyberattack in the past year; the average AI-powered breach costs $5.72 million.
The uncomfortable truth: the cheapest, most scalable use of generative AI so far has been attacking people.
That 54%-versus-12% gap is the number to sit with. The old advice — "look for the typos, the awkward grammar, the generic greeting" — was training people to spot the cheapness of the attack. AI removed the cheapness. The tells are gone. A finance clerk who once would have paused at a clumsy "Dear Valued Employee" now gets a fluent, context-aware message that references a real project, in their manager's writing style, at a plausible moment in the month.
A concrete scenario
Consider the now-classic CFO deepfake: an employee joins a video call with what looks and sounds like senior leadership, is walked through an "urgent confidential acquisition," and authorises a large transfer. Every signal a human uses to establish trust — a familiar face, a familiar voice, the social pressure of seniority — can now be synthesised. The defence cannot be "be more careful on the call," because the call itself is the attack. The defence has to live outside the channel being faked.
The same economics drive the breach figures. With 87% of organisations reporting an AI-driven attack in the past year and the average AI-powered breach costing $5.72 million, this isn't a tail risk reserved for big banks — it's the baseline threat environment for any organisation that moves money or holds data. The volume is the point: when attacks are nearly free to generate, defenders no longer face a handful of careful intrusions but a relentless, automated stream, any one of which only has to work once.
The two-sided nature of AI in security
It would be a mistake to read all this as one-directional. The same capabilities arm defenders too — AI is genuinely good at sifting enormous log volumes for anomalies, triaging alerts, and spotting patterns a human analyst would miss in the noise. The honest framing is an arms race, not a rout. But there's an asymmetry worth internalising: AI lowers the bar for attackers (anyone can now generate a flawless lure) while raising the bar for defenders (you can no longer trust your eyes and ears on high-stakes requests). Net, the burden of proof has shifted onto verification.
What this means for your team
- Assume voice and video can be faked. Add out-of-band verification — a callback to a known number, a second approver, a code word — for anything involving money or access. The verification must travel a different path than the request.
- Treat your own AI features as a new attack surface. If a model in your product reads untrusted input, it can be manipulated through it; prompt injection is the inside-the-house version of this same story.
- Use AI on defence, too. Faster detection, triage and anomaly-spotting are genuine wins — but the bar for human verification on high-stakes actions has gone up, not down.
- Retrain the humans. Drop the "spot the typo" guidance and replace it with process: high-value actions require verification regardless of how convincing the request looks.
The strategic takeaway is uncomfortable but clarifying: you can no longer rely on people detecting a fake by how it looks or sounds. Security has to move from recognition to process — verification steps that hold even when the message is perfect. If you're building AI into a product and want that attack surface reviewed properly, that's exactly the kind of work our team takes on.
The defensive mindset that holds up
The mental model that survives this shift is zero trust applied to communication. You already don't trust a network packet just because it arrived; the same skepticism now has to extend to a voice, a face, and a fluent email from a familiar name. That doesn't mean treating colleagues as adversaries — it means building processes where trust is established by procedure rather than assumed from appearance. A payment over a threshold requires a second approver through a separate channel. A change to banking details requires a callback to a number on file. Access to sensitive systems requires a verification step the attacker can't fake by impersonating someone. The point of all of these is the same: make the high-stakes action depend on something that can't be synthesised, so that even a flawless deepfake hits a wall it can't talk its way past. The organisations that internalise this won't be the ones with the cleverest detection AI — they'll be the ones whose money and access simply can't move on the strength of a convincing message alone.
Sources
- DeepStrike — AI Cyber Attack Statistics 2025