Sophos Anticipates AI-Based Attack Techniques

Sophos, a provider in innovating and delivering cybersecurity as a service, today released two reports about the use of AI in cybercrime. The first report—“The Dark Side of AI: Large-Scale Scam Campaigns Made Possible by Generative AI”—demonstrates how, in the future, scammers could leverage technology like ChatGPT to conduct fraud on a massive scale with minimal technical skills. However, a second report, titled “Cybercriminals Can’t Agree on GPTs,” found that, despite AI’s potential, rather than embracing large language models (LLMs) like ChatGPT, some cybercriminals are skeptical and even concerned about using AI for their attacks.

The Dark Side of AI

Using a simple e-commerce template and LLM tools like GPT-4, Sophos X-Ops was able to build a fully functioning website with AI-generated images, audio, and product descriptions, as well as a fake Facebook login and fake checkout page to steal users’ login credentials and credit card details. The website required minimal technical knowledge to create and operate, and, using the same tool, Sophos X-Ops was able to create hundreds of similar websites in minutes with one button.

“It’s natural—and expected—for criminals to turn to new technology for automation. The original creation of spam emails was a critical step in scamming technology because it changed the scale of the playing field. New AIs are poised to do the same; if an AI technology exists that can create complete, automated threats, people will eventually use it. We have already seen the integration of generative AI elements in classic scams, such as AI-generated text or photographs to lure victims.

“However, part of the reason we conducted this research was to get ahead of the criminals. By creating a system for large-scale fraudulent website generation that is more advanced than the tools criminals are currently using, we have a unique opportunity to analyze and prepare for the threat before it proliferates,” said Ben Gelman, senior data scientist, Sophos.

Cybercriminals Can’t Agree on GPTs

For its research into attacker attitudes towards AI, Sophos X-Ops examined four prominent dark web forums for LLM-related discussions. While cybercriminals’ AI use appears to be in its early stages, threat actors on the dark web are discussing its potential when it comes to social engineering.  Sophos X-Ops has already witnessed the use of AI in romance-based, crypto scams.

In addition, Sophos X-Ops found that the majority of posts were related to compromised ChatGPT accounts for sale and “jailbreaks”—ways to circumvent the protections built into LLMs, so cybercriminals can abuse them for malicious purposes. Sophos X-Ops also found ten ChatGPT-derivatives that the creators claimed could be used to launch cyber-attacks and develop malware. However, threat actors had mixed reactions to these derivatives and other malicious applications of LLMs, with many criminals expressing concern that the creators of the ChatGPT imitators were trying to scam them.

“While there’s been significant concern about the abuse of AI and LLMs by cybercriminals since the release of ChatGPT, our research has found that, so far, threat actors are more skeptical than enthused. Across two of the four forums on the dark web we examined, we only found 100 posts on AI. Compare that to cryptocurrency where we found 1,000 posts for the same period.

“We did see some cybercriminals attempting to create malware or attack tools using LLMs, but the results were rudimentary and often met with skepticism from other users. In one case, a threat actor, eager to showcase the potential of ChatGPT inadvertently revealed significant information about his real identity. We even found numerous ‘thought pieces’ about the potential negative effects of AI on society and the ethical implications of its use. In other words, at least for now, it seems that cybercriminals are having the same debates about LLMs as the rest of us,” said Christopher Budd, director, X-Ops research, Sophos.

Read more exclusives and news in our latest issue here.

Never miss a story… Follow us on:
LinkedIn Security Buyer
Twitter logo @SecurityBuyer
Facebook @Secbuyer

Media Contact
Rebecca Morpeth Spayne,
Editor, Security Portfolio
Tel: +44 (0) 1622 823 922
Email: [email protected]

About Security Buyer

Security Buyer is the leading authority in global security content, delivering expert news, in-depth articles, exclusive interviews, and industry insights across print, digital, and event platforms. Published 10 times a year, the magazine is a trusted resource for professionals seeking updates and analysis on the latest developments in the security sector.

To submit an article, or for sponsorship opportunities, please contact our team below.

Rebecca Spayne picture 2025

Rebecca Spayne

Managing
EDITOR

Georgina Turner image

Georgina Turner

Sales
Manager

Afua Akoto image - Security Buyer

Afua Akoto

Marketing Manager

Read the Latest Issue

Follow us on X

Follow us on X

Click Here

Follow us on LinkedIn

Follow us on LinkedIn

Click Here

Advertise here

Reach decision makers and amplify your marketing

Advertise here

Click Here

Related News

AI’s Digital Pollution

AI’s Digital Pollution

As AI continues to shape industries, responsible AI governance remains a pressing concern. Yolanda Hamblen and Pauline…
Human oversight in the age of AI security image - Security Buyer

Human oversight in the age of AI security

Alex Kazerani, SVP of Cloud Video Security & Access Control, Motorola Solutions explores the power of AI in video security.
Copyright: Security Buyer

ASIS UK Launches “Security is You(th)” Hackathon

ASIS International UK has launched Security is You(th), an initiative designed to engage students and early-career professionals…
Image provided by Veeam

AI and Ransomware: Cutting Through the Hype

Rick Vanover, Vice President Product Strategy, Veeam discusses how It might be the great paradox: Artificial Intelligence (AI)….
ASIS Europe

Genetec to separate AI hype from reality at ASIS Europe 2025

Genetec has announced its plans for ASIS Europe 2025, taking place in Dublin, Ireland from 4-6 March. Its focus will be on the future..
Copyright: Security Buyer

AmiViz Partners with Titania

AmiViz announced a strategic distribution agreement with Titania. This collaboration underscores a shared commitment to enhancing…
Copyright: Security Buyer

Facial Recognition: Innovation vs. Accountability

Facial recognition technology is advancing with AI, IoT, and privacy-first security, but regulatory compliance, ethical AI, accountability…
Malik Alyousef, Co-founder & COO, Mozn

Mozn Unveils a New Generation of AI Fraud Prevention

FOCAL by Mozn strengthens its Fraud Prevention Suite with Device Fingerprinting, Fraud Analytics, and Fraud Management as a Service…
Hospitality

The AI Shift in Hospitality Security

Hannah Larvin assesses how AI-driven security in hospitality enhances surveillance, access control, and incident response, balancing safety..
Rhombus

Rhombus Launches AI Capabilities for Faster, Smarter Security Investigations

Rhombus, a provider of cloud-managed physical security, today announced the launch of three new additions to its AI portfolio that transform
Scroll to Top