Humans are still better than AI at crafting phishing emails, but for how long?

Humans are still better at crafting phishing emails compared to AI, but not by far and likely not for long, according to research conducted by IBM X-Force Red.

Creating phishing emails: Humans vs. AI

The researchers wanted to see whether ChatGPT is as capable of writing a “good” phishing email as attackers are.

“As someone who writes phishing emails for a living, I was excited to find out the answer,” said Stephanie Carruthers, Chief People Hacker for IBM X-Force Red.

The researchers experimented with various prompts and refined them to finally come up with a list of just five that were enough for ChatGPT to produce the phishing email.

First they asked ChatGPT to define top areas of concern for employees within a specific industry. Then they instructed it to choose the social engineering and marketing techniques more likely to maximize the phishing email’s effectiveness.

After prompting ChatGPT to chose the ideal sender to impersonate, they instructed it to draft the phishing email.

AI-generated phishing email. (Source: IBM X-Force Red)

Concurrently, X-Force Red social engineers crafted their own phishing email, after using OSINT techniques to gather information that would help them choose the sender to impersonate and decide on the lure (an internal company survey).

But – and this is important – it took five minutes for ChatGPT to generate the email, while the researchers spent 16 hours on the task.

The final test

Each email – the AI-generated one and the one written by the social engineers – was sent to over 800 employees at a global healthcare organization.

“After an intense round of A/B testing, the results were clear: humans emerged victorious but by the narrowest of margins,” Carruthers said.

Humans can understanding emotions and are thus more successful at weaving narratives that tug at the heartstrings and sound more realistic, she explained.

They are also better at personalizing emails and writing subject lines that are less likely to trigger recipients’ suspicion. (Case in point: the AI-generated phishing email had a higher report rate than the human-crafted one.)

“Armed with creativity, and a dash of psychology, these social engineers created phishing emails that resonated with their targets on a personal level. The human element added an air of authenticity that’s often hard to replicate,” Carruthers concluded.

“Humans may have narrowly won this match, but AI is constantly improving. As technology advances, we can only expect AI to become more sophisticated and potentially even outperform humans one day. As we know, attackers are constantly adapting and innovating. Just this year we’ve seen scammers increasingly use voice clones generated by AI to trick people into sending money, gift cards or divulge sensitive information.”