9 in 10 Attacks Start With Phishing: Can AI Save The Day? Expert Analysis

As the first AI anti-spam and anti-phishing technologies roll out, companies like Ironscales, Cofense, PhisLabs, Barracuda Sentinel, Proofpoint, SlashNext, and others take center stage.

With 9 out of 10 attacks starting with phishing, Techopedia is curious whether generative AI technologies have the potential to solve the global security problems that normally arise thanks to human mistakes.

So we sat with experts from across the emerging field to see if there is merit in artificial intelligence taking the fight to phishing.

Table of Contents Table of Contents

1. Key Takeaways
2. Advanced GenAI & Email Security
3. IronScale: The Other Side of the Coin
4. Understanding AI Anti-Phishing
5. GenAI, BEC Attacks, Cloning, and Prediction
6. When AI Goes on the Offence
7. The Bottom Line
8. References

Show Full Guide

Table of Contents

1. Key Takeaways
2. Advanced GenAI & Email Security
3. IronScale: The Other Side of the Coin
Show Full Guide
4. Understanding AI Anti-Phishing
5. GenAI, BEC Attacks, Cloning, and Prediction
6. When AI Goes on the Offence
7. The Bottom Line
8. References

Advanced GenAI & Email Security

On May 1, SlashNext announced the launch of SlashNext GenAI for Spam and Graymail. The company claims it is the industry’s first AI detection and filtering solution for spam and graymail (email that is technically legitimate and solicited, but sent widely and of different value to different recipients).

It uses large language models (LLMs) and SlashNext CEO Patrick Harr told Techopedia that this solution can detect advanced phishing, business email compromise (BEC), and spam and in many cases eliminate human error globally, which causes 90% of breaches.

“Today, there is an evolution in phishing and social engineering attacks that use GenAI to create, launch, and iterate attacks, which makes it even more difficult for humans to discern if these attacks are real or not.

“This further stresses the GenAI’s LLMs that must learn and anticipate the iteration of these attacks.

“Unlike humans, machines do not get distracted and can easily identify attacks in real-time.”

He added: “It will be an AI race war, and those security vendors that are able to stay ahead of the curve with their own LLMs for BEC, phishing, spam, and graymail will have the advantage — and in many respects, augment humans to intricately reduce the success of these attacks.”

Successful the attacks may be. On April 4, SquareX Labs researchers presented shocking findings from their investigation on top webmail providers’ security. Researchers sent known malware files from Malware Bazaar — a platform for the infosec community to study malicious files — to accounts hosted by Gmail, Gmail, Outlook, Yahoo, AOL, and Apple iCloud Mail.

To everyone’s surprise, all top webmail providers failed to block even the most known versions of malware. This naturally leads to the question; Can AI technology do a better job?

IronScale: The Other Side of the Coin

Markets and Markets reports that the phishing protection market is expected to double by 2028, from $2.2 billion in 2023 to $4.1 billion in 2028. The phishing security industry is driven by an increasingly dangerous global threat landscape, deficits in security cultures, overabundance of devices, and the rise of new technologies based on machine learning (ML) and AI.

Companies like Ironscales are seizing the momentum and expanding into new regions (U.K. and Ireland). Eyal Benishti, CEO at Ironscales spoke to Techopedia about GenAI anti-phishing tech and its potential to solve the human error vector of attack.

He said:

“AI is without a doubt one of the most powerful and promising technologies to ever enter the cybersecurity arsenal, and its potential to transform the field cannot be overstated. However, the idea of actually ‘solving’ the human error problem is still very hard to imagine — at least not anytime in the foreseeable future.

“That’s not to say these innovations aren’t without value. On the contrary, they make life much harder for the threat actors of the world, and the best of them have dramatically reduced the success rates of these would-be attackers.”

Understanding AI Anti-Phishing

Benishti from Ironscale explained the tech to us and discussed AI’s big data and analytical potential.

“One of AI’s greatest strengths is hoovering up and analyzing enormous amounts of data in order to identify patterns.

In the simplest of terms, the way AI helps guard against spam and phishing is by ‘getting to know’ its users.

“In the case of email security, AI tools will ingest thousands of past emails, and possibly other communications that you’ve written, analyze them along countless parameters (e.g., syntax, punctuation, average sentence length, word choice, emoji usage, etc.), and from that analysis, establish a kind of ‘linguistic fingerprint’, which allows it to recognize an email you’ve written as your own.

“Where this becomes valuable is in its ability to then recognize deviations from that fingerprint.

“These AI tools can recognize that that particular email did not match your usual linguistic fingerprint, and could then flag it as a potential threat.”

Like AIs and LLMs in other fields, anti-phishing goes through intensive training sessions.

“These AI-enabled tools are also trained on mountains and mountains of malicious emails, too,” Benishti said.

“All the spam messages, phishing emails, and other kinds of malicious communications are ingested and analyzed by the AI so that it can more reliably identify and isolate these dangerous messages before they even reach your inbox.”

But training is not a once-and-done deal. To avoid ML model drifting (due to outdated key features or values in the algorithms), and to be able to respond to new emerging threats, companies invest in continual and continuous training.

“Perhaps most importantly, this is not a static process. These tools undergo said training continuously, ensuring they are able to recognize even the latest, most sophisticated threats as they begin to emerge.”

Traditional Phishing Security Vs. GenAI Anti-Phishing

Josh Amishav Founder and CEO at Breachsense told Techopedia that LLMs, like those used in generative AI for anti-spam, offer significant advantages over traditional methods by being able to understand and generate human-like text. This helps them better identify subtle nuances and variations in phishing tactics.

“Unlike rule-based systems, LLMs can generalize from past data to recognize and react to previously unseen types of spam and phishing attacks. This adaptability allows them to continuously improve their detection accuracy via continuous training to update their models.

“Generative AI will certainly enhance anti-spam and phishing protection by leveraging predictive capabilities and natural language understanding to better identify and block sophisticated threats,” Amishav said.

“Having said that, there are a number of techniques threat actors could use to evade detection.

“LLMs are only as good as the models they’re trained on. If attackers can poison the training model or develop strategies that bypass the model’s ability to properly classify the attack then the effectiveness of these systems could be compromised.”

GenAI, BEC Attacks, Cloning, and Prediction

SlashNext claims unparalleled accuracy with GenAI. Techopedia asked Harr to elaborate on the specific technical advantages of using LLMs compared to traditional anti-spam methods.

He said:

“A key benefit of using genAI for plain text/BEC attacks and now for the identification of targeted Spam and graymail emails is the unique ability for the models to automatically learn writing styles, tone, and intent of the email attacks and then iterate (create email clones that maintain the intent of the attack but written differently) and predict the next email versions before these new email attacks are launched.

“This approach dramatically lowers false positive rates (extreme accuracy of 1 in 1M false posiives) and exponentially expands training data sets to anticipate the next iteration of the attack before it’s launched,” Harr added.

“For example, SlashNext GenAI extracts the email body (aka the threat) as well as opens the attachment and extracts the banking information. The GenAI engine then creates email “clones” or different versions of that same email attack.”

Harr said this leads to increased accuracy and the AI can even predict the next move an attacker will make by understanding iterations.

From Black-Lists to Content Analysis and Sophisticated Attacks

Benishti from Ironscales explained that traditional anti-spam and anti-phishing technologies actually follow similar principles, however with much less sophistication, and most importantly, without the ability for continuous adaptation.

“At their simplest, traditional email filters block spam and phishing attempts by comparing all incoming mail to what is essentially a “black list” — or a living inventory of known malicious features.”

These traditional tools will monitor and scan emails for known data such as email addresses, subject lines, links, file names (as attachments), and known malware. These tools are able to block dangerous emails based on some very basic, core attributes and content.

He added:

“Modern spam filters will also use some language analysis on an email’s content in order to filter out messages with tell-tale signs of spam — rampant typos, awkward phrasing, strange formatting, references to long lost fortunes and royal heirs…you get the idea.

“Don’t get me wrong, all of these capabilities are incredibly helpful.

“However, they have not been able to keep up with the increasing variety and sophistication of email-based attacks — things like business email compromise (BEC), VIP impersonation, account takeover, image-based attacks like QR-code phishing, and so on.”

As these types of attacks become more widespread — and hackers make increasing use of generative AI themselves —- these types of traditional defenses are simply no longer adequate.

When AI Goes on the Offence

Techopedia asked Benishti what were the most innovative and promising technologies emerging on the horizon.

“The idea is for organizations to use generative AI to generate malicious emails and other communications, which can then be used to proactively battle test and train their own defenses.”

Benishti spoke about the importance of Red Team vs Blue Team attack and defense simulations, concepts linked to ethical hackers, and penetration testers.,

“As more and more bad actors begin to leverage AI for their own malicious ends, there’s very good reason to believe engaging in this type of continuous, AI-driven optimization will become a security necessity, sooner than later.”

The Bottom Line

For years, the security industry has told the world — over and over — that human error is the biggest vector of attack for all types of cyber incidents, from ransomware to frauds and scams, and data exfiltration.

However, should the burden of responsibility fall solely on users? As the underground international cybercriminal world ups the ante and embraces new technologies, the world stands at a crossroads, with AI currently on the side of the attackers.

It can either keep investing in breakthrough AI anti-phishing technology or continue down the “business-as-usual” path.