Please fill in the form below to subscribe to our blog

The Dangers of AI-Driven Chat GPT Phishing Campaigns and How to Avoid Them

July 06, 2023

AI is a Revolutionary Technology for Cybercrime Too


The modern cybercriminal is extremely inventive in their approach to disrupting a business’s operations. They have access to a variety of resources that help them execute cyberattacks, and that list grows daily as technology evolves. They were already successful in scamming 92% of organizations across the globe in 2022 using sophisticated techniques like creative phishing emails, spoofing and fraudulent websites. Some even fell victim to phishing and social media fraud as well. Now bad actors have a new addition to their arsenal: ChatGPT phishing powered by artificial intelligence (AI).


a young, bearded white man in a dress shirt looks pensively at charts on a computer monitor

See the keys to selecting a Managed SOC to find the perfect one for your clients & your MSP. GET CHECKLIST>>


AI is revolutionary for cybercrime too


Recently, AI-based technology has gained a lot of traction. AI-enabled tools and technologies can reduce workloads and eliminate mundane tasks, making them highly desirable for everyone. ChatGPT, for instance, is a large language model (LLM) that has helped millions of people maximize efficiency and easily achieve their goals. Unfortunately, it has also helped cybercriminals launch attacks using new techniques like ChatGPT phishing.

On the flip side, as with many innovations built to better humankind’s way of life, some discover how something inherently designed for good can be leveraged for personal or malicious agendas. Cybercriminals also find AI-enabled tools very successful, and they’re using them to launch even more sophisticated, hard-to-detect cyberattacks. This change adds a new wrinkle to securing a business from cyberattacks.


an ominously dark image of a hacker in a blue grey hoodie with the face obscured.

Explore the nuts and bolts of ransomware and see how a business falls victim to an attack. GET EBOOK>>


How AI assists cybercriminals


Cybercriminals have already begun employing AI-based methodologies to launch cyberattacks and have, unfortunately, been effective in their attempts to exploit organizations. Even those criminals not particularly skilled in technical work like developing ransomware can launch cyberattacks using AI tools like ChatGPT for reference or support. Researchers have discovered that underground hacking teams frequently utilize OpenAI for its quick code generation and email writing capabilities. Research has shown that ChatGPT phishing emails are so well-orchestrated that employees find it difficult to discern from content written by a human. 

The use of AI technology like Chat GPT or GPT-3 has been a game-changer for bad actors, especially those who specialize in phishing. Generative AI is lowering the barrier to entry into cybercrime by enabling threat actors to quickly and effectively launch sophisticated phishing messages and do the work needed to facilitate a ransomware attack easily. This technology paired with the tools available in the Cybercrime-as-a-Service (CaaS) economy and the information like passwords that is readily available from initial access brokers (IAB) makes cybercrime easier – and that’s bad news for businesses.


Find out how Datto EDR helps with Health Insurance Portability and Accountability Act (HIPAA) compliance. GET INFO>>


How does AI benefit cybercriminals?


AI enables cybercriminals to quickly launch attacks, use more sophisticated techniques and improve their effectiveness. These features and more make AI a very attractive technology for any cybercrime organization. 

Flexible and adaptable for better accuracy

Threat actors use new and continuously evolving AI techniques to scale and automate processes, like code generation, to better plan an attack for the highest chance of success. They can also improve on existing cyberattacks or create new ones. The power of automation empowers them to carry out large-scale cyberattacks without having to actually spend time manually building out algorithms. It speeds up the process of detecting vulnerabilities or learning about which employees are most susceptible to manipulation.

AI boosts evasion

Not getting caught is an important skill that not all cybercriminals possess. And considering the progress information security professionals have made in developing cybersecurity measures, hackers need to be more evasive. Technologies such as machine learning (ML) allow cybercriminals to train AI systems to recognize and adapt to companies’ security solutions and practices, which spells trouble for IT teams everywhere. AI-powered attacks can learn and evolve from their interactions with defensive systems, constantly adapting their strategies to avoid detection and improve a cybercriminal’s success rate.

Upgrade a cybercrime group’s capability

This particular upgrade any cybercriminal may achieve can prove to be incredibly resourceful. AI allows them to become more organized or systematic when determining or assessing targets. Threat actors can become extremely efficient in reconnaissance and targeting. They can quickly develop and use sophisticated AI algorithms to analyze vast amounts of data, such as social media profiles and personally identifiable information (PII), to identify potential victims with precision. AI can also be used to deconstruct communication patterns amongst colleagues and management in an enterprise-wide network, fueling the creation of highly personalized, persuasive phishing and other similar targeted attacks. In this scenario, Chat GPT phishing is an attractive option because if a bad actor has plenty of data to feed to the algorithm, it makes it easy for them to obtain a tempting malicious message.


dark web threats represented by a hacker in a hoodie shrouded in shadows with faint binary code

Find out about five of today’s biggest dark web threats to businesses in this infographic. DOWNLOAD IT>>


Bad actors are using AI to compromise businesses via phishing


GPT-3 phishing and other email-based cyberattacks have become commonplace with high success rates, and AI will only make them more successful. It’s only fair to assume that cybercriminals would seek to capitalize on an attack strategy that works. And powering their social engineering attacks with AI has increased their effectiveness.

Phishing emails

Cybercriminals are constantly innovating new ways to supercharge their phishing campaigns, creating a whole library of more deceptive and indistinguishable emails that can fool the best of us. Case in point, an employee at FT Labs of the Financial Times who maintains significant technical expertise fell prey to a well-crafted phishing email, which resulted in a data breach for the organization. This case study displays how even those who otherwise maintain healthy cyber hygiene can become victims of a creative cybercriminal.

Developing countermeasures and relevant software to curtail the adverse effects of phishing emails has become quite a task due to AI’s intervention. Numerous organizations have suffered significant financial losses after a successful phishing attack as customers lost faith in their ability to implement robust email security practices.


Get tips & advice to help you build a smart incident response plan in our guide. GET YOUR GUIDE>>


The latest AI-powered techniques enable threat actors to craft the most convincing emails to date. Some employees may even become accidental actors and disclose sensitive information or install malware without even realizing it as part of these sophisticated schemes. The advent of highly believable malicious emails is a disaster because most of today’s most devastating cyberattacks start with phishing. In a recent study, researchers determined that 85% of all successful account takeover (ATO) attacks were launched with a phishing email.

One of the biggest hallmarks of a phishing email is sloppy grammar and poor language usage. But with ChatGPT phishing, the language used in malicious emails is immaculate, eliminating one of the easiest ways for an employee to spot phishing. Technologies like ChatGPT can generate well-structured emails with flawless and easy-to-digest grammar on any prompt, making Chat GPT phishing a major danger that businesses need to be concerned about. These emails can be refined repeatedly to train ChatGPT and obtain different styles and approaches to writing a new email. What’s more? Foreign scammers can draft emails in their respective languages and have the AI tool translate them into English effortlessly.

Phishing websites

Another handy way cybercriminals leverage AI-driven tools revolves around automating the process of creating phishing websites. They can:

  • Automate the creation and customization of phishing websites
  • Make them appear authentic
  • Increasing their chances of successfully tricking victims

Threat actors can take their website replicating skills to the next level using AI-driven techniques. They can nail the visual elements, layout and content of genuine websites, increasing the chances of successfully scamming individuals. Cybersecurity professionals have their hands full as the attention to detail in creating each of these phishing websites is exceptionally intricate.


Learn to defend against devastating cyber threats with A Comprehensive Guide to Email-based Cyberattacks. GET IT>>


Combating AI-powered cyberattacks


Adopting the ‘fighting fire with fire’ concept, information security professionals are also utilizing AI technology to develop strategies and solutions to fend off cybercriminals. Generative AI helps analyze and detect patterns in phishing emails, identifying subtle indicators of fraudulent activity that human eyes can easily miss. When used in an email security solution, AI/ML helps that tool draw a clear distinction between legitimate and malicious communication. With the rapid evolution of cybercrime technology, companies need to have a plan like improving their email security with AI in place now to stop ChatGPT phishing.

Fight back against all kinds of phishing including AI-powered Chat GPT phishing with a strong training program to make sure every employee is aware of the latest phishing techniques and alert to danger. Powerful email security that uses AI to spot and stop malicious messages is also a must-have.

Learn more about how AI stops phishing cold in this infographic. GET INFOGRAPHIC>>


dark web threats represented by a hacker in a hoodie shrouded in shadows with faint binary code

Find out about five of today’s biggest dark web threats to businesses in this infographic. DOWNLOAD IT>>


Give phishing the 1 -2 punch with BullPhish ID & Graphus


The Kaseya Security Suite offers IT professionals the tools they need to keep businesses safe and compliant with cyber insurance requirements.

Stop phishing attacks with Graphus

When looking to safeguard your organization from AI-powered email cyberattacks, there are only a handful of solutions out there that can get the job done.

Graphus automated AI-driven email security places three powerful layers of protection between your employees and phishing.

Graphus catches more phishing messages than a secure email gateway or legacy email client alone. IT professionals can save time and gain peace of mind that their organization is protected from email-based threats like business email compromise (BEC), ATO and ransomware attacks.

Eliminate human error with BullPhish ID

Employees are a business’s first line of defense against cyberattacks. But creating and launching a training program can be a daunting task.

BullPhish ID enables IT professionals to run effective security awareness training campaigns and phishing simulations easily. Schedule automated delivery through each user’s individual portal that racks their progress and more for hassle-free, set-it-and-forget-it training.

BullPhish ID & Graphus make training even easier with Drop-A-Phish

This integration empowers IT technicians to do away with the process of manual domain whitelisting while granting them the ability to deliver phishing and training emails directly to each employee, saving so many precious hours of tech time. Learn more in this info sheet. GET INFO SHEET>>

Book a demo of our anti-phishing solutions and bolster your email security fast.


dark web threats

Read case studies of MSPs and businesses that have conquered challenges using Kaseya’s Security Suite. SEE CASE STUDIES>>



let us help secure you against passwords reuse with contact information and the ID Agent logo on grey.

Our Partners typically realize ROI in 30 days or less. Contact us today to learn why 3,850 MSPs in 30+ countries choose to Partner with ID Agent!

LEARN MORE>>


Check out an on-demand video demo of BullPhish ID or Dark Web ID WATCH NOW>>

See Graphus in action in an on-demand video demo WATCH NOW>>


Book your demo of Dark Web ID, BullPhish ID, RocketCyber or Graphus now!

SCHEDULE IT NOW>>