Hackers Work Around ChatGPT Malicious Content Restrictions to Create Phishing Email Content
Active discussions in hacker forums on the dark web showcase how using a mixture of the Open AI API and automated bot from the Telegram messenger platform can create malicious emails.
It’s good that from the start, creators of ChatGPT put in content restrictions to keep the popular AI tool from being used for evil purposes. Any request to blatantly write and email or create code that will be misused to victimize another person is met with an “I’m sorry, I can’t generate <content requested>” response.
I wrote previously about ways ChatGPT could be misused – as long as the intent for the generated content isn’t divulged to the AI engine. New research from Checkpoint shows a number of examples of dark web discussions about how to bypass restrictions intent on keeping threat actors from using ChatGPT.
In essence, a hacker has created a bot that works within the messenger service Telegram to automate the writing of maliciously-intended emails and malware code.
Source: Checkpoint
Apparently the API for the Telegram bot does not have the same restrictions as direct interaction with ChatGPT. The hacker has gone as far as to establish a business model charging $5.50 for every 100 queries, making it inexpensive and easy for anyone wanting a well-written phishing email or base piece of malware.
This only means more players can get into the game without the barrier of needing to know how to write well or to code. It also means employees need to be far more vigilant than ever before – something taught with continual Security Awareness Training – scrutinizing every email to be absolutely certain that the content, sender, and intent is legitimate before ever interacting with them.
Free Phishing Security Test
Would your users fall for convincing phishing attacks? Take the first step now and find out before bad actors do. Plus, see how you stack up against your peers with phishing Industry Benchmarks. The Phish-prone percentage is usually higher than you expect and is great ammo to get budget.
Here’s how it works:
- Immediately start your test for up to 100 users (no need to talk to anyone)
- Select from 20+ languages and customize the phishing test template based on your environment
- Choose the landing page your users see after they click
- Show users which red flags they missed, or a 404 page
- Get a PDF emailed to you in 24 hours with your Phish-prone % and charts to share with management
- See how your organization compares to others in your industry
PS: Don’t like to click on redirected buttons? Cut & Paste this link in your browser: https://info.knowbe4.com/phishing-security-test-partner?partnerid=001a000001lWEoJAAW