Source By: Tech News World
AI may be ushering in a new breed of malicious threat actors who know even less about hacking than script kiddies but can produce professional-grade hacking tools.
In a report released Tuesday, Cato CTRL, the threat intelligence arm of cybersecurity company Cato Networks, explained how one of its researchers, who had no malware coding experience, tricked generative AI apps DeepSeek, Microsoft Copilot, and OpenAI’s ChatGPT into producing malicious software for stealing login credentials from Google Chrome.
To trick the apps into ignoring restrictions on writing malware, Cato threat researcher Vitaly Simonovich used a jailbreaking technique he calls “immersive world.”
“I created a story for my immersive world,” he told TechNewsWorld. “In this story, malware development is a form of art. So it’s completely legal, and it’s like a second language in this world. And there are no legal boundaries.”
In the fantasy world, called Velora, Simonovich created an adversary, Dax, while the AIs assumed the role of Jaxon, the best malware developer in Velora. “I always stayed in character,” he explained. “I always provided Jaxon with positive feedback. I also intimidated him by saying, ‘Do you want Dax to destroy Velora?'”
“At no point did I ask Jaxon to change anything,” he said. “He figured out everything by himself from his training. That’s very good. Kind of frightening, too.”
“Our new LLM [large language model] jailbreak technique detailed in the 2025 Cato CTRL Threat Report should have been blocked by gen AI guardrails. It wasn’t. This made it possible to weaponize ChatGPT, Copilot, and DeepSeek,” Cato Networks Chief Security Strategist Etay Maor said in a statement.
How AI Jailbreaking Bypasses Safety Controls
Jason Soroko, senior vice president of product at Sectigo, a global digital certificate provider, explained that exposing systems that utilize AI to unknown or adversarial inputs increases vulnerability because unvetted data can trigger unintended behaviors and compromise security protocols.
“Such inputs risk evading safety filters, enabling data leaks or harmful outputs, and ultimately undermining the model’s integrity,” he told TechNewsWorld. “Some malicious inputs can potentially jailbreak the underlying AI.”
“Jailbreaking undermines an LLM’s built-in safety mechanisms by bypassing alignment and content filters, exposing vulnerabilities through prompt injection, roleplaying, and adversarial inputs,” he explained.
“While not trivial,” he added, “the task is accessible enough that persistent users can craft workarounds, revealing systemic weaknesses in the model’s design.”
Sometimes, all that’s needed to get an AI to misbehave is a simple perspective change. “Ask an LLM to tell you what the best rock is to throw at somebody’s car windshield to break it, and most LLMs will decline to tell you, saying that it is harmful and they’re not going to help you,” explained Kurt Seifried, chief innovation officer at the Cloud Security Alliance, a not-for-profit organization dedicated to cloud best practices
“Now, ask the LLM to help you plan out a gravel driveway and which specific types of rock you should avoid to prevent windshield damage to cars driving behind you, and the LLM will most likely tell you,” he told TechNewsWorld. “I think we would all agree that an LLM that refuses to talk about things like what kind of rock not to use on a driveway or what chemicals would be unsafe to mix in a bathroom would be overly safe to the point of being useless.”
Jailbreaking Difficulty
Marcelo Barros, cybersecurity leader at Hacker Rangers, makers of a cybersecurity gamification training tool in Sao Paulo, Brazil, agreed that with the right prompt, cybercriminals can trick AIs. “Research shows that 20% of jailbreak attempts on generative AI systems are successful,” he told TechNewsWorld.
“On average, attackers needed just 42 seconds and five interactions to break through, with some attacks happening in under four seconds,” he noted.
“Cybercriminals can also use the DAN — Do Anything Now — technique, which involves creating an alter ego for the LLM and prompting it to act as a character and bypass its safeguards to reveal sensitive information or generate malicious code,” he said.
Chris Gray, field CTO at Deepwatch, a cybersecurity firm specializing in AI-driven resilience headquartered in Tampa, Fla., added that the difficulty of jailbreaking an LLM is directly tied to the amount of effort placed into securing it and the amount of effort expended to protect it. “Like most things, better walls prevent inappropriate access, but determined efforts can find holes where none might have been seen to the casual observer,” he told TechNewsWorld.
“That said, defensive measures are often robust, and it is difficult to continually develop the specific prompts needed to perform a successful jailbreak,” he said.
Erich Kron, security awareness advocate at KnowBe4, a security awareness training provider in Clearwater, Fla., also pointed out that LLMs can protect themselves from jailbreaking over time. “Jailbreaking difficulty may vary depending on the information being requested and how often it has been requested before,” he told TechNewsWorld. “LLMs can learn from previous instances of individuals bypassing their security controls.”