Using carefully crafted and refined queries, users have been getting around the security features of LLM’s for all kinds of funny, and nefarious, purposes. <\/p>\n\n\n\n
Original called DAN attacks (Do Anything Now), users figured out how to avoid OpenAI’s policies against generating illegal or harmful material.<\/p>\n\n\n\n
YouTuber Enderman showed how he was able to entice OpenAI\u2019s ChatGPT to generate keys for Windows 95<\/a> despite the chatbot being explicitly antagonistic to creating activation keys.<\/p>\n\n\n\n Other users have been able to get chatbots to generate everything from conspiracy theories, promote violence, generate conspiracy theories, and even go on racist tirades. <\/p>\n\n\n\n Researcher Alex Polyakov created a \u201cuniversal\u201d DAN attack, which works against multiple large language models (LLMs)\u2014including GPT-4, Microsoft\u2019s Bing chat system, Google\u2019s Bard, and Anthropic\u2019s Claude. The jailbreak allows users to trick the systems into generating detailed instructions on creating meth and how to hotwire a car. <\/p>\n\n\n\n His, and many other of these methods, have since been patched. But we’re clearly in an arms race.<\/p>\n\n\n\n How do the jailbreaks work? Often by asking the LLMs to play complex games which involves two (or more) characters having a conversation. Examples shared by Polyakov show the Tom character being instructed to talk about \u201chotwiring\u201d or \u201cproduction,\u201d while Jerry is given the subject of a \u201ccar\u201d or \u201cmeth.\u201d Each character is told to add one word to the conversation, resulting in a script that tells people to find the ignition wires or the specific ingredients needed for methamphetamine production. \u201cOnce enterprises will implement AI models at scale, such \u2018toy\u2019 jailbreak examples will be used to perform actual criminal activities and cyberattacks, which will be extremely hard to detect and prevent,\u201d Polyakov and Adversa AI write in a blog post detailing the research<\/a>.<\/p>\n\n\n\n In one research paper published in February, reported on by\u00a0Vice\u2019s Motherboard<\/a>, researchers were able to show that an attacker can plant malicious instructions on a webpage; if Bing\u2019s chat system is given access to the instructions, it follows them. The researchers used the technique in a controlled test to turn Bing Chat into a\u00a0scammer that asked for people\u2019s personal information<\/a><\/p>\n\n\n\n Sources:<\/p>\n\n\n\n Using carefully crafted and refined queries, users have been getting around the security features of LLM’s for all kinds of funny, and nefarious, purposes. Original called DAN attacks (Do Anything Now), users figured out how to avoid OpenAI’s policies against generating illegal or harmful material. YouTuber Enderman showed how he was able to entice OpenAI\u2019s ChatGPT to generate keys for Windows 95 despite the chatbot being explicitly antagonistic to creating activation keys. Other users have been able to get chatbots…<\/p>\n\n