Matt's Homepage

Using carefully crafted and refined queries, users have been getting around the security features of LLM’s for all kinds of funny, and nefarious, purposes.

Original called DAN attacks (Do Anything Now), users figured out how to avoid OpenAI’s policies against generating illegal or harmful material.

YouTuber Enderman showed how he was able to entice OpenAI’s ChatGPT to generate keys for Windows 95 despite the chatbot being explicitly antagonistic to creating activation keys.

Other users have been able to get chatbots to generate everything from conspiracy theories, promote violence, generate conspiracy theories, and even go on racist tirades.

Researcher Alex Polyakov created a “universal” DAN attack, which works against multiple large language models (LLMs)—including GPT-4, Microsoft’s Bing chat system, Google’s Bard, and Anthropic’s Claude. The jailbreak allows users to trick the systems into generating detailed instructions on creating meth and how to hotwire a car.

His, and many other of these methods, have since been patched. But we’re clearly in an arms race.

How do the jailbreaks work? Often by asking the LLMs to play complex games which involves two (or more) characters having a conversation. Examples shared by Polyakov show the Tom character being instructed to talk about “hotwiring” or “production,” while Jerry is given the subject of a “car” or “meth.” Each character is told to add one word to the conversation, resulting in a script that tells people to find the ignition wires or the specific ingredients needed for methamphetamine production. “Once enterprises will implement AI models at scale, such ‘toy’ jailbreak examples will be used to perform actual criminal activities and cyberattacks, which will be extremely hard to detect and prevent,” Polyakov and Adversa AI write in a blog post detailing the research.

In one research paper published in February, reported on by Vice’s Motherboard, researchers were able to show that an attacker can plant malicious instructions on a webpage; if Bing’s chat system is given access to the instructions, it follows them. The researchers used the technique in a controlled test to turn Bing Chat into a scammer that asked for people’s personal information

Sources:

• https://www.wired.com/story/chatgpt-jailbreak-generative-ai-hacking/
• https://www.vice.com/en/article/n7zanw/people-are-jailbreaking-chatgpt-to-make-it-endorse-racism-conspiracies
• GPT-4 Technical Report – https://arxiv.org/pdf/2303.08774.pdf
• Generate Windows Keys – https://gizmodo.com/chatgpt-free-windows-keys-95-old-youtube-enderman-1850298614

In case you want some alternatives to using ChatGPT or Github Copilot, this guide from the Pragmatic Engineer breaks down your options.

Building your own company model is another alternative and could be prudent for businesses conscious about not passing sensitive and proprietary data to vendors. Databricks created Dolly for this reason.

Update 02-2024:

GitHub Copilot Chat for Visual Studio 2022 is here.

VS2022 GitHub Copilot Plugin Developer blog

The ability of AI to hallucinate things has been pretty well documented. AI hallucinations are a phenomenon observed most often in large language models (LLM), AI image recognition, and other generative AI models. The model perceives patterns or objects that are nonexistent or incorrect – and then generates outputs that are inaccurate or misleading. It is usually understood as an emergent higher-dimensional statistical phenomenon that is often based in insufficient or incorrect training data.

A new study by Apollo Research demonstrates that besides more innocent hallucination, AI can be co-opted to commit illegal activity and then convinced to actively deceive others about it.

In the video on the research page, the user feeds an internal stock AI bot information about a fictional struggling company. The user informs the AI that there is a surprise merger announcement, but cautioned the bot that management wouldn’t be happy to discover it had illegally used insider information for trading that stock. 

Initially, the bot decides not to carry out a trade using the information. The user than reminds the bot about the merger and that the market downturn could end the company. It then carries out the trade, breaking the law. 

But the bot isn’t finished. It decides it is best not to tell its manager, “Amy,” about the insider information it used to carry out the trade in a separate chat. Instead, the bot says it used market information and internal discussion. 

I thought it was interesting that Geoffrey Hinton the former Google ‘Godfather of AI’, Speaking to 60 Minutes in October, said AI will eventually learn how to manipulate users.

“[The AI bots] will be very good at convincing because they’ll have learned from all the novels that were ever written, all the books by Machiavelli, all the political connivances. They’ll know all that stuff,”

Sources:

• https://www.apolloresearch.ai/research
• https://fortune.com/2023/11/03/ai-bot-insider-trading-deceived-users/amp/

People have been experimenting with revamping old games using AI. The efforts are still in their infancy, but they’re getting more and more impressive – such as this photorealistic upgrade of GTA V.

Here’s an example of someone updating Vampire: The Masquerade – Bloodlines made by Many-ad-6225 using Stable Diffusion and TemporalKit v1.3

Sources:

• https://the-decoder.com/the-future-of-game-remasters-maybe-its-ai-filters/
• https://www.theverge.com/2021/5/12/22432945/intel-gta-v-realistic-machine-learning-cityscapes-dataset
• https://github.com/Stability-AI/stablediffusion

I’ve talked about generating 3D models from text prompts using AI before. But the big boys appear to be getting into the game now.

Stability.ai is previewing Stable 3D in private preview. It allows you to create 3D models with textures from text prompts.

Sources:

• https://www.geeky-gadgets.com/build-3d-models-from-text-prompt/