Matching what Jensen Huang said at GTC 2024 where he predicted every pixel of your game will soon be generated by AI, Google just demonstrates AI is capable not only of generating frames, it’s capable of generating the entire game Doom live at 20fps.
Google shared their paper and info at https://gamengen.github.io
Links:
What if you re-told the movie “The Fifth Element” as a 1950’s sci-fi flick using a generative AI. Yes, everything you see is generated by AI – and you can learn the techniques too from Curious Refuge’s AI filmmaking classes.
Wow – the things AI has created…
Aware makes IT solutions that can monitor and identify security risks on internal corporate instant message systems like Slack, Teams, Zoom and other tools many companies use. But recent interviews and statements by the CEO indicate they’re being used for more than that.
Aware’s dozens of AI models, built to read text and process images, can also identify bullying, harassment, discrimination, noncompliance, pornography, nudity and other behaviors.
One of those other AI tools Aware makes can monitor IM comment sentiment. For example if there is a new policy rolled out, the tools could help them gauge which employees are having problems with it and who like it.
“It won’t have names of people, to protect the privacy,” said Aware CEO Jeff Schumann. Rather, he said, clients will see that “maybe the workforce over the age of 40 in this part of the United States is seeing the changes to [a] policy very negatively because of the cost, but everybody else outside of that age group and location sees it positively because it impacts them in a different way.”
Apparently Starbucks, T-Mobile, Chevron, Delta, and Walmart are just some of the companies said to be using these systems. Aware says it has analyzed more than 20 billion interactions across more than three million employees.
Links:
In what is a real problem for AI security, researchers were able to get verbatim data that the AI was trained on – including confidential data. It’s performed in using a new technique called “divergence” attacks.
Security researchers with Google DeepMind and a collection of universities have found that when ChatGPT is told to repeat a word like “poem” or “part” forever, it will do so for about a few hundred repetitions. Then it will have some sort of a meltdown and start spewing apparent gibberish, but that random text exposes random training data and at times contains identifiable data like email address signatures and contact information.
The researchers said that they spent $200 USD total in queries and from that extracted about 10,000 of these blocks of verbatim memorized training data.
This particular vulnerability is unique as it successfully attacks an aligned model. Aligned models have extensive guardrails and have been trained with specific goals to eliminate undesirable outcomes.
Links:
Ubisoft showcased their prototype NPC tech called NEO at GDC 2024. Ubisoft’s Paris R&D studio presented the NEO tech at GDC 2024. It uses Nvidia’s Audio2Face application and Inworld’s Large Language Model (LLM) to create the character animations and interactive dialog in realtime. Simply talk to the bot (yes, it uses voice recognition) and the character responds with AI generated responses, movement, and voice.
AIandGames went and played with the technology. I was pretty impressed. The NPC gave surprisingly good responses to some strange dialog and stayed on track despite attempts to trip it up and get it off topic. It performed on par with the same kind of NPC AI shown at CES 2024 by nVidia and Replica Studios’ NPC tech.
On a side note, in listening to the interaction with the rebel NPC, it’s pretty clear that this kind of dialog technology could fool the average person on a text-based social media platform. If someone trained up a bot in the same way, thousands of them could be unleashed on social media apps to gently persuade all the way up to influence, bully, and spread lies to influence public opinion and elections.
Links:
OpenAI’s first Sora AI generated music video called ‘Worldweight’ was supposed to capture the images a musician visualized in their mind while composing the piece. It’s not particular good, more of a pretentious art student’s fever dream.
Previous attempts were better. This video from 2022 used Dall-E to create a video for the song Canvas by Resonate:
But Sora is capable of more. Indie artist Washed Out used Sora to create an interesting video called “The Hardest Part” that tried to explore the idea of an infinite zoom that would have been too ambitious. I think it came out pretty good:
Links:
YouTuber DemonFlyingFox is an AI artist who creates videos that bring pop culture figures to zenith of their narrative purposes.
Oh what AI has wrought.
By studying real humans completing tasks (such as playing chess or solving a maze), researchers have determined a way to model human behavior. They did this by calculating a peron’s ‘inference budget’. Most humans think for some time, then act. How long they think before acting is called their ‘inference budget’. Researchers found they could measure a person’s individual budget by simply watching how long a person thought about a problem before acting.
“At the end of the day, we saw that the depth of the planning, or how long someone thinks about the problem, is a really good proxy of how humans behave,”
The next step was to run their own model to solve the problem presented to the person. Then, by watching how long the monitored agent took to solve the same problem, they could make very accurate inferences as to when the human stopped planning and know what the person would do next. That value could then be used to predict how that agent would react when solving similar problems.
The researchers tested their approach in three different tasks: inferring navigation goals from previous routes, guessing someone’s communicative intent from their verbal cues, and predicting subsequent moves in human-human chess matches and beat current models.
If we know that a human is about to make a mistake, having seen how they have behaved before, the AI agent could step in and offer a better way to do it. Or the agent could adapt to the weaknesses that its human collaborators have.
In an example from their paper, a person is given different rewards for reaching the blue or orange star. The path to the blue star is always easier than the orange star. As the complexity of the maze grows, the person will start showing bias towards the easier path in some cases. The difference between when they choose the higher reward vs the easier, lower reward can determine a person’s inference budget. When the system determines a problem will be harder than the person’s inference budget allows, the system might offer a hint.
Links: