Matt's Homepage

Ubisoft showcased their prototype NPC tech called NEO at GDC 2024. Ubisoft’s Paris R&D studio presented the NEO tech at GDC 2024. It uses Nvidia’s Audio2Face application and Inworld’s Large Language Model (LLM) to create the character animations and interactive dialog in realtime. Simply talk to the bot (yes, it uses voice recognition) and the character responds with AI generated responses, movement, and voice.

AIandGames went and played with the technology. I was pretty impressed. The NPC gave surprisingly good responses to some strange dialog and stayed on track despite attempts to trip it up and get it off topic. It performed on par with the same kind of NPC AI shown at CES 2024 by nVidia and Replica Studios’ NPC tech.

On a side note, in listening to the interaction with the rebel NPC, it’s pretty clear that this kind of dialog technology could fool the average person on a text-based social media platform. If someone trained up a bot in the same way, thousands of them could be unleashed on social media apps to gently persuade all the way up to influence, bully, and spread lies to influence public opinion and elections.

Links:

• https://news.ubisoft.com/en-us/article/5qXdxhshJBXoanFZApdG3L/how-ubisofts-new-generative-ai-prototype-changes-the-narrative-for-npcs
• https://www.pcgamer.com/software/ai/ubisoft-shows-off-ai-powered-neo-npcs-at-gdc-it-could-be-the-start-of-a-fantastic-paradigm-shift-but-lets-be-honest-it-probably-wont/

By studying real humans completing tasks (such as playing chess or solving a maze), researchers have determined a way to model human behavior. They did this by calculating a peron’s ‘inference budget’. Most humans think for some time, then act. How long they think before acting is called their ‘inference budget’. Researchers found they could measure a person’s individual budget by simply watching how long a person thought about a problem before acting.

“At the end of the day, we saw that the depth of the planning, or how long someone thinks about the problem, is a really good proxy of how humans behave,”

The next step was to run their own model to solve the problem presented to the person. Then, by watching how long the monitored agent took to solve the same problem, they could make very accurate inferences as to when the human stopped planning and know what the person would do next. That value could then be used to predict how that agent would react when solving similar problems.

The researchers tested their approach in three different tasks: inferring navigation goals from previous routes, guessing someone’s communicative intent from their verbal cues, and predicting subsequent moves in human-human chess matches and beat current models.

If we know that a human is about to make a mistake, having seen how they have behaved before, the AI agent could step in and offer a better way to do it. Or the agent could adapt to the weaknesses that its human collaborators have.

In an example from their paper, a person is given different rewards for reaching the blue or orange star. The path to the blue star is always easier than the orange star. As the complexity of the maze grows, the person will start showing bias towards the easier path in some cases. The difference between when they choose the higher reward vs the easier, lower reward can determine a person’s inference budget. When the system determines a problem will be harder than the person’s inference budget allows, the system might offer a hint.

Links:

• Research paper: “Modeling Boundedly Rational Agents With Latent Inference Budgets” by Athul Paul Jacob, Abhishek Gupta and Jacob Andreas, ICLR 2024. OpenReview
• Article:

MIT’s New AI Model Predicts Human Behavior With Uncanny Accuracy