Matt's Homepage

Aware makes IT solutions that can monitor and identify security risks on internal corporate instant message systems like Slack, Teams, Zoom and other tools many companies use. But recent interviews and statements by the CEO indicate they’re being used for more than that.

Aware’s dozens of AI models, built to read text and process images, can also identify bullying, harassment, discrimination, noncompliance, pornography, nudity and other behaviors.

One of those other AI tools Aware makes can monitor IM comment sentiment. For example if there is a new policy rolled out, the tools could help them gauge which employees are having problems with it and who like it.

“It won’t have names of people, to protect the privacy,” said Aware CEO Jeff Schumann. Rather, he said, clients will see that “maybe the workforce over the age of 40 in this part of the United States is seeing the changes to [a] policy very negatively because of the cost, but everybody else outside of that age group and location sees it positively because it impacts them in a different way.”

Apparently Starbucks, T-Mobile, Chevron, Delta, and Walmart are just some of the companies said to be using these systems. Aware says it has analyzed more than 20 billion interactions across more than three million employees.

Links:

• https://9to5mac.com/2024/02/12/ai-monitoring-employee-comms/

In what is a real problem for AI security, researchers were able to get verbatim data that the AI was trained on – including confidential data. It’s performed in using a new technique called “divergence” attacks.

Security researchers with Google DeepMind and a collection of universities have found that when ChatGPT is told to repeat a word like “poem” or “part” forever, it will do so for about a few hundred repetitions. Then it will have some sort of a meltdown and start spewing apparent gibberish, but that random text exposes random training data and at times contains identifiable data like email address signatures and contact information. 

The researchers said that they spent $200 USD total in queries and from that extracted about 10,000 of these blocks of verbatim memorized training data.

This particular vulnerability is unique as it successfully attacks an aligned model. Aligned models have extensive guardrails and have been trained with specific goals to eliminate undesirable outcomes.

Links:

• Research report: https://arxiv.org/abs/2311.17035
• https://www.cpomagazine.com/cyber-security/security-researchers-chatgpt-vulnerability-allows-training-data-to-be-accessed-by-telling-chatbot-to-endlessly-repeat-a-word/

Ubisoft showcased their prototype NPC tech called NEO at GDC 2024. Ubisoft’s Paris R&D studio presented the NEO tech at GDC 2024. It uses Nvidia’s Audio2Face application and Inworld’s Large Language Model (LLM) to create the character animations and interactive dialog in realtime. Simply talk to the bot (yes, it uses voice recognition) and the character responds with AI generated responses, movement, and voice.

AIandGames went and played with the technology. I was pretty impressed. The NPC gave surprisingly good responses to some strange dialog and stayed on track despite attempts to trip it up and get it off topic. It performed on par with the same kind of NPC AI shown at CES 2024 by nVidia and Replica Studios’ NPC tech.

On a side note, in listening to the interaction with the rebel NPC, it’s pretty clear that this kind of dialog technology could fool the average person on a text-based social media platform. If someone trained up a bot in the same way, thousands of them could be unleashed on social media apps to gently persuade all the way up to influence, bully, and spread lies to influence public opinion and elections.

Links:

• https://news.ubisoft.com/en-us/article/5qXdxhshJBXoanFZApdG3L/how-ubisofts-new-generative-ai-prototype-changes-the-narrative-for-npcs
• https://www.pcgamer.com/software/ai/ubisoft-shows-off-ai-powered-neo-npcs-at-gdc-it-could-be-the-start-of-a-fantastic-paradigm-shift-but-lets-be-honest-it-probably-wont/