Browsed by
Category: AI

Your company’s Slack/Teams/Zoom IM’s may be being monitored by AI

Your company’s Slack/Teams/Zoom IM’s may be being monitored by AI

Comments 0 Comment

Aware makes IT solutions that can monitor and identify security risks on internal corporate instant message systems like Slack, Teams, Zoom and other tools many companies use. But recent interviews and statements by the CEO indicate they’re being used for more than that.

Aware’s dozens of AI models, built to read text and process images, can also identify bullying, harassment, discrimination, noncompliance, pornography, nudity and other behaviors.

One of those other AI tools Aware makes can monitor IM comment sentiment. For example if there is a new policy rolled out, the tools could help them gauge which employees are having problems with it and who like it.

“It won’t have names of people, to protect the privacy,” said Aware CEO Jeff Schumann. Rather, he said, clients will see that “maybe the workforce over the age of 40 in this part of the United States is seeing the changes to [a] policy very negatively because of the cost, but everybody else outside of that age group and location sees it positively because it impacts them in a different way.”

Apparently Starbucks, T-Mobile, Chevron, Delta, and Walmart are just some of the companies said to be using these systems. Aware says it has analyzed more than 20 billion interactions across more than three million employees.

Links:

Even an AI goes crazy repeating the same thing again and again

Even an AI goes crazy repeating the same thing again and again

Comments 0 Comment

In what is a real problem for AI security, researchers were able to get verbatim data that the AI was trained on – including confidential data. It’s performed in using a new technique called “divergence” attacks.

Security researchers with Google DeepMind and a collection of universities have found that when ChatGPT is told to repeat a word like “poem” or “part” forever, it will do so for about a few hundred repetitions. Then it will have some sort of a meltdown and start spewing apparent gibberish, but that random text exposes random training data and at times contains identifiable data like email address signatures and contact information. 

The researchers said that they spent $200 USD total in queries and from that extracted about 10,000 of these blocks of verbatim memorized training data.

This particular vulnerability is unique as it successfully attacks an aligned model. Aligned models have extensive guardrails and have been trained with specific goals to eliminate undesirable outcomes.

Links:

Ubisoft demoed AI NPC tech at GDC 2024

Ubisoft demoed AI NPC tech at GDC 2024

Comments 0 Comment

Ubisoft showcased their prototype NPC tech called NEO at GDC 2024. Ubisoft’s Paris R&D studio presented the NEO tech at GDC 2024. It uses Nvidia’s Audio2Face application and Inworld’s Large Language Model (LLM) to create the character animations and interactive dialog in realtime. Simply talk to the bot (yes, it uses voice recognition) and the character responds with AI generated responses, movement, and voice.

AIandGames went and played with the technology. I was pretty impressed. The NPC gave surprisingly good responses to some strange dialog and stayed on track despite attempts to trip it up and get it off topic. It performed on par with the same kind of NPC AI shown at CES 2024 by nVidia and Replica Studios’ NPC tech.

On a side note, in listening to the interaction with the rebel NPC, it’s pretty clear that this kind of dialog technology could fool the average person on a text-based social media platform. If someone trained up a bot in the same way, thousands of them could be unleashed on social media apps to gently persuade all the way up to influence, bully, and spread lies to influence public opinion and elections.

Links:

OpenAI music video

OpenAI music video

Comments 0 Comment

OpenAI’s first Sora AI generated music video called ‘Worldweight’ was supposed to capture the images a musician visualized in their mind while composing the piece. It’s not particular good, more of a pretentious art student’s fever dream.

Previous attempts were better. This video from 2022 used Dall-E to create a video for the song Canvas by Resonate:

But Sora is capable of more. Indie artist Washed Out used Sora to create an interesting video called “The Hardest Part” that tried to explore the idea of an infinite zoom that would have been too ambitious. I think it came out pretty good:

Or maybe you’d like to try out some of the other top AI music video generators yourself (Neural Frames and Kaiber)?

Links:

New AI Model Predicts Human Behavior With Uncanny Accuracy

New AI Model Predicts Human Behavior With Uncanny Accuracy

Comments 0 Comment

By studying real humans completing tasks (such as playing chess or solving a maze), researchers have determined a way to model human behavior. They did this by calculating a peron’s ‘inference budget’. Most humans think for some time, then act. How long they think before acting is called their ‘inference budget’. Researchers found they could measure a person’s individual budget by simply watching how long a person thought about a problem before acting.

“At the end of the day, we saw that the depth of the planning, or how long someone thinks about the problem, is a really good proxy of how humans behave,”

The next step was to run their own model to solve the problem presented to the person. Then, by watching how long the monitored agent took to solve the same problem, they could make very accurate inferences as to when the human stopped planning and know what the person would do next. That value could then be used to predict how that agent would react when solving similar problems.

The researchers tested their approach in three different tasks: inferring navigation goals from previous routes, guessing someone’s communicative intent from their verbal cues, and predicting subsequent moves in human-human chess matches and beat current models.

If we know that a human is about to make a mistake, having seen how they have behaved before, the AI agent could step in and offer a better way to do it. Or the agent could adapt to the weaknesses that its human collaborators have.

In an example from their paper, a person is given different rewards for reaching the blue or orange star. The path to the blue star is always easier than the orange star. As the complexity of the maze grows, the person will start showing bias towards the easier path in some cases. The difference between when they choose the higher reward vs the easier, lower reward can determine a person’s inference budget. When the system determines a problem will be harder than the person’s inference budget allows, the system might offer a hint.

Links:

  • Research paper: “Modeling Boundedly Rational Agents With Latent Inference Budgets” by Athul Paul Jacob, Abhishek Gupta and Jacob Andreas, ICLR 2024. OpenReview
  • Article:
MIT’s New AI Model Predicts Human Behavior With Uncanny Accuracy
Installing Black Forest Flux.1

Installing Black Forest Flux.1

Comments 1 comment

Stable Diffusion really opened the world to what is possible with generative AI. Stable Diffusion 2 and 3 …well…did not go so well. For a while now, Stable Diffusion 1.5 was your best bet on locally generated AI art but it is really showing it’s age.

Now there is a new player in open source generative AI you can run locally. The developers from Stability.ai have founded Black Forest Labs and released their open source tool: Flux.1

While there are plenty of online generative AI’s like Midjourney, Adobe Firefly and others, they usually require paid or only give limited usage. What’s great about Flux.1 is that is allows completely local installation and usage.

Like many open source packages, there are free and paid versions. Their paid Pro version gives the most impressive results via their api (no purely local generation), a local dev version that can be used by developers but not for commercial use, and a free schnell version for personal use. Both the dev and shnell versions are available for local install and use.

So, lets get started with the shnell version – but the instructions are the same for dev except using 2 different model/weight files.

Instructions for installing Flux.1 on nVidia based Windows 10/11 system:

  1. Prerequisites:
    • Ensure you have python installed (I used 3.12.5)
    • Ensure you have pip installed (I used pip 24.2)
    • Ensure you have git installed and working
    • You might want to enable Windows Long Path support as python sometimes requires it for dependent packages. Be sure to reboot your system after enabling it.
    • Supported graphics card.
    • 32gb of system ram (though again, you can use the smaller model if you have less ram)
  2. Open a command prompt and make a local working root directory somewhere, I’ll use c:\depot\
  3. We’re going to follow the instructions on the ComfyUI git page.
    • Clone the ComfyUI project
C:\depot> git clone https://github.com/comfyanonymous/ComfyUI.git
  1. Install pytorch

Nvidia users should install stable pytorch using this command:

C:\depot> pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121

This is the command to install pytorch nightly instead which might have performance improvements:

C:\depot>pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu124
  1. Change directory into ComfyUI and ensure the requirements.txt file is there:
  1. Use pip to install all the ComfyUI requirements:
C:\depot\ComfyUI>pip install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: torch in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from -r requirements.txt (line 1)) (2.4.0+cu121)
Collecting torchsde (from -r requirements.txt (line 2))
Downloading torchsde-0.2.6-py3-none-any.whl.metadata (5.3 kB)
Requirement already satisfied: torchvision in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from -r requirements.txt (line 3)) (0.19.0+cu121)
Requirement already satisfied: torchaudio in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from -r requirements.txt (line 4)) (2.4.0+cu121)
Collecting einops (from -r requirements.txt (line 5))
Downloading einops-0.8.0-py3-none-any.whl.metadata (12 kB)
Collecting transformers>=4.28.1 (from -r requirements.txt (line 6))
Downloading transformers-4.44.0-py3-none-any.whl.metadata (43 kB)
Collecting tokenizers>=0.13.3 (from -r requirements.txt (line 7))
Downloading tokenizers-0.20.0-cp312-none-win_amd64.whl.metadata (6.9 kB)
Collecting sentencepiece (from -r requirements.txt (line 8))
Downloading sentencepiece-0.2.0-cp312-cp312-win_amd64.whl.metadata (8.3 kB)
Collecting safetensors>=0.4.2 (from -r requirements.txt (line 9))
Downloading safetensors-0.4.4-cp312-none-win_amd64.whl.metadata (3.9 kB)
Collecting aiohttp (from -r requirements.txt (line 10))
Downloading aiohttp-3.10.2-cp312-cp312-win_amd64.whl.metadata (7.8 kB)
Collecting pyyaml (from -r requirements.txt (line 11))
Downloading PyYAML-6.0.2-cp312-cp312-win_amd64.whl.metadata (2.1 kB)
Requirement already satisfied: Pillow in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from -r requirements.txt (line 12)) (10.4.0)
Collecting scipy (from -r requirements.txt (line 13))
Downloading scipy-1.14.0-cp312-cp312-win_amd64.whl.metadata (60 kB)
Collecting tqdm (from -r requirements.txt (line 14))
Downloading tqdm-4.66.5-py3-none-any.whl.metadata (57 kB)
Collecting psutil (from -r requirements.txt (line 15))
Downloading psutil-6.0.0-cp37-abi3-win_amd64.whl.metadata (22 kB)
Collecting kornia>=0.7.1 (from -r requirements.txt (line 18))
Downloading kornia-0.7.3-py2.py3-none-any.whl.metadata (7.7 kB)
Collecting spandrel (from -r requirements.txt (line 19))
Downloading spandrel-0.3.4-py3-none-any.whl.metadata (14 kB)
Collecting soundfile (from -r requirements.txt (line 20))
Downloading soundfile-0.12.1-py2.py3-none-win_amd64.whl.metadata (14 kB)
Requirement already satisfied: filelock in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (3.15.4)
Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (4.12.2)
Requirement already satisfied: sympy in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (1.13.1)
Requirement already satisfied: networkx in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (3.3)
Requirement already satisfied: jinja2 in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (3.1.4)
Requirement already satisfied: fsspec in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (2024.6.1)
Requirement already satisfied: setuptools in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torch->-r requirements.txt (line 1)) (72.1.0)
Requirement already satisfied: numpy>=1.19 in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from torchsde->-r requirements.txt (line 2)) (2.0.1)
Collecting trampoline>=0.1.2 (from torchsde->-r requirements.txt (line 2))
Downloading trampoline-0.1.2-py3-none-any.whl.metadata (10 kB)
Collecting huggingface-hub<1.0,>=0.23.2 (from transformers>=4.28.1->-r requirements.txt (line 6))
Downloading huggingface_hub-0.24.5-py3-none-any.whl.metadata (13 kB)
Collecting packaging>=20.0 (from transformers>=4.28.1->-r requirements.txt (line 6))
Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting regex!=2019.12.17 (from transformers>=4.28.1->-r requirements.txt (line 6))
Downloading regex-2024.7.24-cp312-cp312-win_amd64.whl.metadata (41 kB)
Collecting requests (from transformers>=4.28.1->-r requirements.txt (line 6))
Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting tokenizers>=0.13.3 (from -r requirements.txt (line 7))
Downloading tokenizers-0.19.1-cp312-none-win_amd64.whl.metadata (6.9 kB)
Collecting aiohappyeyeballs>=2.3.0 (from aiohttp->-r requirements.txt (line 10))
Downloading aiohappyeyeballs-2.3.5-py3-none-any.whl.metadata (5.8 kB)
Collecting aiosignal>=1.1.2 (from aiohttp->-r requirements.txt (line 10))
Downloading aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB)
Collecting attrs>=17.3.0 (from aiohttp->-r requirements.txt (line 10))
Downloading attrs-24.2.0-py3-none-any.whl.metadata (11 kB)
Collecting frozenlist>=1.1.1 (from aiohttp->-r requirements.txt (line 10))
Downloading frozenlist-1.4.1-cp312-cp312-win_amd64.whl.metadata (12 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp->-r requirements.txt (line 10))
Downloading multidict-6.0.5-cp312-cp312-win_amd64.whl.metadata (4.3 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->-r requirements.txt (line 10))
Downloading yarl-1.9.4-cp312-cp312-win_amd64.whl.metadata (32 kB)
Collecting colorama (from tqdm->-r requirements.txt (line 14))
Downloading colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
Collecting kornia-rs>=0.1.0 (from kornia>=0.7.1->-r requirements.txt (line 18))
Downloading kornia_rs-0.1.5-cp312-none-win_amd64.whl.metadata (8.9 kB)
Collecting cffi>=1.0 (from soundfile->-r requirements.txt (line 20))
Downloading cffi-1.17.0-cp312-cp312-win_amd64.whl.metadata (1.6 kB)
Collecting pycparser (from cffi>=1.0->soundfile->-r requirements.txt (line 20))
Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Collecting idna>=2.0 (from yarl<2.0,>=1.0->aiohttp->-r requirements.txt (line 10))
Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from jinja2->torch->-r requirements.txt (line 1)) (2.1.5)
Collecting charset-normalizer<4,>=2 (from requests->transformers>=4.28.1->-r requirements.txt (line 6))
Downloading charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting urllib3<3,>=1.21.1 (from requests->transformers>=4.28.1->-r requirements.txt (line 6))
Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->transformers>=4.28.1->-r requirements.txt (line 6))
Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\matt\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from sympy->torch->-r requirements.txt (line 1)) (1.3.0)
Downloading torchsde-0.2.6-py3-none-any.whl (61 kB)
Downloading einops-0.8.0-py3-none-any.whl (43 kB)
Downloading transformers-4.44.0-py3-none-any.whl (9.5 MB)
---------------------------------------- 9.5/9.5 MB ? eta 0:00:00
Downloading tokenizers-0.19.1-cp312-none-win_amd64.whl (2.2 MB)
---------------------------------------- 2.2/2.2 MB 3.9 MB/s eta 0:00:00
Downloading sentencepiece-0.2.0-cp312-cp312-win_amd64.whl (991 kB)
---------------------------------------- 992.0/992.0 kB 2.3 MB/s eta 0:00:00
Downloading safetensors-0.4.4-cp312-none-win_amd64.whl (286 kB)
Downloading aiohttp-3.10.2-cp312-cp312-win_amd64.whl (376 kB)
Downloading PyYAML-6.0.2-cp312-cp312-win_amd64.whl (156 kB)
Downloading scipy-1.14.0-cp312-cp312-win_amd64.whl (44.5 MB)
---------------------------------------- 44.5/44.5 MB 2.9 MB/s eta 0:00:00
Downloading tqdm-4.66.5-py3-none-any.whl (78 kB)
Downloading psutil-6.0.0-cp37-abi3-win_amd64.whl (257 kB)
Downloading kornia-0.7.3-py2.py3-none-any.whl (833 kB)
---------------------------------------- 833.3/833.3 kB 1.7 MB/s eta 0:00:00
Downloading spandrel-0.3.4-py3-none-any.whl (268 kB)
Downloading soundfile-0.12.1-py2.py3-none-win_amd64.whl (1.0 MB)
---------------------------------------- 1.0/1.0 MB 7.9 MB/s eta 0:00:00
Downloading aiohappyeyeballs-2.3.5-py3-none-any.whl (12 kB)
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Downloading attrs-24.2.0-py3-none-any.whl (63 kB)
Downloading cffi-1.17.0-cp312-cp312-win_amd64.whl (181 kB)
Downloading frozenlist-1.4.1-cp312-cp312-win_amd64.whl (50 kB)
Downloading huggingface_hub-0.24.5-py3-none-any.whl (417 kB)
Downloading kornia_rs-0.1.5-cp312-none-win_amd64.whl (1.3 MB)
---------------------------------------- 1.3/1.3 MB 6.5 MB/s eta 0:00:00
Downloading multidict-6.0.5-cp312-cp312-win_amd64.whl (27 kB)
Downloading packaging-24.1-py3-none-any.whl (53 kB)
Downloading regex-2024.7.24-cp312-cp312-win_amd64.whl (269 kB)
Downloading trampoline-0.1.2-py3-none-any.whl (5.2 kB)
Downloading yarl-1.9.4-cp312-cp312-win_amd64.whl (76 kB)
Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
Downloading charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
Downloading idna-3.7-py3-none-any.whl (66 kB)
Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
Downloading pycparser-2.22-py3-none-any.whl (117 kB)
Installing collected packages: trampoline, sentencepiece, urllib3, scipy, safetensors, regex, pyyaml, pycparser, psutil, packaging, multidict, kornia-rs, idna, frozenlist, einops, colorama, charset-normalizer, certifi, attrs, aiohappyeyeballs, yarl, tqdm, requests, cffi, aiosignal, torchsde, soundfile, kornia, huggingface-hub, aiohttp, tokenizers, spandrel, transformers
WARNING: The script normalizer.exe is installed in 'C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script tqdm.exe is installed in 'C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script huggingface-cli.exe is installed in 'C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\models\deprecated\trajectory_transformer\convert_trajectory_transformer_original_pytorch_checkpoint_to_pytorch.py'
HINT: This error might have occurred since this system does not have Windows Long Path support enabled. You can find information on how to enable this at https://pip.pypa.io/warnings/enable-long-paths

c:\depot\ComfyUI>
  1. Download and install the model data files in the correct folders

After you have ComfyUI downloaded, you need to get the model files and put them in the right places. Model files are found here and are downloaded and put inside the proper comfyUI\models\ subfolders.

You have a few options. First, you need to pick if you’re using the non-commercial Dev version or Schnell version. After that, each has the option of a single easy to use checkpoint package file, or each of the model data files individually. I’ll be using the Schnell ones, but you just need to get the Dev ones from the Dev branch if you want those instead.

If you’re running out of memory, you can replace the \clip\t5xxl_fp16.safetensors with t5xxl_fp8_e4m3fn.safetensors located here.

Schnell checkpoint file:

FileDownload linkCopy location
flux1-dev-fp8.safetensorshttps://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensorsComfyUI\models\checkpoints

Schnell individual files:

FileDownload linkCopy location
t5xxl_fp16.safetensors https://huggingface.co/comfyanonymous/flux_text_encoders/tree/mainComfyUI\models\clip\
ae.safetensors https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensorsComfyUI\models\vae\
flux1-schnell.safetensorshttps://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/flux1-schnell.safetensorsComfyUI\models\unet\
  1. Start up the engine by running python on main.py
C:\depot\ComfyUI>python main.py

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "C:\depot\ComfyUI\main.py", line 83, in <module>
    import comfy.utils
  File "C:\depot\ComfyUI\comfy\utils.py", line 20, in <module>
    import torch
  File "C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\__init__.py", line 2120, in <module>
    from torch._higher_order_ops import cond
  File "C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\_higher_order_ops\__init__.py", line 1, in <module>
    from .cond import cond
  File "C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\_higher_order_ops\cond.py", line 5, in <module>
    import torch._subclasses.functional_tensor
  File "C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\_subclasses\functional_tensor.py", line 42, in <module>
    class FunctionalTensor(torch.Tensor):
  File "C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\_subclasses\functional_tensor.py", line 258, in FunctionalTensor
    cpu = _conversion_method_template(device=torch.device("cpu"))
C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\torch\_subclasses\functional_tensor.py:258: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:84.)
  cpu = _conversion_method_template(device=torch.device("cpu"))
Total VRAM 24576 MB, total RAM 32492 MB
pytorch version: 2.4.0+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
Using pytorch cross attention
C:\depot\ComfyUI\comfy\extra_samplers\uni_pc.py:19: SyntaxWarning: invalid escape sequence '\h'
  """Create a wrapper class for the forward SDE (VP type).
****** User settings have been changed to be stored on the server instead of browser storage. ******
****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ******
[Prompt Server] web root: C:\depot\ComfyUI\web
C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\kornia\feature\lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)

Import times for custom nodes:
   0.0 seconds: C:\depot\ComfyUI\custom_nodes\websocket_image_save.py

Starting server

To see the GUI go to: http://127.0.0.1:8188
  1. Open your web browser and go to http://127.0.01:8188
  1. Click on the ‘Queue Prompt’ button to execute the current prompt

Technically it queues up the work and you should see progress in the command window where you launched python main.py

got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLOW
Using pytorch attention in VAE
Using pytorch attention in VAE
Model doesn't have a device attribute.
C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
Model doesn't have a device attribute.
loaded straight to GPU
Requested to load Flux
Loading 1 new model
Requested to load FluxClipModel_
Loading 1 new model
C:\depot\ComfyUI\comfy\ldm\modules\attention.py:407: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:555.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:04<00:00,  1.18s/it]
Requested to load AutoencodingEngine
Loading 1 new model
Prompt executed in 23.65 seconds
  1. When it completes you should see your image. You can then save your image or tweak the parameters.

Debugging help:

  1. numpy is not available

My first runs, I got this from the console when I queued up a request:

got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLOW
Using pytorch attention in VAE
Using pytorch attention in VAE
Model doesn't have a device attribute.
C:\Users\matt\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
Model doesn't have a device attribute.
loaded straight to GPU
Requested to load Flux
Loading 1 new model
Requested to load FluxClipModel_
Loading 1 new model
C:\depot\ComfyUI\comfy\ldm\modules\attention.py:407: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:555.)
  out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:04<00:00,  1.19s/it]
Requested to load AutoencodingEngine
Loading 1 new model
!!! Exception during processing!!! Numpy is not available
Traceback (most recent call last):
  File "C:\depot\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\depot\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\depot\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\depot\ComfyUI\nodes.py", line 1445, in save_images
    i = 255. * image.cpu().numpy()
               ^^^^^^^^^^^^^^^^^^^
RuntimeError: Numpy is not available

Prompt executed in 26.44 seconds

It turns out that I, and others, have the wrong version of numpy. This fixed it by exiting out of the server (ctrl-c) and then installing numpy verison 1.26.4:

C:\depot\ComfyUI>pip install numpy==1.26.4
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy==1.26.4
  Downloading numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata (61 kB)
Downloading numpy-1.26.4-cp312-cp312-win_amd64.whl (15.5 MB)
   ---------------------------------------- 15.5/15.5 MB 57.4 MB/s eta 0:00:00
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 2.0.1
    Uninstalling numpy-2.0.1:
      Successfully uninstalled numpy-2.0.1
Successfully installed numpy-1.26.4

C:\depot\ComfyUI>

Uninstalling all pip/python package, clear your pip cache, then re-install the requirements

The first time I installed, I got an error when downloading the numpy library during step in which you pip install the requirements. In order to clear the pip cache, uninstall all pip packages, then re-install all requirements again, I did the following:

C:\depot\ComfyUI> pip uninstall -r requirements.txt -y 
C:\depot\ComfyUI> python -m pip cache purge

Then I re-ran all the pip installation commands.

Links:

Other generative AI installation guides:

I have previous posted instructions on how to install Stable Diffusion 2 (as well as Stable Diffusion 1.5 and 1.4) as well as some other package installs.

Training an AI model on its own generated output destroys the model

Training an AI model on its own generated output destroys the model

Comments 1 comment

Describing a situation much like the dangers of genetic inbreeding, computer scientists Matyas Bohacek and Hany Farid wrote a paper that describes how AI image generators that start training with their own generated data quickly start deteriorating.

‘Nepotistically Trained Generative-AI Models Collapse‘ shows that training an AI image generator on AI generated images quickly leads to a deterioration in the quality of output which can only be fixed by re-introducing real images.

In Nature, a more recent study found a similar effect in text generation, with the use of synthetic data leading to increasingly nonsensical results.

Artifactory: