I wrote about Stable Dreamfusion previously. Dreamfusion first takes normal Stable Diffusion text prompts to generate 2D images of the desired object. Stable Dreamfusion then uses those 2D images to generate 3D meshes.

The authors seemed to be using A100 nVidia cards on an Ubuntu system. I wanted to see if I could get this to work locally on my home Windows PC, and found that I could do so.

System configuration I am using for this tutorial:

• nVidia GeForce GTX 3090
• Intel 12th gen processor
• Windows 10

Setting Stable Dreamfusion up locally:

Step 1: Update your Windows and drivers

1. Update Windows
2. Ensure you have the latest nVidia driver installed.

Step 2: Install Windows Subsystem for Linux (WSL)

1. Install Windows Subsystem for Linux (WSL). WSL install is a simple command line install. You’ll need to reboot after you install. You want to make sure you install Ubuntu 22.04, which is the default in Feb 2023 since that is what Stable Dreamfusion likes. Currently WSL installs the latest Ubuntu distro by default, so this works:
   wsl –install
   If you want to make sure you get Ubuntu 22.04, use this command line:
   wsl –install -d Ubuntu-22.04
2. After installing WSL, Windows will ask to reboot.
3. Upon reboot, the WSL will complete installation and ask you to create a user account.
4. Start Ubuntu 22.04 on WSL by clicking on the Windows Start menu and typing ‘Ubuntu’ or you can type Ubuntu at a command prompt and type ‘Ubuntu’.

Step 2b (optional): Install Ubuntu wherever you want on your Windows system. By default it installs the image on your C:\Users directory – which is kind of annoying.

Step 3: Install dependent packages on Ubuntu

1. If you don’t have Ubuntu started, go ahead and start Ubuntu 22.04 on WSL by clicking on the Windows Start menu and typing ‘Ubuntu’ (or you can type Ubuntu at a command prompt as well). A new shell terminal should appear.
2. You need to install the nVidia CUDA SDK on Ubuntu. You’ll choose one of these two options:
   • GPU Accelerated: https://docs.nvidia.com/cuda/wsl-user-guide/index.html (this does not seem to be using GPU acceleration by default as hoped, but I’m still debugging)
   • Software version: Download the standard Ubuntu CUDA SDK from the nVidia site. On the download page, select the following parameters to get the right install directions:
     • Select Linux for the Operating system
     • Architecture: x86_64
     • Distribution: Ubuntu (you can also select WSL Ubuntu which should be the same as the supposedly GPU accelerated)
     • Version: 2.0
     • Installer type: deb (local)
   • You will then get a set of install instructions at the bottom of the page (wget, apt-get, etc). Simply copy the lines one by one and put them into your Ubuntu terminal. Ensure each step passes without errors before continuing.
   • The ‘sudo apt-get -y install cuda’ line will install a lot of packages. It can take 10-15 minutes.
3. Install python3 pip. This is required for the Dreamfusion requirements installation script.
   • sudo apt install python3-pip

Step 4: Install Stable Dreamfusion and dependent packages

1. You should now follow the install instructions found on the Dreamfusion page.
2. Clone the project as directed: git clone https://github.com/ashawkey/stable-dreamfusion.git
3. Install with PIP: Install the pre-requisites via pip as directed on the Dreamfusion github page:
   • pip install -r requirements.txt
   • I also installed both optional packages nvdiffrast and CLIP.
   • Add this export line to your .bashrc to ensure python can find libcudnn:
     export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH
4. I did not install the build extension options
5. Exit and restart your shell so that all path changes take effect

Step 5: Run a workload!

Follow the instructions in the USAGE section of the Dreamfusion instructions. Instead of ‘python’ use ‘python3’. They have a number of things you can specify like negative prompts, using the GUI interface (which does not work under WSL),

The very first run will take a long time. It will download several gigabytes of training data, then train 100 epoch’s, which can take up to an hour.

$> python3 main.py --text "a hamburger" --workspace trial -O 
$> python3 main.py --text "a hamburger" --workspace trial -O --sd_version 1.5 
$> python3 main.py --workspace trial -O --test 
$> python3 main.py --workspace trial -O --test --save_mesh 

Check Your Output:

Look in the results directory under the workspace name:

.\stable-dreamfusion\<workspace name>\mesh\ #directory holds the .obj, .mat, and .png files
.\stable-dreamfusion\<workspace name>\results\ #directory holds a mp4 video that shows the object rotating

Copying them to Windows:
All Windows drives are pre-mounted in \mnt\<drive letter>\ for WSL.
Ex: \mnt\c\
So you can copy the output files to your windows side by doing:
cp -rP .\<workspace name> \mnt\c\workdir\

Looking at the generated meshes with materials:

1. Install Blender
2. File->Import->Wavefront (.obj) (legacy)
3. Or, use 3D Viewer (though it seems to have issues with material loading at times)

Fixes:

1. You might get an error about missing libcudnn_cnn_infer.so.8

==> Start Training trial Epoch 1, lr=0.050000 …
0% 0/100 [00:00<?, ?it/s]Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory

add this to your .bashrc to ensure it can find libcudnn:
export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH

2. If you load the object in Blender but it doesn’t load the texture maps, try Alt-Z

Links:

• https://superuser.com/questions/1572834/is-there-any-way-to-install-wsl-on-non-c-drive
• https://github.com/ashawkey/stable-dreamfusion
• Using various machine learning on WSL: https://learn.microsoft.com/en-us/windows/ai/directml/gpu-accelerated-training
• https://blender.stackexchange.com/questions/7920/blender-isnt-importing-mtl-correctly-with-obj