Matt's Homepage

In order to get Kokoro text-to-speech to work with my 5090 GPU inside a Docker container on Windows, I had to get the CUDA setup inside WSL2 using Docker images on Windows. Here were the helpful links:

Setup the WSL image with CUDA and Docker support.

1. Ensure you have WSL2 installed on your Windows desktop
2. Install Docker for Windows
3. Reboot
4. Ensure you have the latest NVidia GPU driver installed on your Windows desktop
5. Ensure you have the latest NVidia CUDA sdk installed on your Windows desktop
6. A system reboot here is a good idea
7. Ensure you have Microsoft Visual Studio installed for best results
   • The CUDA sdk requires you have Visual Studio 2019, 2022, or 2026 installed. I have found 2022 is a solid version and seems to work well, but haven’t tried 2019 or 2026 recently.
   • Note you need to use the ‘Visual Studio x64 Native tools for VS2022’ command prompt if you expect to run command line compiler operations.
8. Now you need to install CUDA support inside WSL NVidia guide
   • First follow the instructions to ensure Docker support
   • Second follow the installation of CUDA Toolkit and CUDA Developer Tools inside WSL2 on an Ubuntu 24.04 (or later) WSL distro.
     • NVidia CUDA Toolkit 13.3 WSL2-Ubuntu download
9. Optional: install nvcc so y
   • sudo apt install nvidia-cuda-toolkit
   • nvcc –version
     • This prints out the version and ensures the nvcc sdk compiler is working inside your WSL image
10. Ensure Docker Desktop for Windows is running on your desktop
11. Turn on support for Docker in WSL
    • Settings -> From the General tab, enable Use WSL 2 based engine.
    • Reboot
12. Ensure your WSL image inside WSL has Docker support enabled
    • Settings -> Resources -> WSL integration tab -> Find and enable the slider for the WSL image under ‘Enable integration with additional distros’
13. I had to reboot my system at this point to get both the WSL and Windows Docker Desktop to see each other properly
14. After reboot, ensure Windows Docker Desktop is running
15. Set the user permissions in the WSL session to enable access (permission denied while trying to connect to the docker API at unix:///var/run/docker.sock)
    • sudo usermod -a -G docker <username>

Now run the Kokoro docker command inside your WSL2 image:

• CPU only:
  • docker run –gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest
• NVidia gpu:
  • docker run –gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest-cu128

Connect with the default web link:

• http://localhost:8880/web/

Subsequent runs:

1. Ensure the Docker Desktop is running
2. Start the WSL -d <image> you set up in the first part that’s has all the CUDA stuff installed
3. run the Kokoro docker command inside WSL
4. Connect your web browser to the localhost address

Raw WSL history (commands in WSL)

You can’t follow this 100% directly. Note you have to stop and restart/reboot WSL and desktop as you do the other steps above.

# NVidia CUDA setup inside WSL2 Ubuntu 24.04 image
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/13.3.0/local_installers/cuda-repo-wsl-ubuntu-13-3-local_13.3.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-13-3-local_13.3.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-13-3-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-13-3
exit
sudo apt install nvidia-cuda-toolkit
nvcc --version
exit
sudo usermod -a -G docker matt
exit
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest-cu128