How to Install NVIDIA Container Toolkit and Use GPUs with Docker Containers
Introduction
To use an NVIDIA GPU with Docker containers, you'll need to set up the NVIDIA Container Toolkit, which allows Docker to access NVIDIA GPUs for running GPU-accelerated applications. In this article, we will explore how to use an NVIDIA GPU in Docker, including the installation process, necessary software, and how to verify GPU usage.
What's NVIDIA Container Toolkit?
The NVIDIA Container Toolkit is a set of tools and libraries that enable GPU-accelerated Docker containers. It allows you to leverage NVIDIA GPUs within containers by providing the necessary drivers and runtime components. This toolkit is essential for running CUDA applications, machine learning workloads, and other GPU-accelerated tasks in Docker containers.
Nvidia-Docker2 vs Nvidia-Container-Toolkit
Nvidia-Docker2: This package was the original way to enable GPU support in Docker containers. It included the necessary runtime components and configurations to integrate NVIDIA GPUs with Docker. However, it has been deprecated in favor of the more flexible nvidia-container-toolkit.
Nvidia-Container-Toolkit: This is the current and recommended way to enable NVIDIA GPU support in Docker containers. It is more modular and decouples the runtime from the Docker package, allowing for greater flexibility and easier updates. This toolkit allows you to use the --gpus flag in Docker to specify GPU resources.
Key Features of NVIDIA Container Toolkit
NVIDIA GPU Support: Access NVIDIA GPUs from within Docker containers.
CUDA Toolkit Integration: Provides necessary components to run CUDA applications.
Compatibility: Works with different versions of Docker and various NVIDIA GPUs.
Why do I need the Nvidia Container Toolkit?
The NVIDIA Container Toolkit is essential if you are working with Docker or other containerization platforms and need access to the GPU's computational capabilities inside your containers. It allows users to build and run GPU accelerated containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.
Prerequisites
CUDA-capable GPU: Ensure you have an NVIDIA GPU installed.
Docker: Docker must be installed on your system. You can install Docker by following the official guide here.
NVIDIA Driver: Ensure that the appropriate NVIDIA driver is installed on your system. You can check the installation with the following command: nvidia-smi.
6 Steps to Install NVIDIA Container Toolkit
Step 1. Configure the production repository
$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.listStep 2. Update the packages list from the repository
$ sudo apt-get update
Step 3. Install the NVIDIA Container Toolkit packages
$ sudo apt-get install -y nvidia-container-toolkit
Step 4. Configure the container runtime by using the nvidia-ctk command
$ sudo nvidia-ctk runtime configure --runtime=docker
The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
Step 5. Restart the Docker daemon
$ sudo systemctl restart docker
Step 6. Verify the Setup
Run a test container:
$ sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi $ sudo docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi
This command should output information about your GPU, confirming that the container has access to the GPU.
How to Use an NVIDIA GPU with Docker Containers?
Now, you can run containers with GPU access using the NVIDIA runtime. For instance, to run a container with specific GPU access:
To access all GPUs:
$ docker run --gpus all nvidia/cuda:12.1-base nvidia-smi
To access a specific GPU (e.g., GPU 0):
$ docker run --gpus '"device=0"' nvidia/cuda:12.1-base nvidia-smi
Tips and Best Practices
Check GPU Usage: Use nvidia-smi inside your container to ensure GPU usage is as expected.
Manage GPU Resources: Use --gpus flag options to specify GPU limits, e.g., --gpus "device=0,1" for multi-GPU setups.
Monitoring: Integrate GPU metrics with your monitoring setup to ensure efficient GPU utilization.
How do I specify which GPU to use Docker?
GPUs can be specified to the Docker CLI using either the --gpus option starting with Docker 19.03 or using the environment variable NVIDIA_VISIBLE_DEVICES . This variable controls which GPUs will be made accessible inside the container.
Conclusion
Following these steps will enable you to leverage NVIDIA GPUs within your Docker containers, allowing you to run GPU-accelerated applications seamlessly. Let me know if you need any further assistance with this setup!