Part I · How it works

5.4 Computing on the GPU, and when your program can't see it

alsocudarocmhipopenclnvidia-smirocm-smigpu computegpgpucuda_visible_devicesrender grouprender nodenvidia container toolkitgpu not detectedtorch.cuda.is_available

TL;DR

Compute frameworks talk to the GPU through a vendor stack: CUDA for NVIDIA, ROCm (HIP) for AMD, oneAPI for Intel. Vulkan and OpenCL work across vendors.
Each needs the kernel driver, the matching user-space libraries, and access to device files: /dev/nvidia*, or /dev/kfd and /dev/dri/renderD*.
When a program says "no GPU", check those three, in that order.

nvidia-smi                  # NVIDIA: driver, CUDA version, memory, users
rocminfo | grep -i name     # AMD
ls -l /dev/dri/; groups     # render nodes; are you in their group?

On your machine

alice@laptopexample

$ atfq 5.4
  card0     Intel, driver i915, shows the boot screen
  Screens   eDP-1 1920x1200 · free: DP-1, HDMI-A-1
  Compute   /dev/dri/renderD128

An example: Alice's laptop, one of the test machines. In your terminal, this block shows your own machine, read live.

Pitfalls

Render nodes usually belong to the render group (sometimes video): add yourself, then log in again (15.3).
Containers see no GPU unless given one: --gpus all with the NVIDIA Container Toolkit, or --device /dev/kfd --device /dev/dri for ROCm.
A Python package built for CPU only, or for another CUDA version, reports no GPU though nvidia-smi works: install the build that matches.
CUDA_VISIBLE_DEVICES or HIP_VISIBLE_DEVICES set to an empty or wrong value hides GPUs from a program.

Deep dive 2 questions

What does "CUDA version" in nvidia-smi mean?

It's the newest CUDA version the installed driver supports, not a CUDA installed on the machine. Programs bring their own CUDA runtime (PyTorch wheels include it); they work as long as the driver is at least as new as their CUDA needs. An old driver with a new framework is the usual cause of "CUDA driver version is insufficient".

What is a render node?

/dev/dri/card0 controls the display, which only the compositor should touch. /dev/dri/renderD128 gives the same GPU for drawing and compute only, with no control over screens, so it can be safely opened by ordinary programs and containers. ROCm adds /dev/kfd for its compute queues.

Commands: nvidia-smi rocminfo clinfo groups
Paths: /dev/dri/ /dev/kfd /dev/nvidia0
See also: 5.1 What a GPU does differently
5.2 GPU drivers: the kernel, Mesa and NVIDIA
15.3 sudo and groups