What is a CUDA Context?
A CUDA context is essentially a container for all the resources needed to interact with a specific GPU device from a host (CPU) process. Think of it as the GPU's state as seen by a particular CPU process. Each context is associated with one specific device and one specific host process (though a process can manage multiple contexts for multiple devices).
Think of a CUDA Context as a distributed data structure with:
A "control plane" on the CPU that manages and directs operations A "data plane" on the GPU that stores the actual execution state
When you make CUDA API calls, the CPU-side component of the context interprets these calls and sends appropriate commands to update or use the GPU-side components of the context. This dual-residence nature is why contexts are so important - they maintain the synchronized state between host and device that allows them to work together as a cohesive system.
Key Aspects:
- Resource Management: A context manages GPU resources like memory allocations (device pointers), loaded modules (kernels), streams, and events specific to that context's associated device and process.
- Isolation: Contexts provide isolation. Resources created within one context are generally not directly accessible from another context, even if they target the same physical device.
- CPU Thread Association: While a context belongs to a host process, CUDA API calls relating to a context are typically made from specific CPU threads. CUDA maintains a current context per CPU thread, often managed implicitly or explicitly via context stacks (
cuCtxPushCurrent/cuCtxPopCurrent). - GPU State: It encapsulates the state of the GPU relevant to the host process, including loaded kernels, allocated memory, and configuration settings.
The visualization below illustrates the relationship between CPU threads making API calls, the CUDA contexts they interact with (potentially pushed/popped onto a stack per thread), and the underlying GPU device resources managed by those contexts.