16:30
2026-09-25
leimao.github.io
ai-infrastructure
CUDA Device Max Connections
NVIDIA's CUDA_DEVICE_MAX_CONNECTIONS environment variable defaults to 8 concurrent hardware connections to the GPU, capping real concurrency regardless of how many CUDA streams a developer creates, ac…