Tag: inference
All the articles with the tag "inference".
NVIDIA is not a GPU company anymore
A GPU-sharing problem I never solved cleanly years ago stayed with me, so I spent a weekend back in the NVIDIA world to see what had changed. What I found is that NVIDIA is no longer a GPU vendor. It is a full stack, from CUDA and the warp scheduler at the bottom, through NCCL, NVLink, NVSwitch, and the Mellanox fabric, through Grace CPUs and DGX systems and DGX Cloud, through TensorRT, vLLM, Triton, and Dynamo, up to build.nvidia.com with NIMs, Nemotron, Blueprints, Skills, and NeMo Guardrails. These are my weekend findings, and they end where my old problem started: how you share a GPU.