More Performance from Every Byte: How Full-Stack Codesign Solves AI's Memory Challenge
NVIDIA, Thursday, November 5th, 2026
NVIDIA explains how full-stack codesign of GPUs, CPUs, networking, storage and software gets more useful work from scarce AI memory.
Larger models, longer contexts and agentic workloads are increasing pressure on GPU high-bandwidth memory while supply tightens and prices rise.
NVIDIA says simply adding HBM runs into the memory wall, where bandwidth rather than capacity limits throughput, so it codesigns its full stack to use the whole memory and storage hierarchy.
Techniques include shrinking and distributing model weights across the NVL72 domain, shrinking KV cache and offloading inactive context to cheaper tiers, and compacting RAG embeddings with indexes kept outside HBM.
On the SemiAnalysis AgentX benchmark, NVIDIA says Vera Rubin NVL72 delivers up to 30x higher throughput per dollar than GB300 NVL72 with only about 3% more HBM per GPU.