Supermicro KV Cache Platform with IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens
StorageReview, Thursday, November 5th, 2026
Supermicro, IBM and NVIDIA launched a shared flash KV cache platform they say cuts time to first token 56x at 130,000-token prompts.
Supermicro announced a KV cache storage solution with IBM and NVIDIA that moves the key-value state behind long-context inference off the GPU onto a shared all-flash tier.
It combines Supermicro Petascale storage servers on AMD EPYC 9005 processors, IBM Storage Scale, NVIDIA Dynamo and Spectrum-X Ethernet, backed by a reference architecture IBM published as a Redbook.
The companies claim a 56x reduction in time to first token on 130,000-token prompts versus recomputing context on the GPU, which matters for agentic, coding, RAG and multi-turn workloads.
The reference design uses eight 1U nodes, each with 16 Micron 7.68TB E3.S drives for about 983TB raw, and Supermicro offers it as a turnkey rack-integrated build.