DeepSeek has co-authored a landmark systems research paper with teams from Peking University and Tsinghua University, proposing a new inference architecture called DualPath that addresses one of the most consequential but least visible constraints on large-scale agentic AI deployment: storage I/O bandwidth.

The paper arrives as the AI industry accelerates its shift from single-turn chatbots toward autonomous agents capable of multi-step reasoning and tool use — a transition that fundamentally rewrites the hardware bottleneck equation. Rather than compute power, it is the speed at which a system can retrieve historical context from storage that now limits performance in these settings.

DualPath's core innovation is a dual-path KV-Cache loading mechanism that redistributes I/O load across both prefill and decode nodes in a GPU cluster, effectively pooling previously idle storage bandwidth. In large-scale validation on a 1,152-GPU cluster, the system achieved up to a 1.87x improvement in offline inference throughput and an average 1.96x improvement in online service throughput — results that carry direct implications for the economics of running frontier AI models at scale.

The Architecture Problem That Compute Alone Cannot Solve

The emergence of agentic AI workflows — where a model interacts with its environment across dozens or even hundreds of turns — creates a fundamentally different computational profile than conversational AI. In these settings, each new request appends only a small number of tokens to an already lengthy context, meaning KV-Cache hit rates typically exceed 95%. The GPU is not waiting to compute; it is waiting for data.

Because GPU high-bandwidth memory (HBM) and host DRAM cannot hold the full KV-Cache for long-context agentic tasks, the cache must be offloaded to slower SSD-based external storage. In conventional prefill-decode separated architectures — the dominant design in modern large model inference systems — all KV-Cache retrieval flows through the prefill node's storage network interface card (SNIC). This creates a severe asymmetry: prefill nodes run their storage bandwidth at full saturation, while decode nodes' equivalent bandwidth sits largely idle.

The paper also contextualizes this imbalance within a broader hardware trend: GPU compute capacity (measured in FLOPS) has grown substantially faster than network bandwidth or memory capacity across successive generations of NVIDIA hardware, making the compute-to-I/O ratio progressively more lopsided. The implication is that the bottleneck identified in this research is structural, not incidental, and will intensify as model capabilities advance.

A Second Path Through Idle Infrastructure

DualPath's solution is conceptually direct but technically demanding to execute. The system introduces a secondary KV-Cache loading channel — a "storage → decode → prefill" path that runs in parallel with the conventional "storage → prefill" path. By dynamically allocating data traffic across both routes, DualPath converts what was a single-node I/O constraint into a cluster-wide resource pooling problem, aggregating the storage bandwidth of all nodes simultaneously.

In the decode-side path, KV-Cache data is first loaded into the decode node's memory buffer, then streamed layer-by-layer to the prefill node via inter-node high-speed compute networking using RDMA (Remote Direct Memory Access) technology. This pipelined, layer-wise transmission is designed to overlap data movement with computation, minimizing latency penalties.

The architecture supports DeepSeek-V3.2 660B, DS 27B, and Qwen2.5-32B models, and was validated using real agentic reinforcement learning trajectory datasets — a methodological choice that strengthens the practical relevance of the results.

Engineering the Gap Between Concept and Production

The paper devotes considerable attention to two implementation challenges that separate a theoretical architecture from a deployable system.

The first is network traffic interference. Introducing additional KV-Cache transfers across the compute network risks colliding with latency-critical collective communication operations inherent to mixture-of-experts (MoE) architectures, such as AllToAll operations. DualPath addresses this through a compute network interface card (CNIC)-centric traffic management design that enforces strict Quality of Service (QoS) separation. Inference communication is assigned to a high-priority channel allocated 99% of available bandwidth; KV-Cache transfers are routed through a low-priority channel that transmits only during idle intervals in the compute network. The result is effective traffic isolation without degrading model inference latency.

The second challenge is dynamic load balancing across a heterogeneous, time-varying request stream. DualPath introduces an adaptive request scheduler that monitors disk read queue lengths across nodes and uses token count as the primary load metric. Nodes are classified into three states — overloaded, low read queue, and high read queue — and new requests are preferentially routed to nodes that are neither overloaded nor backlogged on storage reads. Within individual nodes, a time-estimation-based batching mechanism groups requests with similar projected execution times, reducing the GPU idle "bubbles" that arise when a batch must wait for its slowest member.

Performance at Scale: Throughput Gains Hold Across Cluster Sizes

The evaluation results, conducted on an NVIDIA Hopper GPU cluster with InfiniBand networking and a 3FS distributed storage system, demonstrate consistent gains across both offline and online workloads.

In offline batch inference — the mode most relevant to reinforcement learning rollout pipelines — DualPath reduced task completion time substantially for the 660B model, with peak throughput improvements of 1.87x over the baseline. Performance gains remained stable across varying agent concurrency levels, maximum context lengths, and per-turn token append sizes, indicating that the system has genuinely eliminated the storage network as a binding constraint rather than optimizing around it in specific configurations.

In online serving under strict latency service-level agreements — specifically, a time-to-first-token threshold of under four seconds — DualPath supported up to 2.25x higher request arrival rates compared to the baseline while maintaining low end-to-end generation latency. Ablation studies confirmed that both the dual-path loading mechanism and the adaptive scheduling algorithm are necessary contributors to the observed gains; neither alone accounts for the full improvement.

Perhaps most consequentially for infrastructure planning, the system demonstrated near-linear scaling on the 1,152-GPU cluster configuration (48 prefill nodes, 96 decode nodes), suggesting that the architecture does not introduce new bottlenecks at the scale at which frontier model providers actually operate.

Implications for Agentic AI Infrastructure Investment

The timing of the paper — released ahead of a widely anticipated DeepSeek V4 launch — positions DualPath as a foundational infrastructure component for next-generation model deployments rather than an incremental optimization. The research implicitly argues that as context lengths grow and agentic use cases proliferate, the competitive differentiation in AI infrastructure will increasingly shift from raw GPU count toward systems-level efficiency in managing data movement.

For operators of large-scale AI clusters, the paper's core finding carries a pointed economic message: significant throughput capacity may already exist within deployed hardware, locked behind architectural inefficiencies rather than hardware limitations. Unlocking it requires not more GPUs, but a rethinking of how data flows between the ones already running.