DeepSeek’s DeepGEMM Overhaul Signals Nvidia Blackwell Integration Amid Compute Squeeze
Chinese artificial intelligence developer DeepSeek has restructured its core computing library to radically optimize Mixture-of-Experts (MoE) architectures, deploying a "Mega MoE" framework that strongly indicates the company is leveraging Nvidia Corp.’s advanced Blackwell hardware despite ongoing semiconductor export restrictions.
The quiet update to the open-source DeepGEMM repository, spotted by infrastructure developers this week, marks a pivot from theoretical algorithm design to aggressive hardware utilization. By fusing fragmented computational steps into a continuous pipeline, DeepSeek aims to eliminate GPU idle times that typically plague massive AI models.
Industry analysts note the inclusion of FP4 (4-bit floating point) indexing specifically points toward Nvidia’s latest B-series accelerators, challenging recent market speculation that Chinese AI champions had fully transitioned to domestic silicon in 2026.
Mega MoE Fuses Fragmented GPU Pipelines
Traditional MoE operations operate akin to a fragmented assembly line—dispatching tokens, applying linear transformations, processing through activation functions (SwiGLU), and combining results. Each step previously required initializing a separate kernel, creating bottlenecks via constant inter-GPU data communication.
DeepSeek’s infrastructure team engineered Mega MoE to fuse these disparate steps into a single mega-kernel. Crucially, the architecture overlaps Tensor Core computation with NVLink data transmission. This concurrent processing prevents GPUs from pausing to await data transfers, significantly boosting cluster utilization rates for large-scale deployments.
FP4 Integration Challenges Domestic Chip Narrative
Beyond the mega-kernel, the DeepGEMM update introduces FP8 by FP4 precision combinations and an FP4 indexer tailored for Multi-Query Attention (MQA) logits. The explicit focus on FP4—a precision standard heavily optimized in Nvidia's Blackwell architecture—suggests DeepSeek is training on top-tier US silicon.
This technical footprint contradicts prevalent rumors from late 2025 that the company relied exclusively on domestic AI accelerators. Utilizing FP4 allows developers to push the boundaries of computational efficiency, extracting maximum matrix multiplication throughput from limited hardware inventories.
JIT Compilation Paves Path For DeepSeek-V4
DeepSeek has concurrently redefined DeepGEMM as a unified, high-performance Tensor Core library. The system now utilizes a lightweight Just-In-Time (JIT) compilation module, bypassing the need for standard CUDA compilation during installation.
While DeepSeek stated that Mega MoE remains under active development with performance metrics pending, the infrastructure overhaul lays the groundwork for next-generation models. The engineering focus on extreme optimization indicates that the foundational architecture for what the industry anticipates as DeepSeek-V4 is already active in multi-node testing environments.
Related Coverage：
DeepSeek V4 Targets Late April Launch, Betting on Trillion-Parameter Efficiency