DeepSeek Open Sources DeepInfer Under Apache 2.0: Production Serving Engine for NVIDIA SM100 and Huawei Ascend 950

On the morning of October 21, 2026, Chinese frontier artificial intelligence laboratory DeepSeek published the open-source repository for DeepInfer under the Apache 2.0 license. DeepInfer represents the high-throughput, bare-metal offline inference engine deployed internally to power production API endpoints for the DeepSeek-V4.1-Flash model series.

The release represents a major milestone for distributed systems engineers, offering an unencumbered, micro-kernel optimized inference stack designed to maximize arithmetic intensity on next-generation accelerator hardware.

Custom Low-Level Kernels and Dual-Architecture Silicon Support

At the core of DeepInfer lies DeepSeek's proprietary suite of high-performance primitives: FlashMLA (an ultra-optimized Multi-Head Latent Attention operator), DeepGEMM (FP8 mixed-precision matrix multiplication routines), DeepEP (an all-to-all communication library engineered for Mixture-of-Experts routing), TileKernels, and DeepSelect. Crucially, the engine delivers co-equal, first-class compilation targets for both NVIDIA SM100 (Blackwell architecture) and Huawei's Ascend 950 NPU, bridging disparate silicon paradigms within a unified runtime codebase.

For large-scale multi-accelerator execution, DeepInfer orchestrates a three-tiered parallelism strategy combining Attention Data Parallelism, Context Parallelism for long-sequence context balancing, and Expert Parallelism driven by Expert Parallelism Load Balancing (EPLB) heuristics to eliminate straggler nodes. It also incorporates DSpark, a speculative decoding implementation leveraging an auxiliary confidence head to verify speculative draft tokens with minimal memory bandwidth consumption.

Scope Boundaries and Ecosystem Integration

Alongside the core engine, DeepSeek released deepseek-recipe, a deterministic Rust library handling high-speed tokenization, JSON schema extraction, and streaming wire protocols. DeepSeek deliberately delineated the initial release boundaries: the runtime focuses strictly on node-level kernel execution, intentionally omitting distributed KV-cache cluster pooling and inter-replica ingress layers to allow clean integration into external container orchestrators like Kubernetes.

“By open-sourcing DeepInfer, we aim to provide the systems community with an uncompromising kernel execution core that eliminates runtime overhead and maximizes hardware arithmetic intensity on modern accelerator architectures.”

The release of DeepInfer equips infrastructure teams with a transparent, highly audited serving framework capable of driving production MoE models. By decoupling inference runtime performance from vendor-specific software stacks, DeepInfer accelerates the broader industry shift toward portable, open systems engineering.