AI’s soaring costs and power demands require scrutiny. DeepSeek’s latest model update offers exactly that. With V3.1, the company leans into an ultra-efficient 8-bit floating-point approach called FP8 and signals support for upcoming Chinese accelerators. While the engineering is complex, the goal is simple: move the same or better intelligence through the system. The system achieves its objectives by using fewer bits, less memory traffic, less bandwidth, and less power per task. In plain terms, that means lower cloud costs today and more room for greener AI tomorrow.
DeepSeek is not starting from zero. The team previously trained a frontier-class Mixture-of-Experts model, DeepSeek-V3, on export-compliant NVIDIA H800s while keeping the training budget lean by industry standards. V3.1 builds on that base with efficiency-first choices and a public note that its new precision scheme is tailored for domestic chips expected in China. Investors, data center planners, and policymakers are now recalibrating their strategies for the next phase of the AI race based on this signal.
Quick Facts: V3.1 at a Glance
- Core Technology: Leverages FP8 efficiency and microscaling to dramatically reduce AI model cost and AI energy consumption.
- Key Innovation: A custom UE8M0 scaling format designed to maximize performance on existing and future AI hardware accelerators.
- Strategic Goal: Signals a clear path toward high-performance models that can run on domestic AI chips in China, independent of export-restricted hardware.
- Architectural Advantage: Built on a Mixture-of-Experts (MoE) foundation, which minimizes data movement and power draw by activating only a fraction of the model’s parameters for any given task.
Why This Matters Now
Training and running large models can be astonishingly power hungry. Methods that cut memory movement and increase compute-per-watt directly reduce monthly operational costs and the overall energy footprint of AI infrastructure. These efficiency gains are one piece of a much larger puzzle; they help explain why Chinese AI models are consistently priced at a fraction of their Western counterparts. With the new DeepSeek V4 price increase now adding peak hour multipliers, our guide to DeepSeek price increase alternatives compares MiMo-V2.5-Pro, Kimi and MiniMax on real cache hit rates.
A Quick Definition: What FP8 Means in Practice
FP8 is a family of numerical formats that stores values in just eight bits. NVIDIA’s production variants on Hopper GPUs are E4M3 and E5M2, which trade off precision and dynamic range.
A reduction in numerical precision can seem counterintuitive. In deep learning, however, careful scaling and training recipes keep accuracy intact while slashing memory bandwidth and boosting Tensor Core throughput. FP8’s raw efficiency makes it a headline tool for training and inference.
What V3 Already Proved (and Why It Matters)
DeepSeek-V3 established that a very large Mixture-of-Experts model can be trained at competitive quality while prioritizing efficiency across the stack. The technical report describes a 671-billion-parameter MoE where only about 37 billion parameters are active per token. This specific design actively limits data movement through memory at any given moment.
The team also introduced several key innovations for better learning dynamics:
- Multi-Head Latent Attention (MHLA): To restructure attention and reduce overhead.
- Auxiliary-Loss-Free MoE Balancing: To improve the efficiency of the Mixture-of-Experts routing.
- A Multi-Token Prediction Objective: To enhance the model’s predictive accuracy.
All of this was trained on 14.8 trillion tokens with approximately 2.788 million H800 GPU hours.
The Architecture in Plain English
Mixture-of-Experts routes each token through a small subset of specialized “experts” rather than the entire network. The architecture functions like a specialized consultation, routing tasks to a few relevant experts instead of an entire department. This approach keeps both compute and data movement bounded. Multi-Head Latent Attention restructures attention to reduce overhead without throwing away signal. These choices keep the active model size modest during inference while preserving rich capacity on demand.
Why Fewer Active Parameters Help Energy and Cost
Activating only a slice of the model per token means the platform moves fewer weights across interconnects and HBM. This reduction is critical, as data movement is expensive in both time and power. This efficiency multiplies at the cluster scale, where fabric bandwidth and cooling dominate operational costs.
A similar theme is prevalent in an explainer on DeepSeek’s open-sourced 3FS storage, which targets bottlenecks around fast, parallel data access for training pipelines.
The Training Reality: H800 Instead of H100
DeepSeek trained V3 on NVIDIA’s H800, the export-compliant sibling of H100 that still supports FP8 Tensor Cores but with reduced interconnect. That constraint forced careful engineering around communication overhead and parallelism, a choice that paid off in total training cost. It also foreshadowed the team’s interest in models that run well on non-flagship or domestically produced accelerators.
Performance Baselines That Anchor the Conversation
The V3 report outlines competitive results against open and closed peers at the time, including reasoning and knowledge benchmarks, while documenting a stable training run. These baselines are essential for evaluating any new claims for V3.1, as they establish a measured floor for both capability and cost.
What Is New in V3.1: UE8M0 FP8, Hybrid Inference, And A Path to Domestic Chips
V3.1 keeps the V3 foundation and introduces two major advancements. First, DeepSeek says it trained V3.1 using a UE8M0 FP8 scale scheme for both weights and activations to ensure compatibility with microscaling data formats. Second, it advertises a hybrid inference structure that speeds up “thinking” and strengthens agent-style tool use.
The company also told Chinese social media followers that the precision choice aligns with next-generation domestic chips that are expected to arrive soon. Treat that as a directional signal rather than a formal compatibility list.
UE8M0 And Microscaling, Explained Simply
Microscaling on NVIDIA Blackwell uses narrow floating-point data types with a per-block scale that rescues range and stability. In that context, UE8M0 is best read as an exponent-only scaling factor stored in eight bits that accompanies FP8 payloads. This functions as a micro-regulator for each block of numbers, keeping them in a healthy range without the overhead of a full-precision mantissa.
Researchers have demonstrated that careful rounding and scale computation allow MXFP8 to converge on long pre-training runs, making it an ideal recipe for working in reduced precision.
Why This Matters for Energy and Cost
Scaling factors make FP8 practical across more layers because they reduce the risk of numerical overflow or underflow. The payoff translates directly to less bandwidth through HBM, fewer bytes on the wire between devices, and higher effective Tensor Core throughput. The result is more tokens processed per joule and per dollar—the core value proposition of the V3.1 architecture.
Hybrid Inference and Faster “Thinking”
DeepSeek describes a hybrid path where the system can switch between lighter and heavier reasoning modes, improving latency without abandoning quality when a question requires deeper chains of thought or tool calls. This hybrid approach is especially relevant for agent workflows that must interleave code execution, search, or structured tools with language reasoning. The company framed these upgrades in its public release notes for V3.1.
Reader Context: Where Hybrid Modes Help
Short prompts and routine tasks benefit from a lean path that starts producing tokens quickly. Complex analytics, planning, or multi-tool actions need a deeper path that invests more compute to stay accurate. An intelligent router capable of this distinction can simultaneously improve user experience and reduce operational cloud costs.
The China Chip Angle: What V3.1 Is Pointing Toward
DeepSeek’s mention of domestic “next-generation chips” lines up with an emerging ecosystem around Huawei’s Ascend 910C and new cluster systems like CloudMatrix 384, along with training accelerators from Cambricon and Enflame. Huawei’s platform connects 384 Ascend 910C processors in an all-to-all or super-node configuration and publicly claims high BF16 throughput with a tradeoff in power draw compared with NVIDIA’s NVL72. This environment is precisely where FP8 combined with microscaling can boost effective performance per rack. Keep in mind that specific support matrices have not been published by DeepSeek.
Other Domestic Accelerators Readers Should Know
Each of these products differs in maturity and ecosystem support. They nevertheless represent realistic targets for FP8-friendly inference and, over time, more competitive training.
Technical Deep Dive: FP8, Microscaling, and the UE8M0 Format
The idea is simple:
- Store numbers in eight bits instead of sixteen or thirty-two.
- Let the hardware compensate with smart scaling to keep accuracy on track.
NVIDIA’s Hopper generation implements two FP8 formats, E4M3 and E5M2, that cut memory movement and raise tensor throughput compared with FP16 or BF16. Moving data through high-bandwidth memory and across server interconnects is one of the biggest energy and time costs during training and inference. In deep learning, minimizing this cost is critical for efficiency. See the Hopper FP8 primer and Transformer Engine documentation for a clear overview of how these formats work in practice.
FP8 On Hopper, In Plain Language
FP8 on Hopper combines low-precision math with guardrails that keep training stable. The Transformer Engine can choose E4M3 or E5M2 on a per-tensor basis and even use a hybrid setting that mixes both during the forward and backward passes. This design preserves accuracy while letting the system process more tokens per second and per watt.
Why Less Precision Can Still Be Accurate
Neural networks tolerate lower precision when tensors are scaled into a safe numerical range. Hopper automates much of this through per-tensor amax tracking and dynamic scaling. The result is a cascade of efficiencies:
- Smaller activations and weights on the wire.
- Faster mathematical operations in the cores.
- Lower energy expenditure on memory traffic.
Microscaling (MXFP8): The Blackwell Twist
Blackwell adds microscaling formats, including MXFP8, that attach a small-scale value to blocks of FP8 data. This per-block scale expands the usable range and improves stability during large pre-training runs.
A recent engineering paper details training recipes that keep multi-trillion-token runs convergent in MXFP8 by adjusting rounding and scale computation. Microscaling functions like a precision dial for each block of numbers, ensuring they remain in the optimal range for low-precision math.
What Microscaling Buys You
Per-block scaling attacks a core bottleneck: it reduces overflows and underflows without requiring a fallback to wider formats. The direct result? Fewer bytes moved through HBM, higher effective throughput, and a better energy profile at the rack level. It also opens the door for accelerators that implement compatible FP8 paths to run modern LLMs efficiently.
What UE8M0 Likely Is and Why DeepSeek Uses It
To ensure microscaling compatibility, DeepSeek trained V3.1 using a UE8M0 FP8 scale format on both weights and activations. On the model card and recent commit notes, UE8M0 appears as an exponent-only scale that accompanies FP8 payloads. It behaves like a compact scale field, not a full floating-point value with a mantissa. This design keeps overhead low while enabling robust scaling.
Why UE8M0 Matters for Cost and Energy
A lightweight scale format allows more layers to live comfortably in FP8. That means fewer bytes per tensor, less bandwidth on the fabric, and more tokens per joule. For operators, the outcome translates to lower power draw and reduced cooling pressure. For users, the result is lower cloud bills and faster responses on the same hardware.
What DeepSeek Said About Domestic Chips
DeepSeek publicly stated that its UE8M0 FP8 precision is optimized for “soon-to-be-released next-generation domestic chips.” The company did not list specific vendors or SKUs, so treat this as a directional alignment rather than a support matrix. Multiple outlets captured the WeChat comment and the broader point that V3.1 is leaning into formats that emerging Chinese accelerators can implement.
The Hardware Ecosystem: From NVIDIA H800 to Domestic Accelerators
DeepSeek’s published technical report for V3 confirms training on NVIDIA H800 clusters. H800 is the export-compliant sibling of H100, with industry reporting indicating reduced chip-to-chip transfer rates while retaining Hopper’s FP8 Tensor Cores.
This detail matters. It shows the model family was engineered under interconnect constraints, making the later FP8-plus-microscaling posture in V3.1 a natural evolution toward efficient inference and portability.
What Trained V3: NVIDIA H800 At a Glance
- Architecture and precision: Hopper class with FP8 support, using E4M3 and E5M2 through the Transformer Engine.
- Constraint to note: Lower chip-to-chip bandwidth than the H100, according to Reuters reporting on the export variant.
- Operational takeaway: Efficient routing, reduced active parameters per token, and attention optimizations helped keep total training at about 2.788 million GPU hours on H800.
Why Interconnect Matters for MoE
A throttled fabric raises the penalty for routing activations and expert weights between devices. Designs that lower communication volume, plus FP8 to shrink tensor sizes, directly translate to training stability and cost control on export-limited clusters.
Where V3.1 Is Pointing: Domestic Accelerators In China
DeepSeek’s precision choice signals toward an ecosystem that includes Huawei’s Ascend 910C and new large systems such as CloudMatrix 384, along with training cards from Cambricon and Enflame. Public sources describe CloudMatrix as a 16-rack cluster using 384 Ascend 910C processors in a super-node topology. Analysts report roughly 300 PFLOPS BF16 at the system level with tradeoffs in efficiency and software maturity. While not official endorsements from DeepSeek, these systems are the most visible near-term targets for FP8-friendly workloads inside China.
Power And Efficiency Trade-Offs
According to reports, CloudMatrix achieves its higher rack-level compute over NVIDIA systems by using a greater number of chips, which results in significantly more power draw per cluster. The system emphasizes optical interconnects and scale-out density to compensate for lower per-chip performance. This approach heightens the value of formats that cut bandwidth and memory movement, such as FP8 with microscaling.
Other Domestic Options Technologists Should Track
- Cambricon MLU370-X8: Dual-die training and inference card with FP16 and BF16 support, MLU-Link interconnect, and an advertised 200 GB per second communications bandwidth per card. The vendor positions it for eight-card servers and distributed training.
- Enflame DTU 1.0: Training-oriented accelerator with HBM2 on package and 200 GB per second interconnect as presented at Hot Chips 33. Independent coverage confirms PCIe Gen4 x16 and a 300-watt envelope.
Software Ecosystem and Portability
The maturity of compilers, kernels, and ecosystem tooling varies widely outside CUDA. DeepSeek’s specific mention of DeepGEMM and the use of a format like UE8M0 are therefore notable. They point to a portability strategy that can map core kernels to different accelerators while keeping the numerical contract stable under FP8. Readers should watch for official runtime and kernel releases that target Ascend, Cambricon, or Enflame back ends.
Reality Check for Readers and Planners
- DeepSeek has not published a support matrix for V3.1 on specific domestic chips. Treat mapping to any vendor as an informed inference, not a confirmed configuration.
- For now, cost and capability estimates should be anchored in the official V3 technical report and the known properties of the H800. Consider domestic systems like CloudMatrix 384 as indicative of where FP8-first models may land in China over the next deployment cycles.
The Energy and Efficiency Dividend for AI: How FP8 And Microscaling Cut Power, Bandwidth, And Cloud Costs
Large language models burn energy every time tensors move through high-bandwidth memory and across server interconnects. FP8 precision reduces those bytes on the wire, while microscaling preserves numerical stability so accuracy holds up. NVIDIA’s Transformer Engine on Hopper introduced production FP8 formats E4M3 and E5M2, which increase matrix throughput and shrink memory traffic compared with FP16 or BF16, directly improving tokens per joule and tokens per dollar.
How FP8 Delivers Real Energy Savings
FP8 stores numbers in eight bits, then pairs them with automatic scaling so layers do not overflow or underflow. The practical win is less data moved through HBM and NVLink plus faster tensor cores, which means lower power draw at identical throughput or higher throughput at the same power. NVIDIA’s FP8 primer and Hopper architecture notes document these gains for modern transformers.
Why This Matters Inside Data Centers
Cooling often tracks compute almost 1:1 at conventional facilities. Cutting memory movement reduces heat, which lowers fan speeds, chiller load, and water use. For a broader sustainability lens, pair these model-level savings with facility-level strategies in modern tech innovations in data center cooling and energy supply options in Oklo’s nuclear vision for tech datacenter power.
Microscaling On Blackwell: Stability Without Going Back to BF16
Microscaling attaches a compact scale value to small blocks of FP8 data, widening the effective range so more layers can stay in low precision during long pre-training runs. Engineering recipes for MXFP8 show how per-block scales and careful rounding make FP8 training converge like BF16 while keeping the bandwidth win. This is the mechanism that turns reduced precision into dependable energy savings at cluster scale.
DeepSeek’s UE8M0 In This Picture
DeepSeek states that V3.1 was trained using a UE8M0 FP8 scale on both weights and activations to ensure microscaling compatibility. Read this as a lean, exponent-only scale that travels with FP8 payloads. The intent is clear even if the exact encoding is still being formalized in public docs: keep tensors stable in FP8 so the model runs faster and cooler without sacrificing accuracy.
Localized AI Supply Chains and Greener Compute: Self-Sufficiency as a Sustainability Strategy
DeepSeek’s UE8M0 FP8 choice dovetails with China’s push to build a domestic AI stack where models, compilers, and chips evolve together. The company publicly hinted that V3.1’s precision is aligned with soon-to-launch domestic accelerators, a signal covered by regional tech outlets and wire services. Framed through sustainability, shorter supply chains and local silicon can trim logistics emissions while improving resilience.
The Domestic Chip Context You Should Know
Chinese industry reporting points to near-term deployments on Huawei Ascend 910C clusters such as CloudMatrix 384, combined with maturing options from Cambricon and Enflame. These systems emphasize scale-out design and optical interconnects to offset per-chip gaps versus NVIDIA. Precision formats that cut bandwidth, such as FP8 with microscaling, therefore deliver a disproportionately high impact in this ecosystem.
Evidence, Not Hype
- CloudMatrix 384 was unveiled publicly at WAIC and reported as a 384-chip Ascend 910C super-node designed to rival GB200 NVL72 at the system level, with tradeoffs in efficiency.
- Cambricon MLU370-X8 lists FP32, FP16, and BF16 support, ~96 TFLOPS FP16 or BF16, and MLU-Link interconnect on official product pages.
- Enflame DTU 1.0 Hot Chips materials confirm on-package HBM2 and a 200 GB per second device interconnect, oriented to data center training.
From Sovereignty to Your Devices: What FP8-First Models Mean for Consumers
When model providers squeeze more work from each watt, costs decline and availability rises. Efficiency-centric designs like DeepSeek-V3 and V3.1 can reduce serving costs on cloud platforms today and make edge AI more practical tomorrow as compact accelerators adopt FP8 and microscaling.
This process translates into tangible benefits for consumers, including:
- Faster features in productivity tools.
- Lower-latency assistants on laptops and phones.
- A viable path to more private, on-device inference.
Why Lower Precision Helps at the Edge
Edge devices are thermally and power constrained. If accelerators inherit stable FP8 pipelines, vendors can ship capable local models without large batteries or fans. NVIDIA’s documentation on Transformer Engine provides the blueprint for FP8 layer behavior that competing vendors can mirror, while Blackwell-era work on MXFP8 shows how to keep accuracy steady at low precision.
Signals Already Pointing This Way
- V3’s cost footprint shows what careful routing and reduced precision can achieve even on export-limited H800 clusters. Lower cost to train correlates with lower cost to serve.
- Coverage of DGX Spark and CUDA on RISC-V points to a wider shift toward more accessible computing and a more diverse ecosystem as software stacks open up toward democratized compute and diversified ecosystems that benefit end users as software stacks open up.
The Inevitable Path to Efficient AI
What is the final takeaway from DeepSeek’s V3.1 strategy? It is a clear and decisive statement: the future of scalable AI is efficient AI. The era of growth at any cost, measured only by parameter count, is giving way to a more mature phase defined by compute-per-watt. The architectural choices are clear: mixture-of-experts, hybrid inference, and a deep commitment to FP8 with microscaling. These choices directly counter the unsustainable energy and financial costs of current-generation models.
This shift has profound implications. For data center operators, it offers a tangible path to reducing operational expenses and thermal loads. For the global AI ecosystem, it signals the rise of a multipolar hardware landscape where optimized models can thrive on diverse, localized silicon. DeepSeek’s focus on domestic accelerators is not merely a geopolitical footnote; it is a blueprint for a more resilient and accessible AI supply chain. DeepSeek V3.1 demonstrates that the most intelligent move in the AI race is not just building bigger models but building smarter, leaner ones.
