DeepSeek-R1 is an open mannequin with state-of-the-art reasoning capabilities. As an alternative of providing direct responses, reasoning fashions like DeepSeek-R1 carry out a number of inference passes over a question, conducting chain-of-thought, consensus and search strategies to generate the most effective reply.
Performing this sequence of inference passes — utilizing cause to reach at the most effective reply — is named test-time scaling. DeepSeek-R1 is an ideal instance of this scaling legislation, demonstrating why accelerated computing is vital for the calls for of agentic AI inference.
As fashions are allowed to iteratively “suppose” by way of the issue, they create extra output tokens and longer technology cycles, so mannequin high quality continues to scale. Important test-time compute is vital to allow each real-time inference and higher-quality responses from reasoning fashions like DeepSeek-R1, requiring bigger inference deployments.
R1 delivers main accuracy for duties demanding logical inference, reasoning, math, coding and language understanding whereas additionally delivering excessive inference effectivity.
To assist builders securely experiment with these capabilities and construct their very own specialised brokers, the 671-billion-parameter DeepSeek-R1 mannequin is now out there as an NVIDIA NIM microservice preview on construct.nvidia.com. The DeepSeek-R1 NIM microservice can ship as much as 3,872 tokens per second on a single NVIDIA HGX H200 system.
Builders can check and experiment with the appliance programming interface (API), which is predicted to be out there quickly as a downloadable NIM microservice, a part of the NVIDIA AI Enterprise software program platform.
The DeepSeek-R1 NIM microservice simplifies deployments with help for industry-standard APIs. Enterprises can maximize safety and information privateness by operating the NIM microservice on their most popular accelerated computing infrastructure. Utilizing NVIDIA AI Foundry with NVIDIA NeMo software program, enterprises can even be capable of create personalized DeepSeek-R1 NIM microservices for specialised AI brokers.
DeepSeek-R1 — a Good Instance of Check-Time Scaling
DeepSeek-R1 is a big mixture-of-experts (MoE) mannequin. It incorporates a powerful 671 billion parameters — 10x greater than many different widespread open-source LLMs — supporting a big enter context size of 128,000 tokens. The mannequin additionally makes use of an excessive variety of consultants per layer. Every layer of R1 has 256 consultants, with every token routed to eight separate consultants in parallel for analysis.
Delivering real-time solutions for R1 requires many GPUs with excessive compute efficiency, related with high-bandwidth and low-latency communication to route immediate tokens to all of the consultants for inference. Mixed with the software program optimizations out there within the NVIDIA NIM microservice, a single server with eight H200 GPUs related utilizing NVLink and NVLink Change can run the total, 671-billion-parameter DeepSeek-R1 mannequin at as much as 3,872 tokens per second. This throughput is made attainable by utilizing the NVIDIA Hopper structure’s FP8 Transformer Engine at each layer — and the 900 GB/s of NVLink bandwidth for MoE skilled communication.
Getting each floating level operation per second (FLOPS) of efficiency out of a GPU is vital for real-time inference. The next-generation NVIDIA Blackwell structure will give test-time scaling on reasoning fashions like DeepSeek-R1 an enormous enhance with fifth-generation Tensor Cores that may ship as much as 20 petaflops of peak FP4 compute efficiency and a 72-GPU NVLink area particularly optimized for inference.
Get Began Now With the DeepSeek-R1 NIM Microservice
Builders can expertise the DeepSeek-R1 NIM microservice, now out there on construct.nvidia.com. Watch the way it works:
With NVIDIA NIM, enterprises can deploy DeepSeek-R1 with ease and guarantee they get the excessive effectivity wanted for agentic AI programs.
See discover relating to software program product data.
