Nemotron 3 Ultra: NVIDIA’s Open 550B Model Built Around Its Own Chips

Nemotron 3 Ultra is a 550B open-weight hybrid model from NVIDIA with open training data and a native 4-bit build. What it is good at and where it trails.

Disclosure: this article is based on NVIDIA’s model card, technical materials and third-party coverage. We have not deployed Nemotron 3 Ultra on our own hardware. Benchmark figures are NVIDIA’s unless another source is named.

NVIDIA released Nemotron 3 Ultra on June 4, 2026, after pre-announcing it at Computex. It is a 550-billion-parameter open-weight mixture-of-experts model with 55 billion active parameters per token, a one-million-token context and a licence that permits commercial use. Unusually, NVIDIA also published the training data and recipes.

The honest summary: Nemotron 3 Ultra is not the smartest open model, and NVIDIA does not really claim it is. Its pitch is speed and efficiency on NVIDIA hardware, especially Blackwell, where its natively 4-bit build runs without the usual quality loss from quantising after training. If you already run NVIDIA GPUs and want an open model you can inspect end to end, it deserves a place on your shortlist. If you want the highest benchmark score per dollar, look elsewhere.

Table of Contents

Nemotron 3 Ultra server rack with status lights
Nemotron 3 Ultra is less a model than a sales argument for Blackwell, and a good one: it was trained to run in 4-bit from day one.

What Nemotron 3 Ultra Is

Nemotron is NVIDIA’s family of open models, and Nemotron 3 comes in three tiers. Nano is about 30 billion parameters, Super about 100 billion, and Ultra about 550 billion. All three share a hybrid design and a one-million-token context window. Nano arrived first; Ultra is the largest and most capable.

NVIDIA positions the family for agentic AI: reasoning, coding, planning and tool calling. The model card describes Nemotron 3 Ultra as designed for strong agentic, reasoning and conversational capabilities, and suitable for complex instruction following and long-context reasoning over very large documents and codebases.

It ships in two forms. The BF16 checkpoint is the full-precision version. The NVFP4 checkpoint is a 4-bit version in NVIDIA’s own format, intended for smaller footprints. Reasoning can be switched on or off per request through the chat template.

The model supports ten languages including English, French, Spanish, Italian, German, Japanese, Hindi, Korean, Brazilian Portuguese and Chinese, according to the model summary.

The Nemotron 3 Ultra Architecture

This is where the model gets interesting. Nemotron 3 Ultra is not a plain transformer. NVIDIA describes it as a LatentMoE hybrid that combines Mamba-2 layers, mixture-of-experts layers and attention layers, with multi-token prediction.

Each piece has a job. Mamba-2 is a state-space architecture that handles long sequences more cheaply than attention, because its cost grows roughly linearly with length rather than quadratically. Attention layers remain for the tasks where precise recall across the context matters. The mixture of experts keeps compute per token at 55 billion parameters while storing 550 billion.

LatentMoE is NVIDIA’s twist. Tokens are projected into a smaller latent dimension for expert routing and computation, which NVIDIA says improves accuracy per byte. Multi-token prediction lets the model draft several tokens at once, which speeds up generation.

The combined effect is throughput. NVIDIA reports roughly five times higher inference throughput than other open frontier models, and The New Stack described it as the fastest among direct competitors such as Kimi-K2.6, Qwen-3.5 and GLM-5.1 at the time of release.

Why Native 4-Bit Matters

Most open models are trained at high precision and then quantised to 4-bit after the fact, which saves memory but costs some quality. Nemotron 3 Ultra was pre-trained using an NVFP4 recipe, sharing the quantisation-aware approach used across the Nemotron 3 family.

AI Weekly made the key point: the interesting thing about the release is not the parameter count but that the same checkpoint comes pre-quantised to NVFP4, which Blackwell GPUs run natively. A model trained to live in 4-bit loses far less when it runs in 4-bit.

The published benchmark table shows how small the gap is. On Terminal Bench 2.1, BF16 scores 56.4 and NVFP4 53.9. On GDPVal, NVFP4 actually scores slightly higher, 47.9 against 46.7. For most workloads, the 4-bit version is the one to run.

Halving or quartering memory means fewer GPUs per instance, more users per GPU, or both. NVIDIA’s own claim is up to 30% cost savings against comparable models. That is a vendor number, but the mechanism is sound.

Licence and Openness

Nemotron 3 Ultra uses the OpenMDW License Agreement, version 1.1. AI Weekly summarised it as granting royalty-free commercial use and explicitly carving model outputs out of the licence, so what the model generates is yours without strings.

The openness goes further than most labs. NVIDIA describes Nemotron as a family with open weights, training data and recipes. Publishing the post-training and pre-training datasets lets researchers and enterprises inspect what the model learned from, reproduce parts of training and build domain variants with a clear provenance trail.

For regulated industries, that provenance can matter more than benchmark points. Knowing what data shaped a model is increasingly part of AI governance reviews, and very few frontier-scale open models offer it.

Nemotron 3 Ultra Benchmarks

Benchmark BF16 NVFP4
Terminal Bench 2.1 56.4 53.9
GDPVal 46.7 47.9
SWE-Bench Verified About 65% to 70.4% depending on scaffold
ProfBench Search 56.0

AI Weekly reported an Artificial Analysis intelligence score near 48. For context, more recent open models have pushed higher: Meta’s Muse Spark Open, released in October, scores 76.4 on the same Terminal-Bench 2.1 test in Meta’s own table, and Kimi K3 scores 85.0. Nemotron 3 Ultra is five months older and was not built to top those charts.

That gap is the honest trade-off. You are buying throughput, a native 4-bit build and open data, not the highest score in the open field.

Where and How to Run It

NVIDIA released the model on Hugging Face, ModelScope, OpenRouter, where a free endpoint was offered at launch, and build.nvidia.com. Several inference providers list it too; one, Haimaker, lists it at $0.50 per million input tokens and $2.50 output with a 16,000-token output cap on its endpoint, so check each provider’s limits.

For self-hosting, Dell lists compatible platforms including the XE9680 with H200 GPUs and the XE9780 with B300 GPUs. At 550 billion parameters, the NVFP4 build needs roughly 275 GB just for weights by simple arithmetic, before the context cache. That is a multi-GPU server, and Blackwell-generation hardware is where it shines.

The hybrid architecture helps with long contexts. Because Mamba layers keep a fixed-size state rather than a cache that grows with every token, long prompts use less memory than in a pure transformer of similar size. For one-million-token workloads, that is a practical advantage.

NVIDIA also reports that Nemotron 3 Nano delivers four times the throughput of Nemotron 2 Nano and cuts reasoning-token use by up to 60%. If Ultra is too large for your hardware, the smaller tiers share the design.

Nemotron 3 Ultra vs Other Open Giants

Model Size Licence Edge
Nemotron 3 Ultra 550B, 55B active OpenMDW-1.1 Throughput, native NVFP4, open data
DeepSeek V4.1 Flash 552B, about 8B active MIT Price and index score
Muse Spark Open 816B, 42B active Muse Model License Higher coding score, US origin
Mistral Large 4 About 1T, 49B active Modified MIT Coding and cyber focus

The comparison clarifies who Nemotron 3 Ultra is for. If you want the cheapest capable API, DeepSeek wins. If you want the highest coding score from a US open model, Muse Spark Open wins. If you want a model optimised for NVIDIA hardware with published training data and a clean output licence, Nemotron 3 Ultra stands alone.

Good Uses

Enterprise agents on NVIDIA infrastructure

AI Weekly reported that thirteen major enterprises, including Palantir, CrowdStrike and Perplexity, were integrating Nemotron 3 models. For companies already standardised on NVIDIA’s stack, the integration path is short.

High-throughput serving

Customer-facing agents that must handle many concurrent sessions benefit most from the throughput gains. Speed per GPU translates directly into lower cost per conversation.

Auditable and sovereign deployments

Open training data makes it easier to answer governance questions about provenance. Combined with self-hosting, that suits public-sector and regulated buyers.

Long-document reasoning

The million-token context and Mamba layers make it well suited to large codebases and document collections on private infrastructure.

Why NVIDIA Gives Away a Frontier-Scale Model

It is fair to ask why a chip company spends heavily to train a 550-billion-parameter model and then gives it away. The answer is that Nemotron 3 Ultra is an argument for NVIDIA hardware as much as it is a model.

Every open model that runs best on NVIDIA GPUs makes those GPUs more valuable. A model pre-trained in NVFP4 is a showcase for Blackwell’s native 4-bit support: the same task on older hardware, or on a competitor’s accelerator without that format, runs slower or needs more memory. NVIDIA benefits whenever developers standardise on a model that rewards its newest chips.

There is an ecosystem logic too. NVIDIA sells inference software, microservices and enterprise support. A strong open model gives customers a reason to adopt that software stack without committing to any single closed model vendor. AI Weekly described NVIDIA, AWS and Google Cloud as building an open agentic model stack together, with Nemotron 3 as an open standard for agent workloads.

For buyers, the motive does not make the model worse. It does explain the design choices. Nemotron 3 Ultra optimises for throughput on NVIDIA hardware and for enterprise adoption, rather than for topping public leaderboards. Judge it by those goals.

Customising Nemotron 3 Ultra With Open Recipes

The published training data and recipes change what customisation looks like. With most open models you get weights and a model card. With the Nemotron 3 family you also get the post-training datasets, which means you can see how the model learned to follow instructions and call tools, and build on the same foundations.

In practice, most teams will not retrain Nemotron 3 Ultra from scratch. At 550 billion parameters, even fine-tuning is a substantial job. The more realistic route is to fine-tune a smaller family member, such as Nano or Super, using NVIDIA’s recipes and your own domain data, then reserve Ultra for the hardest requests.

When you do tune the large model, adapter methods that train a small number of extra weights keep the job manageable. The quantisation-aware pre-training also means a tuned model can still be served in NVFP4 without a large quality drop, which is not guaranteed with models trained purely at high precision.

Keep the provenance chain intact. Record which base checkpoint, which datasets and which settings produced each tuned variant. One of the main reasons to choose a model with open data is auditability, and that benefit disappears if your own additions are undocumented.

Evaluating Nemotron 3 Ultra Fairly

Because Nemotron 3 Ultra competes on efficiency rather than raw score, a fair evaluation measures efficiency, not just accuracy.

Pick a realistic workload, ideally a few hundred real requests with typical context lengths. Run it on Nemotron 3 Ultra in NVFP4 and on one or two rivals, on the same hardware if you self-host or through comparable endpoints if you do not. Record accuracy, latency at your expected concurrency, and cost per thousand requests.

Concurrency is the number most comparisons miss. A model that is slightly less accurate but serves three times as many users per GPU may be the better business choice, especially for customer-facing agents. Throughput claims only matter if they hold at your real load.

Test long contexts separately. The hybrid design is meant to keep memory in check on very long prompts, so check whether that holds for your documents. If your workload rarely exceeds a few thousand tokens, that advantage is irrelevant and other models may suit you better.

Finally, compare reasoning on and off. Switching reasoning off for simple requests saves tokens and time. Measuring both modes tells you which requests need it.

Common Deployment Mistakes

A few errors come up repeatedly with large hybrid models, and they are cheap to avoid.

The first is running the full-precision checkpoint by default. Teams download BF16 because it sounds safer, then discover they need twice the hardware. For this model family, the 4-bit build is the intended serving format, and the published table shows it performing almost identically.

The second is testing on old hardware and drawing conclusions about new hardware, or the reverse. Throughput depends heavily on whether the GPU supports the 4-bit format natively. Benchmark on the generation you will actually deploy.

The third is leaving reasoning on for everything. Many requests do not need it, and turning it off for simple tasks saves both tokens and latency. Route by task type rather than using one setting for all traffic.

The fourth is trusting a provider’s defaults blindly. Hosted endpoints may cap output length or context well below the model’s limits, as at least one listing does. Check the actual limits before you build a feature that depends on long outputs.

Who Should Use What

If you are Start with Why Link
Running Blackwell GPUs Nemotron 3 Ultra NVFP4 Native 4-bit, high throughput See the model
Needing auditable training data Nemotron 3 family Open data and recipes Read more
Wanting the cheapest capable API DeepSeek V4.1 Flash Higher index, lower price See specs
On smaller hardware Nemotron 3 Nano or Super Same design, smaller footprint See the family

Related on this site: for more context, see Vera Rubin Is Shipping, Nvidia Earnings and Muse Spark Open.

Frequently Asked Questions About Nemotron 3 Ultra

What is Nemotron 3 Ultra?
NVIDIA’s largest open-weight model: a 550B mixture-of-experts hybrid of Mamba-2, MoE and attention layers, with 55B active parameters and a one-million-token context.

When was it released?
June 4, 2026, after a pre-announcement at Computex. It is available on Hugging Face, ModelScope, OpenRouter and build.nvidia.com.

What licence does it use?
OpenMDW-1.1, which grants royalty-free commercial use and excludes model outputs from the licence.

What is NVFP4?
NVIDIA’s 4-bit number format, which Blackwell GPUs run natively. Nemotron 3 Ultra was pre-trained with an NVFP4 recipe, so its 4-bit build loses very little quality.

How does it score?
NVIDIA reports 56.4 on Terminal Bench 2.1 in BF16 and SWE-Bench Verified between about 65% and 70.4% depending on scaffold. Newer open models score higher on coding.

What hardware does it need?
A multi-GPU server. The NVFP4 weights are roughly 275 GB by simple arithmetic, and Dell lists H200 and B300 platforms as compatible.

Verdict

If your infrastructure is NVIDIA and your priority is serving an open agentic model efficiently, Nemotron 3 Ultra is the model to test first. Its native 4-bit build, high throughput and open training data are a combination no other frontier-scale open model offers.

The caveat is capability. Newer open models from Meta, Moonshot and DeepSeek beat it on coding benchmarks, and most performance figures are NVIDIA’s own. Throughput claims depend heavily on hardware generation.

Next step: try the free OpenRouter or build.nvidia.com endpoint on your real agent tasks, and only plan a self-hosted deployment if the efficiency holds at your expected concurrency.

Sources and Further Reading

On this site: compare serving stacks in LLM inference servers compared, see smaller open options in best local LLMs, ground answers with retrieval-augmented generation, and weigh vendors in AI models compared.

2 Comments

Comments are closed.