Instructions to use nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2

SGLang

How to use nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 with Docker Model Runner:
```
docker model run hf.co/nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2
```

Model Overview

Description

NVIDIA-Nemotron-3-Super-120B-A12B-BF16-MTPv2 is an updated, separately distributed Multi-Token Prediction (MTP) head for speculative decoding with NVIDIA Nemotron 3 Super. It is not a standalone language model and does not contain the target model's backbone weights.

The checkpoint was initialized from the built-in MTP head in nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 and further trained with NVIDIA Megatron-LM, using the same Megatron training infrastructure used for supervised fine-tuning of Nemotron models. The Nemotron 3 Super backbone remained frozen during this additional MTP training.

The MTPv2 head can be used with the following Nemotron 3 Super target checkpoints:

During speculative decoding, the MTP head proposes draft tokens and the target model verifies them. The target model determines the accepted output, so adopting this checkpoint does not replace or modify the target model.

This model is ready for commercial or non-commercial use.

License/Terms of Use

Governing Download Terms: Use of this model/checkpoint is governed by the NVIDIA Nemotron Open Model License.

Deployment Geography

Global

Use Case

This checkpoint is intended for developers serving NVIDIA Nemotron 3 Super who want to improve inference performance through opt-in speculative decoding. It must be used together with a compatible Nemotron 3 Super target checkpoint and a serving runtime that supports loading a separate MTP head.

Release Date

Hugging Face 08/11/2026 via https://huggingface.co/nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2

References

Model Architecture

Architecture Type: Multi-Token Prediction draft head for speculative decoding
Network Architecture: Nemotron 3 Super MTP head, trained with Megatron-LM
Target Model Architecture: Mamba-2, Mixture-of-Experts, and Attention hybrid with 120B total parameters and 12B active parameters
Number of MTP Head Parameters: 3B
Initialization and Training: Initialized from the built-in Nemotron 3 Super MTP head and further trained while the target backbone remained frozen.

Input

Input Type(s): Text
Input Format(s): String
Input Parameters: One Dimensional (1D)
Other Properties Related to Input: Maximum context length up to 262144 tokens. Supported languages include: English, French, German, Italian, Japanese, Spanish, and Chinese

Output

Output Type(s): Text
Output Format: String
Output Parameters: One-Dimensional (1D)
Other Properties Related to Output: None

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware and software frameworks, the model can accelerate inference compared with unaccelerated autoregressive decoding when used in a supported configuration.

Software Integration

Supported Runtime Engine(s): vLLM, SGLang, TensorRT-LLM

Supported Hardware Microarchitecture Compatibility:

NVIDIA Blackwell
NVIDIA Hopper

Preferred Operating System(s):

Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Iterative testing and validation at both model and system levels are essential to mitigate risks and confirm that speculative decoding improves the intended workload.

Model Version(s)

MTPv2 is a separately distributed, updated MTP head for NVIDIA Nemotron 3 Super. It was trained with Megatron-LM.

Training and Evaluation Datasets

Training Dataset

Link: nvidia/Nemotron-Post-Training-Dataset-v2
Data Modality: Text
Text Training Data Size: Refer to the source dataset card
Data Collection Method by Dataset: Hybrid: Automated, Synthetic
Labeling Method by Dataset: Hybrid: Automated, Synthetic
Properties: The MTPv2 head was initialized from the built-in Nemotron 3 Super MTP head and further trained using Nemotron Post-Training Dataset v2. The source dataset combines prompts sourced from public and open corpora or synthetically generated prompts; prompts are filtered for quality and complexity, and responses are synthetically generated by public and open models.

Evaluation Dataset

Link: nvidia/SPEED-Bench
Data Collection Method by Dataset: Hybrid: Manually-Labeled, Synthetic
Labeling Method by Dataset: Hybrid: Manually-Labeled, Synthetic
Properties: SPEED-Bench contains 880 curated multi-turn dialogue sequences across 11 categories, with 80 samples per category. Mean acceptance length was measured on the qualitative split with draft length 7 and temperature 0.

Paired target-model quality sanity checks on GPQA, HLE, MMLU-Pro, and AIME 2025 were closely aligned between the built-in MTP head and MTPv2. Because exact speculative decoding verifies draft tokens with the target model, the target checkpoint remains responsible for the final output distribution.

Inference

Acceleration Engine: vLLM
Test Hardware: NVIDIA Hopper and Blackwell GPUs

Multi-Token Prediction Speculative Decoding

The MTPv2 head predicts multiple candidate tokens beyond the next token. During generation, the target Nemotron 3 Super model verifies these candidates and accepts the longest valid prefix. The number of accepted tokens per target-model verification step is called the acceptance length. Higher acceptance length can reduce the number of target-model decoding steps, but end-to-end speedup depends on hardware, workload, concurrency, context length, serving configuration, and draft length.

On the SPEED-Bench qualitative split with draft length 7 and temperature 0, the updated MTPv2 head improved mean acceptance length from approximately 3.45 to 4.31 accepted tokens per speculative step relative to the built-in MTP head.

Model	Coding	Humanities	Math	Multilingual	QA	RAG	Reasoning	Roleplay	STEM	Summarization	Writing	Overall
Built-in Nemotron 3 Super MTP head	3.78	3.26	3.73	4.05	3.16	3.78	3.59	2.82	3.30	3.48	2.99	3.45
Nemotron 3 Super MTPv2 head	4.86	4.02	4.71	5.31	3.91	4.80	4.49	3.35	4.11	4.29	3.60	4.31

Usage

The following example serves the BF16 target with the separate MTPv2 head in vLLM. Tested on v20.22.0 vLLM version.

vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --speculative-config '{
    "method": "mtp",
    "model": "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2",
    "num_speculative_tokens": 7
  }'

The target model's normal serving arguments, chat template, reasoning parser, tool-call parser, sampling parameters, and context-length settings continue to apply. Users should benchmark the draft length and serving configuration on their own workload.

Model Limitations

This checkpoint is not a standalone language model and cannot generate user-visible output without a compatible Nemotron 3 Super target checkpoint.
Compatibility depends on the serving runtime correctly supporting a separately distributed MTP head.
The checkpoint inherits the capabilities, limitations and safety considerations of the target Nemotron 3 Super model.
Serving or kernel defects can affect correctness or performance even though exact speculative decoding is designed to preserve the target model's output distribution.
The MTP head does not provide independent safety controls. Deployers must retain target-model safeguards, application guardrails, least-privilege access, and runtime security validation.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

SUBCARDS:

Explainability

Field	Response
Intended Task/Domain	Speculative-decoding acceleration for text generation, reasoning, and chat with NVIDIA Nemotron 3 Super
Model Type	Separately distributed Multi-Token Prediction draft head
Intended Users	Developers and researchers serving compatible NVIDIA Nemotron 3 Super checkpoints
Output	Draft token candidates that are verified by the target model
Describe how the model works	The MTP head proposes multiple future tokens from target-model representations. The target model verifies the proposed sequence and accepts only the valid prefix.
Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of	Not Applicable
Technical Limitations & Mitigation	The head must be used with a compatible target checkpoint and serving runtime. Performance is workload-dependent. Validate target-model quality and end-to-end behavior for the intended deployment.
The MTP head does not provide independent safety controls. Deployers must retain target-model safeguards, application guardrails, least-privilege access, and runtime security validation.
Verified to have met prescribed NVIDIA quality standards	Yes
Performance Metrics	Mean acceptance length on SPEED-Bench and end-to-end serving performance
Potential Known Risks	Runtime defects or an unsuitable speculative configuration can reduce performance or affect system behavior.
Licensing	NVIDIA Nemotron Open Model License

Bias

Field	Response
Participation considerations from adversely impacted groups in model design and testing	None
Measures taken to mitigate against unwanted bias	None
Bias Metric	None

Safety & Security

Field	Response
Model Application Field(s)	Speculative-decoding acceleration for chat, instruction following, code generation, reasoning, and other supported Nemotron 3 Super use cases
Describe the life-critical impact, if present	Not Applicable
Use Case Restrictions	Use must comply with the NVIDIA Nemotron Open Model License and the restrictions applicable to the target model and complete application.
Model and Dataset Restrictions	The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.

Privacy

Field	Response
Generatable or Reverse engineerable personal data?	No
Personal data used to create this model?	No
Was consent obtained for any personal data used?	Not Applicable
How often is dataset reviewed?	Before Release
Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model?	No
Is there provenance for all datasets used in training?	Yes
Does data labeling (annotation, metadata) comply with privacy laws?	Yes
Is data compliant with data subject requests for data correction or removal, if such a request was made?	Not Applicable
Applicable NVIDIA Privacy Policy	https://www.nvidia.com/en-us/about-nvidia/privacy-policy/

Downloads last month: 2,917

Safetensors

Model size

3B params

Tensor type

F32

BF16

Model tree for nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2

Base model

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16

Finetuned

(17)

this model

nvidia
/

Nemotron-3-Super-120B-A12B-BF16-MTPv2