Skip to main content

Includes 694 hourly rounds up to 2026-11-16 from NIMStats by Mauro Druwel (source), measured with a different prompt and timeout. Our own probe adds a round every 30 minutes.

NVIDIA NIM API status and speed

Which free build.nvidia.com models answer right now, and how fast. 709 probe rounds in the last 30 days · last (9 min ago)

9/ 15 Answering now tracked models, last probe
61.0% Avg success rate across all rounds and models
1.63 s Fastest full reply nemotron-3.5-bolt-e4b
126tok/s Best throughput nemotron-3.5-super-vl-120b-a12b
100% Most reliable nemotron-3.5-bolt-e4b

Model leaderboard

Ranked by composite score. Click a model to explore it, a column to sort.

StatusReply time trend
1nvidia nemotron-3.5-bolt-e4b Availability steady95Serving100% 114 ms1.63 s1.46 s86.3 tok/s1
2nvidia nemotron-3.5-super-vl-120b-a12b Availability steady93Serving100% 343 ms2.00 s957 ms126 tok/s2
3poolside laguna-xs-2.1 Availability steady91Serving100% 175 ms4.87 s1.75 s39.8 tok/s0
4nvidia nemotron-3-ultra-550b-a55b Availability steady82Serving94.9% 1.73 s10.2 s1.79 s48.5 tok/s190
5nvidia nemotron-3-nano-omni-30b-a3b-reasoning Availability steady78Serving81.1% 814 ms6.14 s1.51 s81.8 tok/s342
6meta muse-glimmer-30b Availability steady62Serving86.7% 26.4 s52.4 s20.6 s8.2 tok/s0
7openai gpt-oss-20b Availability steady61Unstable84.9% 37.1 s43.6 s5.88 s11.0 tok/s65
8stepfun step-5-preview Availability steady60Unstable77.3% 17.0 s23.7 s1.43 s18.0 tok/s62
9minimax minimax-m3.1-flash Availability steady58Unstable71.6% 16.6 s22.6 s6.18 s23.0 tok/s0
10meta llama-3.2-90b-vision-instruct Availability falling56Unstable78.0% 18.3 s89.5 s31.9 s4.4 tok/s0
11mistral mistral-large-4-1t-a52b Availability improving51Unstable66.3% 23.3 s38.1 s4.85 s11.9 tok/s0
12z glm-5.3-flash Availability steady47Unstable74.6% 122.4 s129.8 s3.05 s2.0 tok/s19
13google gemma-4-31b-it Availability steady43Unstable62.8% 98.6 s119.8 s10.2 s4.2 tok/s0
14nvidia nemotron-3.5-lightning-30b-a3b Availability steady35Unstable4.2% 305 ms2.86 s758 ms104 tok/s19
15nvidia nemotron-3-super-120b-a12b Availability steady28Unstable5.8% 1.68 s10.8 s3.60 s44.7 tok/s9
16deepseek deepseek-v4.1-flash Availability steady27Unstable41.6% 187.0 s189.1 s8.90 s2.0 tok/s0

Models answering per round

Of 15 tracked models, last 30 days.

048121610-18 17:0011-17 16:13 UTC
answered no answer

Median first token per round

Across the models that answered. Lower is better.

0 ms125.0 s250.0 s375.0 s500.0 s10-18 17:0011-17 16:13 UTC
answered no answer

Top throughput

Median tokens per second, last 30 days.

0.0 tok/s50.0 tok/s100 tok/s150 tok/s200 tok/snemotron-3.5-super-vl-120b-a1…126 tok/snemotron-3.5-super-vl-120b-a12b: 126 tok/snemotron-3.5-lightning-30b-a3b104 tok/snemotron-3.5-lightning-30b-a3b: 104 tok/snemotron-3.5-bolt-e4b86.3 tok/snemotron-3.5-bolt-e4b: 86.3 tok/snemotron-3-nano-omni-30b-a3b-…81.8 tok/snemotron-3-nano-omni-30b-a3b-reasoning: 81.8 tok/snemotron-3-ultra-550b-a55b48.5 tok/snemotron-3-ultra-550b-a55b: 48.5 tok/snemotron-3-super-120b-a12b44.7 tok/snemotron-3-super-120b-a12b: 44.7 tok/slaguna-xs-2.139.8 tok/slaguna-xs-2.1: 39.8 tok/sminimax-m3.1-flash23.0 tok/sminimax-m3.1-flash: 23.0 tok/sstep-5-preview18.0 tok/sstep-5-preview: 18.0 tok/smistral-large-4-1t-a52b11.9 tok/smistral-large-4-1t-a52b: 11.9 tok/s

Quickest to first token

Median wait before the reply starts, last 30 days.

0 ms5.00 s10.0 s15.0 s20.0 snemotron-3.5-bolt-e4b114 msnemotron-3.5-bolt-e4b: 114 mslaguna-xs-2.1175 mslaguna-xs-2.1: 175 msnemotron-3.5-lightning-30b-a3b305 msnemotron-3.5-lightning-30b-a3b: 305 msnemotron-3.5-super-vl-120b-a1…343 msnemotron-3.5-super-vl-120b-a12b: 343 msnemotron-3-nano-omni-30b-a3b-…814 msnemotron-3-nano-omni-30b-a3b-reasoning: 814 msnemotron-3-super-120b-a12b1.68 snemotron-3-super-120b-a12b: 1.68 snemotron-3-ultra-550b-a55b1.73 snemotron-3-ultra-550b-a55b: 1.73 sminimax-m3.1-flash16.6 sminimax-m3.1-flash: 16.6 sstep-5-preview17.0 sstep-5-preview: 17.0 sllama-3.2-90b-vision-instruct18.3 sllama-3.2-90b-vision-instruct: 18.3 s

Listed but not served

The /v1/models listing includes 40 more chat models that returned 404 on every probe in the last 30 days. Requests to them fail, even though they still appear in the catalog, so they are left out of the leaderboard.

Show the 40 models
  • 01-ai/yi-large
  • ai21labs/jamba-1.5-large-instruct
  • aisingapore/sea-lion-7b-instruct
  • bigcode/starcoder2-15b
  • databricks/dbrx-instruct
  • deepseek-ai/deepseek-coder-6.7b-instruct
  • google/codegemma-1.1-7b
  • google/codegemma-7b
  • google/gemma-2b
  • google/gemma-3-12b-it
  • google/gemma-3-4b-it
  • google/recurrentgemma-2b
  • ibm/granite-3.0-3b-a800m-instruct
  • ibm/granite-3.0-8b-instruct
  • ibm/granite-34b-code-instruct
  • ibm/granite-8b-code-instruct
  • meta/codellama-70b
  • meta/llama2-70b
  • microsoft/phi-3-vision-128k-instruct
  • microsoft/phi-3.5-moe-instruct
  • mistralai/codestral-22b-instruct-v0.1
  • mistralai/mistral-7b-instruct-v0.3
  • mistralai/mistral-large
  • mistralai/mistral-large-2-instruct
  • mistralai/mixtral-8x22b-v0.1
  • moonshotai/kimi-k2.6
  • nv-mistralai/mistral-nemo-12b-instruct
  • nvidia/cosmos-reason2-8b
  • nvidia/llama-3.1-nemotron-51b-instruct
  • nvidia/llama-3.1-nemotron-70b-instruct
  • nvidia/llama-3.1-nemotron-ultra-253b-v1
  • nvidia/llama3-chatqa-1.5-70b
  • nvidia/mistral-nemo-minitron-8b-8k-instruct
  • nvidia/nemotron-4-340b-instruct
  • nvidia/nemotron-nano-3-30b-a3b
  • writer/palmyra-creative-122b
  • writer/palmyra-fin-70b-32k
  • writer/palmyra-med-70b
  • writer/palmyra-med-70b-32k
  • zyphra/zamba2-7b-instruct

How we measure

Every 30 minutes a round lists the chat models on integrate.api.nvidia.com/v1/models and sends each one the same streamed request at temperature 0, capped at 200 output tokens:

Explain how a rainbow forms, in about 120 words.

We record time to first token, full reply time and completion tokens. A request that has not finished after 240 seconds counts as no answer. A win is the shortest full reply in a round.

Score (0 to 100) is 60% uptime, 20% time to first token and 20% throughput. The two speed parts are on log scales: 100 ms to first token or 300 tok/s scores full marks, 60 s or 1 tok/s scores none.

Earlier history is imported from NIMStats (hourly since 17 Oct 2026), which asks for a short Python function, allows 300 seconds, counts a reply that is only reasoning as a failure, and times the first token as the first answer token after any reasoning; its throughput is taken over the whole request. Those rounds are blended into every number on this page, so a model that mostly emits reasoning can show lower uptime until our own rounds outweigh them.

One probe location and one account, so absolute numbers include our network path. Compare models to each other, not to numbers measured elsewhere.

FAQ

Is the NVIDIA NIM API free?
Yes. Every model on build.nvidia.com can be called for free with an API key from an NVIDIA developer account. The free tier is rate limited (NVIDIA publishes up to 40 requests per minute) and has no per-token billing. It is meant for prototyping, not production traffic.
Which NVIDIA NIM models actually work?
At the last probe 9 of the 15 models we track answered. 40 more ids appear in the endpoint's model listing but return 404 on every request. The leaderboard on this page is rebuilt from fresh probes every 30 minutes.
What is the fastest model on NVIDIA NIM?
Over the last 30 days, nemotron-3.5-super-vl-120b-a12b generated fastest at a median 126 tok/s. The shortest full reply came from nemotron-3.5-bolt-e4b, with a median of 1.63 s.
Why is NVIDIA NIM slow or timing out?
The free endpoint runs on shared capacity, so queueing time varies by model and hour. Large models can wait a minute or more before the first token, and some time out entirely during busy periods. Each model page shows the hour-by-hour history, so you can tell a bad hour from a model that is down.
How do I call the NVIDIA NIM API?
The API is OpenAI-compatible at https://integrate.api.nvidia.com/v1. Send a chat completion with your nvapi- key as a Bearer token and the model id (for example nvidia/nemotron-3.5-lightning-30b-a3b). Any OpenAI SDK works by changing the base URL.