Skip to main content

Includes 694 hourly rounds up to 2026-11-16 from NIMStats by Mauro Druwel (source), measured with a different prompt and timeout. Our own probe adds a round every 30 minutes.

NVIDIA NIM API status and speed

Which free build.nvidia.com models answer right now, and how fast. 325 probe rounds in the last 14 days · last (9 min ago)

9/ 15 Answering now tracked models, last probe
70.2% Avg success rate across all rounds and models
1.58 s Fastest full reply nemotron-3.5-lightning-30b-a3b
154tok/s Best throughput nemotron-3.5-lightning-30b-a3b
100% Most reliable nemotron-3.5-bolt-e4b

Model leaderboard

Ranked by composite score. Click a model to explore it, a column to sort.

StatusReply time trend
1nvidia nemotron-3.5-bolt-e4b Availability steady95Serving100% 114 ms1.63 s1.46 s86.3 tok/s1
2nvidia nemotron-3.5-super-vl-120b-a12b Availability steady93Serving100% 343 ms2.00 s957 ms126 tok/s2
3poolside laguna-xs-2.1 Availability steady91Serving100% 175 ms4.87 s1.75 s39.8 tok/s0
4nvidia nemotron-3-nano-omni-30b-a3b-reasoning Availability steady89Serving99.1% 717 ms5.11 s1.70 s95.8 tok/s248
5nvidia nemotron-3-ultra-550b-a55b Availability steady80Serving92.6% 1.72 s9.59 s1.79 s49.9 tok/s27
6stepfun step-5-preview Availability steady63Unstable79.4% 13.0 s19.4 s1.43 s21.3 tok/s26
7meta muse-glimmer-30b Availability steady62Serving86.7% 26.4 s52.4 s20.6 s8.2 tok/s0
8meta llama-3.2-90b-vision-instruct Availability falling61Unstable83.1% 9.18 s79.6 s31.9 s5.0 tok/s0
9minimax minimax-m3.1-flash Availability steady58Unstable71.6% 16.6 s22.6 s6.18 s23.0 tok/s0
10openai gpt-oss-20b Availability steady51Unstable77.2% 142.6 s147.8 s6.98 s3.4 tok/s2
11mistral mistral-large-4-1t-a52b Availability improving51Unstable66.3% 23.3 s38.1 s4.85 s11.9 tok/s0
12google gemma-4-31b-it Availability falling48Unstable69.5% 58.7 s76.2 s10.2 s6.6 tok/s0
13z glm-5.3-flash Availability steady47Unstable76.0% 167.8 s174.9 s3.62 s1.5 tok/s5
14nvidia nemotron-3.5-lightning-30b-a3b Availability improving39Unstable6.5% 200 ms1.58 s758 ms154 tok/s14
15deepseek deepseek-v4.1-flash Availability steady28Unstable42.5% 199.4 s209.5 s8.90 s1.8 tok/s0

Models answering per round

Of 15 tracked models, last 14 days.

048121611-03 17:0011-17 16:13 UTC
answered no answer

Median first token per round

Across the models that answered. Lower is better.

0 ms125.0 s250.0 s375.0 s500.0 s11-03 17:0011-17 16:13 UTC
answered no answer

Top throughput

Median tokens per second, last 14 days.

0.0 tok/s50.0 tok/s100 tok/s150 tok/s200 tok/snemotron-3.5-lightning-30b-a3b154 tok/snemotron-3.5-lightning-30b-a3b: 154 tok/snemotron-3.5-super-vl-120b-a1…126 tok/snemotron-3.5-super-vl-120b-a12b: 126 tok/snemotron-3-nano-omni-30b-a3b-…95.8 tok/snemotron-3-nano-omni-30b-a3b-reasoning: 95.8 tok/snemotron-3.5-bolt-e4b86.3 tok/snemotron-3.5-bolt-e4b: 86.3 tok/snemotron-3-ultra-550b-a55b49.9 tok/snemotron-3-ultra-550b-a55b: 49.9 tok/slaguna-xs-2.139.8 tok/slaguna-xs-2.1: 39.8 tok/sminimax-m3.1-flash23.0 tok/sminimax-m3.1-flash: 23.0 tok/sstep-5-preview21.3 tok/sstep-5-preview: 21.3 tok/smistral-large-4-1t-a52b11.9 tok/smistral-large-4-1t-a52b: 11.9 tok/smuse-glimmer-30b8.2 tok/smuse-glimmer-30b: 8.2 tok/s

Quickest to first token

Median wait before the reply starts, last 14 days.

0 ms6.25 s12.5 s18.8 s25.0 snemotron-3.5-bolt-e4b114 msnemotron-3.5-bolt-e4b: 114 mslaguna-xs-2.1175 mslaguna-xs-2.1: 175 msnemotron-3.5-lightning-30b-a3b200 msnemotron-3.5-lightning-30b-a3b: 200 msnemotron-3.5-super-vl-120b-a1…343 msnemotron-3.5-super-vl-120b-a12b: 343 msnemotron-3-nano-omni-30b-a3b-…717 msnemotron-3-nano-omni-30b-a3b-reasoning: 717 msnemotron-3-ultra-550b-a55b1.72 snemotron-3-ultra-550b-a55b: 1.72 sllama-3.2-90b-vision-instruct9.18 sllama-3.2-90b-vision-instruct: 9.18 sstep-5-preview13.0 sstep-5-preview: 13.0 sminimax-m3.1-flash16.6 sminimax-m3.1-flash: 16.6 smistral-large-4-1t-a52b23.3 smistral-large-4-1t-a52b: 23.3 s

Listed but not served

The /v1/models listing includes 41 more chat models that returned 404 on every probe in the last 14 days. Requests to them fail, even though they still appear in the catalog, so they are left out of the leaderboard.

Show the 41 models
  • 01-ai/yi-large
  • ai21labs/jamba-1.5-large-instruct
  • aisingapore/sea-lion-7b-instruct
  • bigcode/starcoder2-15b
  • databricks/dbrx-instruct
  • deepseek-ai/deepseek-coder-6.7b-instruct
  • google/codegemma-1.1-7b
  • google/codegemma-7b
  • google/gemma-2b
  • google/gemma-3-12b-it
  • google/gemma-3-4b-it
  • google/recurrentgemma-2b
  • ibm/granite-3.0-3b-a800m-instruct
  • ibm/granite-3.0-8b-instruct
  • ibm/granite-34b-code-instruct
  • ibm/granite-8b-code-instruct
  • meta/codellama-70b
  • meta/llama2-70b
  • microsoft/phi-3-vision-128k-instruct
  • microsoft/phi-3.5-moe-instruct
  • mistralai/codestral-22b-instruct-v0.1
  • mistralai/mistral-7b-instruct-v0.3
  • mistralai/mistral-large
  • mistralai/mistral-large-2-instruct
  • mistralai/mixtral-8x22b-v0.1
  • moonshotai/kimi-k2.6
  • nv-mistralai/mistral-nemo-12b-instruct
  • nvidia/cosmos-reason2-8b
  • nvidia/llama-3.1-nemotron-51b-instruct
  • nvidia/llama-3.1-nemotron-70b-instruct
  • nvidia/llama-3.1-nemotron-ultra-253b-v1
  • nvidia/llama3-chatqa-1.5-70b
  • nvidia/mistral-nemo-minitron-8b-8k-instruct
  • nvidia/nemotron-3-super-120b-a12b
  • nvidia/nemotron-4-340b-instruct
  • nvidia/nemotron-nano-3-30b-a3b
  • writer/palmyra-creative-122b
  • writer/palmyra-fin-70b-32k
  • writer/palmyra-med-70b
  • writer/palmyra-med-70b-32k
  • zyphra/zamba2-7b-instruct

How we measure

Every 30 minutes a round lists the chat models on integrate.api.nvidia.com/v1/models and sends each one the same streamed request at temperature 0, capped at 200 output tokens:

Explain how a rainbow forms, in about 120 words.

We record time to first token, full reply time and completion tokens. A request that has not finished after 240 seconds counts as no answer. A win is the shortest full reply in a round.

Score (0 to 100) is 60% uptime, 20% time to first token and 20% throughput. The two speed parts are on log scales: 100 ms to first token or 300 tok/s scores full marks, 60 s or 1 tok/s scores none.

Earlier history is imported from NIMStats (hourly since 17 Oct 2026), which asks for a short Python function, allows 300 seconds, counts a reply that is only reasoning as a failure, and times the first token as the first answer token after any reasoning; its throughput is taken over the whole request. Those rounds are blended into every number on this page, so a model that mostly emits reasoning can show lower uptime until our own rounds outweigh them.

One probe location and one account, so absolute numbers include our network path. Compare models to each other, not to numbers measured elsewhere.

FAQ

Is the NVIDIA NIM API free?
Yes. Every model on build.nvidia.com can be called for free with an API key from an NVIDIA developer account. The free tier is rate limited (NVIDIA publishes up to 40 requests per minute) and has no per-token billing. It is meant for prototyping, not production traffic.
Which NVIDIA NIM models actually work?
At the last probe 9 of the 15 models we track answered. 41 more ids appear in the endpoint's model listing but return 404 on every request. The leaderboard on this page is rebuilt from fresh probes every 30 minutes.
What is the fastest model on NVIDIA NIM?
Over the last 14 days, nemotron-3.5-lightning-30b-a3b generated fastest at a median 154 tok/s. The shortest full reply came from nemotron-3.5-lightning-30b-a3b, with a median of 1.58 s.
Why is NVIDIA NIM slow or timing out?
The free endpoint runs on shared capacity, so queueing time varies by model and hour. Large models can wait a minute or more before the first token, and some time out entirely during busy periods. Each model page shows the hour-by-hour history, so you can tell a bad hour from a model that is down.
How do I call the NVIDIA NIM API?
The API is OpenAI-compatible at https://integrate.api.nvidia.com/v1. Send a chat completion with your nvapi- key as a Bearer token and the model id (for example nvidia/nemotron-3.5-lightning-30b-a3b). Any OpenAI SDK works by changing the base URL.