Includes 694 hourly rounds up to 2026-11-16 from NIMStats by Mauro Druwel (source), measured with a different prompt and timeout. Our own probe adds a round every 30 minutes.
NVIDIA NIM API status and speed
Which free build.nvidia.com models answer right now, and how fast. 325 probe rounds in the last 14 days · last (9 min ago)
Model leaderboard
Ranked by composite score. Click a model to explore it, a column to sort.
| Status | Reply time trend | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | nvidia nemotron-3.5-bolt-e4b Availability steady | 95 | Serving | 100% | 114 ms | 1.63 s | 1.46 s | 86.3 tok/s | 1 | |
| 2 | nvidia nemotron-3.5-super-vl-120b-a12b Availability steady | 93 | Serving | 100% | 343 ms | 2.00 s | 957 ms | 126 tok/s | 2 | |
| 3 | poolside laguna-xs-2.1 Availability steady | 91 | Serving | 100% | 175 ms | 4.87 s | 1.75 s | 39.8 tok/s | 0 | |
| 4 | nvidia nemotron-3-nano-omni-30b-a3b-reasoning Availability steady | 89 | Serving | 99.1% | 717 ms | 5.11 s | 1.70 s | 95.8 tok/s | 248 | |
| 5 | nvidia nemotron-3-ultra-550b-a55b Availability steady | 80 | Serving | 92.6% | 1.72 s | 9.59 s | 1.79 s | 49.9 tok/s | 27 | |
| 6 | stepfun step-5-preview Availability steady | 63 | Unstable | 79.4% | 13.0 s | 19.4 s | 1.43 s | 21.3 tok/s | 26 | |
| 7 | meta muse-glimmer-30b Availability steady | 62 | Serving | 86.7% | 26.4 s | 52.4 s | 20.6 s | 8.2 tok/s | 0 | |
| 8 | meta llama-3.2-90b-vision-instruct Availability falling | 61 | Unstable | 83.1% | 9.18 s | 79.6 s | 31.9 s | 5.0 tok/s | 0 | |
| 9 | minimax minimax-m3.1-flash Availability steady | 58 | Unstable | 71.6% | 16.6 s | 22.6 s | 6.18 s | 23.0 tok/s | 0 | |
| 10 | openai gpt-oss-20b Availability steady | 51 | Unstable | 77.2% | 142.6 s | 147.8 s | 6.98 s | 3.4 tok/s | 2 | |
| 11 | mistral mistral-large-4-1t-a52b Availability improving | 51 | Unstable | 66.3% | 23.3 s | 38.1 s | 4.85 s | 11.9 tok/s | 0 | |
| 12 | google gemma-4-31b-it Availability falling | 48 | Unstable | 69.5% | 58.7 s | 76.2 s | 10.2 s | 6.6 tok/s | 0 | |
| 13 | z glm-5.3-flash Availability steady | 47 | Unstable | 76.0% | 167.8 s | 174.9 s | 3.62 s | 1.5 tok/s | 5 | |
| 14 | nvidia nemotron-3.5-lightning-30b-a3b Availability improving | 39 | Unstable | 6.5% | 200 ms | 1.58 s | 758 ms | 154 tok/s | 14 | |
| 15 | deepseek deepseek-v4.1-flash Availability steady | 28 | Unstable | 42.5% | 199.4 s | 209.5 s | 8.90 s | 1.8 tok/s | 0 |
Models answering per round
Of 15 tracked models, last 14 days.
Median first token per round
Across the models that answered. Lower is better.
Top throughput
Median tokens per second, last 14 days.
Quickest to first token
Median wait before the reply starts, last 14 days.
Listed but not served
The /v1/models listing includes 41 more chat models that
returned 404 on every probe in the last 14 days. Requests to them fail, even though
they still appear in the catalog, so they are left out of the leaderboard.
Show the 41 models
01-ai/yi-largeai21labs/jamba-1.5-large-instructaisingapore/sea-lion-7b-instructbigcode/starcoder2-15bdatabricks/dbrx-instructdeepseek-ai/deepseek-coder-6.7b-instructgoogle/codegemma-1.1-7bgoogle/codegemma-7bgoogle/gemma-2bgoogle/gemma-3-12b-itgoogle/gemma-3-4b-itgoogle/recurrentgemma-2bibm/granite-3.0-3b-a800m-instructibm/granite-3.0-8b-instructibm/granite-34b-code-instructibm/granite-8b-code-instructmeta/codellama-70bmeta/llama2-70bmicrosoft/phi-3-vision-128k-instructmicrosoft/phi-3.5-moe-instructmistralai/codestral-22b-instruct-v0.1mistralai/mistral-7b-instruct-v0.3mistralai/mistral-largemistralai/mistral-large-2-instructmistralai/mixtral-8x22b-v0.1moonshotai/kimi-k2.6nv-mistralai/mistral-nemo-12b-instructnvidia/cosmos-reason2-8bnvidia/llama-3.1-nemotron-51b-instructnvidia/llama-3.1-nemotron-70b-instructnvidia/llama-3.1-nemotron-ultra-253b-v1nvidia/llama3-chatqa-1.5-70bnvidia/mistral-nemo-minitron-8b-8k-instructnvidia/nemotron-3-super-120b-a12bnvidia/nemotron-4-340b-instructnvidia/nemotron-nano-3-30b-a3bwriter/palmyra-creative-122bwriter/palmyra-fin-70b-32kwriter/palmyra-med-70bwriter/palmyra-med-70b-32kzyphra/zamba2-7b-instruct
How we measure
Every 30 minutes a round lists the chat models on integrate.api.nvidia.com/v1/models and sends each one the same streamed
request at temperature 0, capped at 200 output tokens:
Explain how a rainbow forms, in about 120 words. We record time to first token, full reply time and completion tokens. A request that has not finished after 240 seconds counts as no answer. A win is the shortest full reply in a round.
Score (0 to 100) is 60% uptime, 20% time to first token and 20% throughput. The two speed parts are on log scales: 100 ms to first token or 300 tok/s scores full marks, 60 s or 1 tok/s scores none.
Earlier history is imported from NIMStats (hourly since 17 Oct 2026), which asks for a short Python function, allows 300 seconds, counts a reply that is only reasoning as a failure, and times the first token as the first answer token after any reasoning; its throughput is taken over the whole request. Those rounds are blended into every number on this page, so a model that mostly emits reasoning can show lower uptime until our own rounds outweigh them.
One probe location and one account, so absolute numbers include our network path. Compare models to each other, not to numbers measured elsewhere.
FAQ
- Is the NVIDIA NIM API free?
- Yes. Every model on build.nvidia.com can be called for free with an API key from an NVIDIA developer account. The free tier is rate limited (NVIDIA publishes up to 40 requests per minute) and has no per-token billing. It is meant for prototyping, not production traffic.
- Which NVIDIA NIM models actually work?
- At the last probe 9 of the 15 models we track answered. 41 more ids appear in the endpoint's model listing but return 404 on every request. The leaderboard on this page is rebuilt from fresh probes every 30 minutes.
- What is the fastest model on NVIDIA NIM?
- Over the last 14 days, nemotron-3.5-lightning-30b-a3b generated fastest at a median 154 tok/s. The shortest full reply came from nemotron-3.5-lightning-30b-a3b, with a median of 1.58 s.
- Why is NVIDIA NIM slow or timing out?
- The free endpoint runs on shared capacity, so queueing time varies by model and hour. Large models can wait a minute or more before the first token, and some time out entirely during busy periods. Each model page shows the hour-by-hour history, so you can tell a bad hour from a model that is down.
- How do I call the NVIDIA NIM API?
- The API is OpenAI-compatible at https://integrate.api.nvidia.com/v1. Send a chat completion with your nvapi- key as a Bearer token and the model id (for example nvidia/nemotron-3.5-lightning-30b-a3b). Any OpenAI SDK works by changing the base URL.