Model guide · Dracon Bench
step-5-preview
Tested
Step 5 Preview on the NVIDIA NIM free endpoint: IFEval-50 70%, IFBench 20%, MMLU-Pro 4/10, GSM8K 70%, JSON 3/5, giving 52.0 overall across five suites, rank 18. All five SVG briefs timed out, so this row has no SVG column. This is a different lane from the credit-blocked OpenCode Go preview id of the same name. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.
- Rank
- #18
- Overall
- 52.0
- Suites
- 5 of 6
- Ranked
- #18 of 21
When the world heard about it
Timeline
- First reachable for us 2026-11-18 our first probe
- First tested by us 2026-11-19 Dracon Bench
- Officially announced 2026-09-16 models.dev
- First official benchmark not recorded no source lists it
- Reachable before we scored it 1 day
- Announced before it reached the lane we served 63 days
Track A · script-scored subsets
Scores
| Suite | Score | Sample | Precision |
|---|---|---|---|
| IFEval-50 prompt-strict | 70.0% | 7/10 prompts | 39.7%–89.2% |
| IFEval-50 instruction-strict | 64.7% | 11/17 constraints | 41.3%–82.7% |
| IFBench-50 prompt-strict | 20.0% | 2/10 prompts | 5.7%–51.0% |
| IFBench-50 instruction-strict | 18.2% | 2/11 constraints | 5.1%–47.7% |
| MMLU-Pro-50 | 40.0% | 4/10 served | 16.8%–68.7% |
| JSON briefs conformance | 60.0% | 3/5 served | 23.1%–88.2% |
| GSM8K-50 | 70.0% | 7/10 served | 39.7%–89.2% |
The range beside a script-scored figure is a 95% interval over that figure's own sample: the prompts it followed, the constraints it met, or the items it answered. The instruction-strict figures are measured over CONSTRAINTS, which is why their sample line counts constraints rather than prompts. The judged total is not a proportion, so it has no interval: it carries the measured judge band instead: how far a re-judging of the same artifacts moved them, which is a different kind of number. The method page says what each one covers and what it does not.
Track A · per-type breakdown
What it failed
The instruction types these two suites check, and the checks the model missed. A type is checked once per prompt that carries it, so a type with seven checks was asked for seven times. The counts beside a type are that type's own; the suite's totals are in the scores table above. These are the failures behind the instruction-strict figure, which is the sum of them.
IFEval-50
Missed at least one check in 5 of the 15 instruction types checked.
| Instruction type | Checks | Missed |
|---|---|---|
| startend · quotation | 2 | 2 |
| change case · capital word frequency | 1 | 1 |
| detectable format · json format | 1 | 1 |
| length constraints · number words | 1 | 1 |
| punctuation · no comma | 2 | 1 |
IFBench-50
Missed at least one check in 7 of the 8 verifiers checked.
| Verifier | Checks | Missed |
|---|---|---|
| count · unique word count | 3 | 3 |
| count · conjunctions | 1 | 1 |
| count · numbers | 2 | 1 |
| count · person names | 1 | 1 |
| count · word count range | 1 | 1 |
| format · list | 1 | 1 |
| words · keywords specific position | 1 | 1 |
Provider-reported usage
Thinking cost
What the run cost the model in thinking: provider-reported completion tokens per prompt, the thinking share, and the cost per correct answer. Reasoning-heavy models can exhaust fixed budgets before answering.
| Suite | Cost | Reading |
|---|---|---|
| IFEval-50 | 1,093/prompt | 71% thinking· 1,093/correct |
| GSM8K-50 | 197/prompt | 69% thinking· 281/correct |
Head to head
Compare with
- ernie-5.5-flash-preview
- north-mini-code
- dots-3-note-preview
- gemma-4-31b-it
- lfm-2.5-2.6b
- lfm-3-1.7b
- muse-glimmer-30b
- nemotron-3.5-bolt-e4b
- nemotron-3.5-lightning-30b-a3b
- nemotron-3.5-lightning
- nemotron-3.5-super-vl-120b-a12b
- nemotron-3.5-super-vl
- nemotron-3-nano-omni-30b-a3b-reasoning
- nemotron-3-nano-omni-30b-a3b-reasoning
- nemotron-3-ultra-550b-a55b
- nemotron-3-ultra-550b-a55b
- hy4-flash
- particle-1
- laguna-xs-2.1
- flint-v1.0
Run record
Evidence and notes
- NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1); two clean minimal probes before the pilot
- No SVG score: all five briefs timed out, so the overall covers five suites
- IFBench 2/10 and JSON 3/5 are the weak points; GSM8K 7/10 is the strong one