Skip to main content

← Dracon Bench leaderboard

Model guide · Dracon Bench

step-5-preview

Tested First seen 2026-11-18 (first probed by us) · First tested 2026-11-19 · NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1)

Step 5 Preview on the NVIDIA NIM free endpoint: IFEval-50 70%, IFBench 20%, MMLU-Pro 4/10, GSM8K 70%, JSON 3/5, giving 52.0 overall across five suites, rank 18. All five SVG briefs timed out, so this row has no SVG column. This is a different lane from the credit-blocked OpenCode Go preview id of the same name. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.

Rank
#18
Overall
52.0
Suites
5 of 6
Ranked
#18 of 21

When the world heard about it

Timeline

  1. First reachable for us 2026-11-18 our first probe
  2. First tested by us 2026-11-19 Dracon Bench
  3. Officially announced 2026-09-16 models.dev
  4. First official benchmark not recorded no source lists it
  • Reachable before we scored it 1 day
  • Announced before it reached the lane we served 63 days

Track A · script-scored subsets

Scores

Subset scores for step-5-preview.
SuiteScoreSamplePrecision
IFEval-50 prompt-strict70.0%7/10 prompts39.7%–89.2%
IFEval-50 instruction-strict64.7%11/17 constraints41.3%–82.7%
IFBench-50 prompt-strict20.0%2/10 prompts5.7%–51.0%
IFBench-50 instruction-strict18.2%2/11 constraints5.1%–47.7%
MMLU-Pro-5040.0%4/10 served16.8%–68.7%
JSON briefs conformance60.0%3/5 served23.1%–88.2%
GSM8K-5070.0%7/10 served39.7%–89.2%

The range beside a script-scored figure is a 95% interval over that figure's own sample: the prompts it followed, the constraints it met, or the items it answered. The instruction-strict figures are measured over CONSTRAINTS, which is why their sample line counts constraints rather than prompts. The judged total is not a proportion, so it has no interval: it carries the measured judge band instead: how far a re-judging of the same artifacts moved them, which is a different kind of number. The method page says what each one covers and what it does not.

Track A · per-type breakdown

What it failed

The instruction types these two suites check, and the checks the model missed. A type is checked once per prompt that carries it, so a type with seven checks was asked for seven times. The counts beside a type are that type's own; the suite's totals are in the scores table above. These are the failures behind the instruction-strict figure, which is the sum of them.

IFEval-50

Missed at least one check in 5 of the 15 instruction types checked.

The instruction types 5 of 15 the IFEval-50 suite checked that stepfun-ai/step-5-preview missed, most missed first.
Instruction typeChecksMissed
startend · quotation22
change case · capital word frequency11
detectable format · json format11
length constraints · number words11
punctuation · no comma21

IFBench-50

Missed at least one check in 7 of the 8 verifiers checked.

The verifiers 7 of 8 the IFBench-50 suite checked that stepfun-ai/step-5-preview missed, most missed first.
VerifierChecksMissed
count · unique word count33
count · conjunctions11
count · numbers21
count · person names11
count · word count range11
format · list11
words · keywords specific position11

Provider-reported usage

Thinking cost

What the run cost the model in thinking: provider-reported completion tokens per prompt, the thinking share, and the cost per correct answer. Reasoning-heavy models can exhaust fixed budgets before answering.

Thinking cost for step-5-preview.
SuiteCostReading
IFEval-501,093/prompt71% thinking· 1,093/correct
GSM8K-50197/prompt69% thinking· 281/correct

Head to head

Compare with

Run record

Evidence and notes

  • NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1); two clean minimal probes before the pilot
  • No SVG score: all five briefs timed out, so the overall covers five suites
  • IFBench 2/10 and JSON 3/5 are the weak points; GSM8K 7/10 is the strong one