Skip to main content

← Dracon Bench leaderboard

Model guide · Dracon Bench

mistral-large-4-1t-a52b

Tested First seen 2026-11-18 (first probed by us) · First tested 2026-11-18 · NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1)

Mistral Large 4 on NIM sits at 46.0 overall across five suites, rank 21. IFEval-50 80%, IFBench 0%, MMLU-Pro 1/10, GSM8K 60%, JSON 4/5; no SVG brief produced an admissible artifact, so this row has no SVG column and no sixth suite. Several IFEval prompts needed retries after timeouts, but the final not_delivered count is 0. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.

Rank
#21
Overall
46.0
Suites
5 of 6
Ranked
#21 of 21

When the world heard about it

Timeline

  1. First reachable for us 2026-11-18 our first probe
  2. First tested by us 2026-11-18 Dracon Bench
  3. Officially announced not recorded no source lists it
  4. First official benchmark not recorded no source lists it
  • Reachable before we scored it same day

Track A · script-scored subsets

Scores

Subset scores for mistral-large-4-1t-a52b.
SuiteScoreSamplePrecision
IFEval-50 prompt-strict80.0%8/10 prompts49.0%–94.3%
IFEval-50 instruction-strict76.5%13/17 constraints52.7%–90.4%
IFBench-50 prompt-strict0.0%0/10 prompts0.0%–27.8%
IFBench-50 instruction-strict0.0%0/11 constraints0.0%–25.9%
MMLU-Pro-5010.0%1/10 served1.8%–40.4%
JSON briefs conformance80.0%4/5 served37.6%–96.4%
GSM8K-5060.0%6/10 served31.3%–83.2%

The range beside a script-scored figure is a 95% interval over that figure's own sample: the prompts it followed, the constraints it met, or the items it answered. The instruction-strict figures are measured over CONSTRAINTS, which is why their sample line counts constraints rather than prompts. The judged total is not a proportion, so it has no interval: it carries the measured judge band instead: how far a re-judging of the same artifacts moved them, which is a different kind of number. The method page says what each one covers and what it does not.

Track A · per-type breakdown

What it failed

The instruction types these two suites check, and the checks the model missed. A type is checked once per prompt that carries it, so a type with seven checks was asked for seven times. The counts beside a type are that type's own; the suite's totals are in the scores table above. These are the failures behind the instruction-strict figure, which is the sum of them.

IFEval-50

Missed at least one check in 3 of the 15 instruction types checked.

The instruction types 3 of 15 the IFEval-50 suite checked that mistralai/mistral-large-4-1t-a52b missed, most missed first.
Instruction typeChecksMissed
startend · quotation22
detectable format · json format11
length constraints · number words11

IFBench-50

Missed at least one check in 8 of the 8 verifiers checked.

The verifiers 8 of 8 the IFBench-50 suite checked that mistralai/mistral-large-4-1t-a52b missed, most missed first.
VerifierChecksMissed
count · unique word count33
count · numbers22
count · conjunctions11
count · person names11
count · word count range11
format · list11
words · keywords specific position11
words · words position11

Provider-reported usage

Thinking cost

What the run cost the model in thinking: provider-reported completion tokens per prompt, the thinking share, and the cost per correct answer. Reasoning-heavy models can exhaust fixed budgets before answering.

Thinking cost for mistral-large-4-1t-a52b.
SuiteCostReading
IFEval-501,901/prompt82% thinking· 1,901/correct
GSM8K-50375/prompt71% thinking· 563/correct

Head to head

Compare with

Run record

Evidence and notes

  • NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1), same key and base as every NIM row
  • IFBench 0/10, MMLU-Pro 1/10, GSM8K 60%
  • No SVG score: every brief failed to produce an admissible artifact, so the overall covers five suites