Skip to main content

← Dracon Bench leaderboard

Model guide · Dracon Bench

nemotron-3.5-super-vl-120b-a12b

Tested First seen 2026-11-18 (first probed by us) · First tested 2026-11-18 · NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1)

The NIM serving of Nemotron 3.5 Super VL scores 64.3 overall across six suites, rank 9: IFEval-50 80%, IFBench 70%, JSON 5/5, GSM8K 60%, MMLU-Pro 2/10, SVG 84/150 from three admissible artifacts. It is a separate row from the OpenRouter free lane of the same family. Disclosure: its SVG total is self-judged because Super VL is the pinned judge, so treat it as a sanity check, not an independent verdict. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.

Rank
#9
Overall
64.3
Suites
6 of 6
Ranked
#9 of 21

When the world heard about it

Timeline

  1. First reachable for us 2026-11-18 our first probe
  2. First tested by us 2026-11-18 Dracon Bench
  3. Officially announced 2026-10-15 llm-stats
  4. First official benchmark 2026-11-19 llm-stats: a public score existed by then — the first day we saw one
  • Reachable before we scored it same day
  • Announced before it reached the lane we served 34 days
  • A public score existed within 1 day

Track A · script-scored subsets

Scores

Subset scores for nemotron-3.5-super-vl-120b-a12b.
SuiteScoreSamplePrecision
IFEval-50 prompt-strict80.0%8/10 prompts49.0%–94.3%
IFEval-50 instruction-strict70.6%12/17 constraints46.9%–86.7%
IFBench-50 prompt-strict70.0%7/10 prompts39.7%–89.2%
IFBench-50 instruction-strict72.7%8/11 constraints43.4%–90.3%
MMLU-Pro-5020.0%2/10 served5.7%–51.0%
JSON briefs conformance100.0%5/5 served56.6%–100.0%
GSM8K-5060.0%6/10 served31.3%–83.2%
SVG blind rubric84/150blind rubricjudge band ±8.3 pts

The range beside a script-scored figure is a 95% interval over that figure's own sample: the prompts it followed, the constraints it met, or the items it answered. The instruction-strict figures are measured over CONSTRAINTS, which is why their sample line counts constraints rather than prompts. The judged total is not a proportion, so it has no interval: it carries the measured judge band instead: how far a re-judging of the same artifacts moved them, which is a different kind of number. The method page says what each one covers and what it does not.

Track A · per-type breakdown

What it failed

The instruction types these two suites check, and the checks the model missed. A type is checked once per prompt that carries it, so a type with seven checks was asked for seven times. The counts beside a type are that type's own; the suite's totals are in the scores table above. These are the failures behind the instruction-strict figure, which is the sum of them.

IFEval-50

Missed at least one check in 4 of the 15 instruction types checked.

The instruction types 4 of 15 the IFEval-50 suite checked that nvidia/nemotron-3.5-super-vl-120b-a12b missed, most missed first.
Instruction typeChecksMissed
startend · quotation22
combination · two responses11
length constraints · number words11
punctuation · no comma21

IFBench-50

Missed at least one check in 2 of the 8 verifiers checked.

The verifiers 2 of 8 the IFBench-50 suite checked that nvidia/nemotron-3.5-super-vl-120b-a12b missed, most missed first.
VerifierChecksMissed
count · unique word count32
words · keywords specific position11

Provider-reported usage

Thinking cost

What the run cost the model in thinking: provider-reported completion tokens per prompt, the thinking share, and the cost per correct answer. Reasoning-heavy models can exhaust fixed budgets before answering.

Thinking cost for nemotron-3.5-super-vl-120b-a12b.
SuiteCostReading
IFEval-501,614/prompt61% thinking· 1,614/correct
GSM8K-50472/prompt73% thinking· 787/correct

Track B · blind-judged

Artifacts

Blind-judged by nvidia/nemotron-3.5-super-vl:free: the judge saw the brief and the rendered image only, never the model name. Briefs are frozen; expand one to read it.

beach

28/50
Judge reasoning
Brief fidelity (4/10)
Most requested elements appear, but the two dolphins arcing out of the waves are missing or unrecognizable, a headline element that caps this criterion at 4.
Construction (6/10)
Shapes are simple silhouettes and geometric forms with no melted or duplicated parts, though the water shapes are ambiguous and lack clear anatomy.
Composition (7/10)
The frame is well-used with the palm tree left, sun center, sailboat right, and a clear horizon dividing sky, sea, and sand.
Detail finish (5/10)
Includes a starfish, birds, and a shimmering reflection path, but overall detail and shading are minimal with no complex textures.
Visual appeal (6/10)
The warm-to-cool gradient palette is coherent and pleasing, with confident silhouette linework, though the scene is quite sparse.
Brief

Draw a beautiful sunset beach scene: a large sun sitting low over the sea with a shimmering reflection path on the water, a leaning palm tree silhouette on the left, two dolphins arcing out of the waves, a small sailboat on the right, birds in the sky, sandy foreground with a starfish. Warm oranges and pinks fading to teal. Coherent perspective: distant objects smaller and hazier than near ones. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

clock

unjudged

no svg

Brief

Draw a round wall clock face showing exactly 10:10: a circular dial with all 12 hour numerals in the correct positions, minute tick marks, an hour hand pointing just past 10, a minute hand pointing exactly at 2, and a thin second hand pointing at 6. The word DRACON in small capitals under the center. Dark rim, light face, flat vector style, centered composition. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

duo

unjudged

no svg

Brief

Draw a comical scene: a dog driving a jet ski at speed, paws on the handlebars, with a raccoon riding behind holding on, both leaning into a turn. Sea spray behind the jet ski, sun and clouds in the sky, a distant shoreline. Both characters must be clearly doing something, not just posed side by side. Bright cartoon vector style. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

monalisa

26/50
Judge reasoning
Brief fidelity (5/10)
Most requested elements are present in abstract form, dark dress, parted hair, smile, landscape, river, and right-side bridge, but the hands are barely visible and the portrait is highly stylized rather than a recognizable bust.
Construction (4/10)
Shapes are clean flat geometry rather than melted forms, but the face, neck, and hands are symbolic and not assembled anatomically, so the figure reads more as an icon than a constructed body.
Composition (6/10)
The centered bust fills the lower frame with the hazy landscape behind and the bridge placed on the right, giving a balanced, if flat and scale-distorted, use of the 800x600 frame.
Detail finish (5/10)
The markup renders cleanly with no stray shapes or cut-off edges, but detail is minimal, flat colors with only a smile line, part line, and bridge posts as small touches.
Visual appeal (6/10)
The muted green, brown, and smoky-blue palette is coherent and the linework is confident, yet the extreme abstraction limits charm and enigmatic presence.
Brief

Redraw the Mona Lisa as a flat vector portrait, bust composition: a seated woman in three-quarter view facing the viewer, dark dress with neckline detail, hands folded at the bottom edge, soft enigmatic smile, hair parted in the middle. Behind her a hazy landscape with a winding river, a bridge on the right, and blue mountains. Muted greens, browns, and smoky blues. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

pelican

30/50
Judge reasoning
Brief fidelity (7/10)
The pelican, large beak with throat pouch, long neck, folded wing, sky, and sea horizon are present, but the webbed feet are only a small indistinct rectangle rather than clearly webbed.
Construction (6/10)
Shapes are cleanly assembled without melting or duplication, though the neck is a thin line and the feet are a crude rectangle, making the anatomy very simplified.
Composition (6/10)
The bird sits left of center with a clear horizon and ground plane, but it is small in the frame leaving large empty sky.
Detail finish (5/10)
The artwork uses flat colors with a small eye dot and no shading or textures, and the markup appears clean with no cut-off edges.
Visual appeal (6/10)
The limited earthy and blue palette is coherent and the clean vector style has modest charm, though the execution is quite basic.
Brief

Draw a brown pelican standing in side profile, facing left, full body visible: large beak with throat pouch, long neck, folded wing, webbed feet on flat ground. Simple sky background with a hint of sea on the horizon. Flat vector style, clean shapes, limited palette. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

Head to head

Compare with

Run record

Evidence and notes

  • NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1); separate lane from the OpenRouter row
  • SVG 84/150 is self-judged (this model is the pinned judge)
  • MMLU-Pro 2/10 and two non-admissible SVG briefs are the weak points