Skip to main content

← Dracon Bench leaderboard

Model guide · Dracon Bench

nemotron-3.5-bolt-e4b

Tested First seen 2026-11-18 (first probed by us) · First tested 2026-11-18 · NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1)

Nemotron 3.5 Bolt E4B on NIM: IFEval-50 100% (n=10), IFBench 40%, MMLU-Pro 7/10, GSM8K 70%, JSON 5/5, and SVG 102/250 from five blind-judged artifacts. Overall 70.1 across six suites, rank 4; it ties for the best IFEval-50 on the board. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.

Rank
#4
Overall
70.1
Suites
6 of 6
Ranked
#4 of 21

When the world heard about it

Timeline

  1. First reachable for us 2026-11-18 our first probe
  2. First tested by us 2026-11-18 Dracon Bench
  3. Officially announced not recorded no source lists it
  4. First official benchmark not recorded no source lists it
  • Reachable before we scored it same day

Track A · script-scored subsets

Scores

Subset scores for nemotron-3.5-bolt-e4b.
SuiteScoreSamplePrecision
IFEval-50 prompt-strict100.0%10/10 prompts72.2%–100.0%
IFEval-50 instruction-strict100.0%17/17 constraints81.6%–100.0%
IFBench-50 prompt-strict40.0%4/10 prompts16.8%–68.7%
IFBench-50 instruction-strict45.5%5/11 constraints21.3%–72.0%
MMLU-Pro-5070.0%7/10 served39.7%–89.2%
JSON briefs conformance100.0%5/5 served56.6%–100.0%
GSM8K-5070.0%7/10 served39.7%–89.2%
SVG blind rubric102/250blind rubricjudge band ±8.3 pts

The range beside a script-scored figure is a 95% interval over that figure's own sample: the prompts it followed, the constraints it met, or the items it answered. The instruction-strict figures are measured over CONSTRAINTS, which is why their sample line counts constraints rather than prompts. The judged total is not a proportion, so it has no interval: it carries the measured judge band instead: how far a re-judging of the same artifacts moved them, which is a different kind of number. The method page says what each one covers and what it does not.

Track A · per-type breakdown

What it failed

The instruction types these two suites check, and the checks the model missed. A type is checked once per prompt that carries it, so a type with seven checks was asked for seven times. The counts beside a type are that type's own; the suite's totals are in the scores table above. These are the failures behind the instruction-strict figure, which is the sum of them.

IFEval-50

Missed no checks in the 15 instruction types the IFEval-50 suite checked.

IFBench-50

Missed at least one check in 5 of the 8 verifiers checked.

The verifiers 5 of 8 the IFBench-50 suite checked that nvidia/nemotron-3.5-bolt-e4b missed, most missed first.
VerifierChecksMissed
count · numbers22
count · conjunctions11
count · word count range11
words · keywords specific position11
words · words position11

Provider-reported usage

Thinking cost

What the run cost the model in thinking: provider-reported completion tokens per prompt, the thinking share, and the cost per correct answer. Reasoning-heavy models can exhaust fixed budgets before answering.

Thinking cost for nemotron-3.5-bolt-e4b.
SuiteCostReading
IFEval-50319/prompt0% thinking· 319/correct
GSM8K-50201/prompt0% thinking· 287/correct

Track B · blind-judged

Artifacts

Blind-judged by nvidia/nemotron-3.5-super-vl:free: the judge saw the brief and the rendered image only, never the model name. Briefs are frozen; expand one to read it.

beach

19/50
Judge reasoning
Brief fidelity (3/10)
The sun, sky gradient, and sandy foreground are present, but dolphins are entirely missing, the palm tree is an unrecognizable dark triangle, the sailboat is a bare triangle, and birds are mere dots, so headline elements are absent or unrecognizable.
Construction (4/10)
All objects are simple geometric primitives (circles, triangles, dots) with no anatomical errors or melted forms, but they are so abstract that assembly quality is hard to judge beyond basic shape validity.
Composition (5/10)
The horizon divides the frame and major elements are placed left/right/center, but relative sizes are distorted and atmospheric perspective is not convincingly rendered.
Detail finish (3/10)
The image uses flat fills with no shading, textures, or small finishing touches; the white curves near the sun are the only attempt at a reflection path, and edges are clean but sparse.
Visual appeal (4/10)
The warm-to-teal gradient sky is pleasant, but the overall illustration is extremely minimal and lacks charm, confidence, or visual interest beyond the background.
Brief

Draw a beautiful sunset beach scene: a large sun sitting low over the sea with a shimmering reflection path on the water, a leaning palm tree silhouette on the left, two dolphins arcing out of the waves, a small sailboat on the right, birds in the sky, sandy foreground with a starfish. Warm oranges and pinks fading to teal. Coherent perspective: distant objects smaller and hazier than near ones. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

clock

26/50
12 1 2 3 6 9 8 7 10 11 DRACON
Judge reasoning
Brief fidelity (3/10)
The hour and minute hands are correct at 10 and 2, but the second hand points to 12 instead of 6, minute tick marks are absent, and several numerals are cut off at the frame edges.
Construction (8/10)
All shapes are cleanly assembled vector geometry with no melted, duplicated, or misaligned parts.
Composition (4/10)
The clock is centered but oversized for the frame, cutting off numerals 8, 9, 2, and 3 and crowding the edges.
Detail finish (5/10)
The flat vector style and dark rim/light face palette are correct, but missing minute ticks and the wrong second hand position are finish flaws.
Visual appeal (6/10)
The limited palette and confident linework are pleasant, but the cropped numerals and incorrect second hand reduce overall charm.
Brief

Draw a round wall clock face showing exactly 10:10: a circular dial with all 12 hour numerals in the correct positions, minute tick marks, an hour hand pointing just past 10, a minute hand pointing exactly at 2, and a thin second hand pointing at 6. The word DRACON in small capitals under the center. Dark rim, light face, flat vector style, centered composition. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

duo

20/50
Judge reasoning
Brief fidelity (4/10)
The dog, raccoon, jet ski, sun, clouds, and shoreline are present, but paws on handlebars, the raccoon holding on, and clear sea spray are missing, capping the score.
Construction (3/10)
Bodies and objects are crude geometric blobs with no recognizable paws, handlebars, or assembled anatomy, appearing melted rather than constructed.
Composition (5/10)
The subjects sit small in the lower center beneath a vast empty sky with flat perspective and no clear depth or dynamic framing.
Detail finish (4/10)
The image uses flat colors with minimal shading, a few white spray dots, and an odd sky squiggle, lacking textures or refined finishing.
Visual appeal (4/10)
The bright palette is coherent but the primitive, unfinished forms give little charm or confidence, making it feel more like a rough sketch than a polished illustration.
Brief

Draw a comical scene: a dog driving a jet ski at speed, paws on the handlebars, with a raccoon riding behind holding on, both leaning into a turn. Sea spray behind the jet ski, sun and clouds in the sky, a distant shoreline. Both characters must be clearly doing something, not just posed side by side. Bright cartoon vector style. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

monalisa

16/50
Judge reasoning
Brief fidelity (3/10)
The image lacks most requested elements: it is frontal rather than three-quarter, has no visible neckline detail, no folded hands, no middle hair part, and only a token river and bridge, so it misses multiple headline elements.
Construction (2/10)
The figure is reduced to a circle face, a trapezoidal hair mass, and a blocky dress with no anatomical assembly, making body parts look like floating geometric shapes rather than a constructed portrait.
Composition (4/10)
The composition is centered and uses the full frame, but the oversized hair shape and tiny face create awkward relative sizes with flat, non-perspective space.
Detail finish (3/10)
There is virtually no shading, texture, or small detail beyond a simple smile line, and the bridge is just an unadorned rectangle, leaving the finish bare.
Visual appeal (4/10)
The muted blue-brown palette is coherent and the flat vector style is clean, but the extreme abstraction gives it little charm or recognizability as the Mona Lisa.
Brief

Redraw the Mona Lisa as a flat vector portrait, bust composition: a seated woman in three-quarter view facing the viewer, dark dress with neckline detail, hands folded at the bottom edge, soft enigmatic smile, hair parted in the middle. Behind her a hazy landscape with a winding river, a bridge on the right, and blue mountains. Muted greens, browns, and smoky blues. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

pelican

21/50
Judge reasoning
Brief fidelity (3/10)
The pelican faces right instead of the requested left, and the throat pouch and clearly webbed feet are missing or indistinct, so multiple headline elements are absent or incorrect, capping this criterion.
Construction (4/10)
The body, neck, head, beak, wing, and feet are assembled from clean geometric shapes, but the neck loop and head-to-body connection look anatomically odd rather than natural.
Composition (5/10)
The pelican sits left of center with a clear horizon dividing sky and sea, but the frame is heavily unbalanced with excessive empty sky and a disproportionately large body.
Detail finish (4/10)
The artwork uses a flat, limited palette with clean vector shapes and no visible broken markup, but it lacks shading, textures, or the small finishing touches called for in the brief.
Visual appeal (5/10)
The coherent brown-and-blue palette and confident simplicity give it a minimalist charm, though the anatomical errors and missing features keep it from being fully satisfying.
Brief

Draw a brown pelican standing in side profile, facing left, full body visible: large beak with throat pouch, long neck, folded wing, webbed feet on flat ground. Simple sky background with a hint of sea on the horizon. Flat vector style, clean shapes, limited palette. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

Head to head

Compare with

Run record

Evidence and notes

  • NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1); probes and pilot used the same base
  • Rank #4 overall (0.701) across six suites; 100% IFEval-50 (n=10)
  • SVG 102/250 from five blind-judged artifacts