TestedFirst seen 2026-11-18 (first probed by us) · First tested 2026-11-18 · NVIDIA NIM free endpoint (integrate.api.nvidia.com/v1)
Muse Glimmer 30B on NIM: IFEval-50 60%, IFBench 20%, MMLU-Pro 1/10, GSM8K 1/10, JSON 5/5, and SVG 41/50 from a single admissible brief. Overall 47.0 across six suites, rank 20. Disclosure: four of five SVG briefs produced no artifact, so its SVG denominator is 1/5 rather than 5/5; reasoning traces routinely hit the token ceiling. Pilot depth: 10 prompts per text suite and the 5 SVG briefs; every suite prints its own n and not_delivered.
Rank
#20
Overall
47.0
Suites
6 of 6
Ranked
#20 of 21
When the world heard about it
Timeline
First reachable for us2026-11-18our first probe
First tested by us2026-11-18Dracon Bench
Officially announced2026-08-10llm-stats
First official benchmark2026-11-19the provider's benchmarks block: a public score existed by then — the first day we saw one
Reachable before we scored itsame day
Announced before it reached the lane we served100 days
A public score existed within1 day
Track A · script-scored subsets
Scores
Subset scores for muse-glimmer-30b.
Suite
Score
Sample
Precision
IFEval-50 prompt-strict
60.0%
6/10 prompts
31.3%–83.2%
IFEval-50 instruction-strict
47.1%
8/17 constraints
26.2%–69.0%
IFBench-50 prompt-strict
20.0%
2/10 prompts
5.7%–51.0%
IFBench-50 instruction-strict
18.2%
2/11 constraints
5.1%–47.7%
MMLU-Pro-50
10.0%
1/10 served
1.8%–40.4%
JSON briefs conformance
100.0%
5/5 served
56.6%–100.0%
GSM8K-50
10.0%
1/10 served
1.8%–40.4%
SVG blind rubric
41/50
blind rubric
judge band ±8.3 pts
The range beside a script-scored figure is a 95% interval over that
figure's own sample: the prompts it followed, the constraints it
met, or the items it answered. The instruction-strict figures are
measured over CONSTRAINTS, which is why their sample line counts
constraints rather than prompts. The judged total is not a
proportion, so it has no interval: it carries the measured judge
band instead: how far a re-judging of the same artifacts moved
them, which is a different kind of number. The method page says
what each one covers and what it does not.
Track A · per-type breakdown
What it failed
The instruction types these two suites check, and the checks the model
missed. A type is checked once per prompt that carries it, so a type
with seven checks was asked for seven times. The counts beside a type
are that type's own; the suite's totals are in the scores table above.
These are the failures behind the instruction-strict figure, which is
the sum of them.
IFEval-50
Missed at least one check in 8 of the 15
instruction types checked.
The instruction types 8 of 15 the
IFEval-50 suite checked that meta/muse-glimmer-30b missed, most missed first.
Instruction type
Checks
Missed
startend · quotation
2
2
change case · capital word frequency
1
1
change case · english capital
1
1
detectable content · number placeholders
1
1
detectable content · postscript
1
1
keywords · frequency
1
1
length constraints · number words
1
1
punctuation · no comma
2
1
IFBench-50
Missed at least one check in 7 of the 8
verifiers checked.
The verifiers 7 of 8 the
IFBench-50 suite checked that meta/muse-glimmer-30b missed, most missed first.
Verifier
Checks
Missed
count · numbers
2
2
count · unique word count
3
2
count · conjunctions
1
1
count · word count range
1
1
format · list
1
1
words · keywords specific position
1
1
words · words position
1
1
Provider-reported usage
Thinking cost
What the run cost the model in thinking: provider-reported completion
tokens per prompt, the thinking share, and the cost per correct answer.
Reasoning-heavy models can exhaust fixed budgets before answering.
Thinking cost for muse-glimmer-30b.
Suite
Cost
Reading
IFEval-50
781/prompt
0%
thinking· 781/correct
GSM8K-50
563/prompt
0%
thinking· 5,064/correct
Track B · blind-judged
Artifacts
Blind-judged by nvidia/nemotron-3.5-super-vl:free: the judge saw
the brief and the rendered image only, never the model name. Briefs
are frozen; expand one to read it.
beach
unjudged
no svg
Brief
Draw a beautiful sunset beach scene: a large sun sitting low over the sea
with a shimmering reflection path on the water, a leaning palm tree
silhouette on the left, two dolphins arcing out of the waves, a small
sailboat on the right, birds in the sky, sandy foreground with a starfish.
Warm oranges and pinks fading to teal. Coherent perspective: distant
objects smaller and hazier than near ones.
Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external
references) inside a single ```svg fenced block. Short prose outside the
fence is allowed.
clock
unjudged
no svg
Brief
Draw a round wall clock face showing exactly 10:10: a circular dial with
all 12 hour numerals in the correct positions, minute tick marks, an hour
hand pointing just past 10, a minute hand pointing exactly at 2, and a thin
second hand pointing at 6. The word DRACON in small capitals under the
center. Dark rim, light face, flat vector style, centered composition.
Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external
references) inside a single ```svg fenced block. Short prose outside the
fence is allowed.
duo
unjudged
no svg
Brief
Draw a comical scene: a dog driving a jet ski at speed, paws on the
handlebars, with a raccoon riding behind holding on, both leaning into a
turn. Sea spray behind the jet ski, sun and clouds in the sky, a distant
shoreline. Both characters must be clearly doing something, not just
posed side by side. Bright cartoon vector style.
Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external
references) inside a single ```svg fenced block. Short prose outside the
fence is allowed.
monalisa
unjudged
no svg
Brief
Redraw the Mona Lisa as a flat vector portrait, bust composition: a seated
woman in three-quarter view facing the viewer, dark dress with neckline
detail, hands folded at the bottom edge, soft enigmatic smile, hair parted
in the middle. Behind her a hazy landscape with a winding river, a bridge
on the right, and blue mountains. Muted greens, browns, and smoky blues.
Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external
references) inside a single ```svg fenced block. Short prose outside the
fence is allowed.
pelican
41/50Judge reasoning
Brief fidelity (9/10)
All requested elements are present: a brown pelican in left-facing side profile with a large beak and throat pouch, long neck, folded wing, webbed feet on flat ground, sky background with a sea horizon, executed in flat vector style with a limited palette.
Construction (8/10)
The bird is built from clean geometric primitives (ovals, circles, a triangle tail) that read as assembled parts rather than melted forms, though the neck-to-body transition and feet are highly simplified.
Composition (8/10)
The pelican is well-centered with generous sky above and ground below, using the 800x600 frame effectively without any cut-off edges or awkward cropping.
Detail finish (8/10)
The finish is clean and intentional with a white eye dot, distinct beak, pouch, wing, and tail; no broken markup, stray shapes, or unfinished edges are visible.
Visual appeal (8/10)
The coherent earth-tone and blue palette, confident flat shapes, and minimalist charm make this a pleasant, visitor-friendly illustration.
Brief
Draw a brown pelican standing in side profile, facing left, full body visible:
large beak with throat pouch, long neck, folded wing, webbed feet on flat
ground. Simple sky background with a hint of sea on the horizon. Flat vector
style, clean shapes, limited palette.
Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external
references) inside a single ```svg fenced block. Short prose outside the
fence is allowed.