kilo-auto/free

Tested First tested 2026-11-11 · Kilo gateway, anonymous

Kilo Auto router (anonymous free tier): IFEval-50 87.8%, GSM8K-50 92.0% with zero failures, SVG 145/250 over 5 judged tests including the best clock on the board (43/50). Collection survived the anonymous promo limit and AtlasCloud 400s; 1 IFEval prompt unrecoverable, counted as failure. Router product: scores describe the route, not one model.

Scores

Suite	Score
IFEval-50 prompt-strict	87.8% (n=49)
IFEval-50 instruction-strict	91.0% (n=49)
GSM8K-50	92.0% (46/50)
SVG blind rubric	145/250

Artifacts

Blind-judged by nvidia/nemotron-3.5-super-vl:free: the judge saw the brief and the rendered image only, never the model name. Briefs are frozen; expand one to read it.

beach · 30/50

Brief fidelity (6/10): All headline subjects are present, sun, reflection path, leaning palm, two dolphin shapes, sailboat, birds, starfish, and sand, but the dolphins are abstract ovals rather than arcing figures and the sailboat is minimal.
Construction (6/10): Shapes are cleanly assembled vector forms with no melting or duplication, though the dolphins and sailboat lack anatomical or structural detail.
Composition (7/10): Clear horizontal zones of sky, sea, and sand with the palm left, sun center, sailboat right, and starfish in the foreground; perspective is basic but coherent.
Detail finish (5/10): The gradient sky and sun glow provide atmosphere, but there are no textures, wave details, or shimmering effects; no broken markup or cut-off edges are visible.
Visual appeal (6/10): The warm-to-cool gradient palette is pleasing and the linework is confident, but extreme minimalism limits charm and visual impact.

Brief

Draw a beautiful sunset beach scene: a large sun sitting low over the sea with a shimmering reflection path on the water, a leaning palm tree silhouette on the left, two dolphins arcing out of the waves, a small sailboat on the right, birds in the sky, sandy foreground with a starfish. Warm oranges and pinks fading to teal. Coherent perspective: distant objects smaller and hazier than near ones. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

clock · 43/50

Brief fidelity (9/10): All requested elements are present: 12 ordered numerals, minute tick marks, hour hand near 10, minute hand at 2, thin red second hand at 6, DRACON text under center, dark rim, light face, and centered composition.
Construction (9/10): Geometric shapes are clean and assembled correctly; hands meet at a single center point, numerals are legible, and nothing appears melted, melted, or duplicated.
Composition (9/10): The clock is centered with balanced proportions; the dark rim frames the face well and numerals are evenly spaced around the dial within the 800x600 frame.
Detail finish (8/10): Flat vector style is consistent with the brief; the red second hand and small-capitals text add finishing touches, with no visible broken markup or cut-off edges.
Visual appeal (8/10): The palette of dark gray, off-white, black, and red accent is coherent and confident; the minimalist design is clean and effective.

Brief

Draw a round wall clock face showing exactly 10:10: a circular dial with all 12 hour numerals in the correct positions, minute tick marks, an hour hand pointing just past 10, a minute hand pointing exactly at 2, and a thin second hand pointing at 6. The word DRACON in small capitals under the center. Dark rim, light face, flat vector style, centered composition. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

duo · 31/50

Brief fidelity (7/10): All requested elements are present: dog driving, raccoon passenger behind, jet ski, sea spray dots, sun, clouds, and distant shoreline, with both characters actively engaged in the ride.
Construction (5/10): Bodies and faces are recognizable but highly simplified geometric assemblies; paws are stick lines and bodies merge with the jet ski, though nothing is melted or duplicated.
Composition (7/10): The frame uses clear horizontal bands for sky, shore, and water, with the angled jet ski and characters centered in the lower third for a readable side-profile action layout.
Detail finish (5/10): The image uses flat cartoon colors with minimal shading and only basic spray dots as texture, but the markup appears clean with no cut-off edges or stray shapes.
Visual appeal (7/10): A bright, coherent palette and confident black outlines give the comical scene charm, though the extreme simplicity limits visual sophistication.

Brief

Draw a comical scene: a dog driving a jet ski at speed, paws on the handlebars, with a raccoon riding behind holding on, both leaning into a turn. Sea spray behind the jet ski, sun and clouds in the sky, a distant shoreline. Both characters must be clearly doing something, not just posed side by side. Bright cartoon vector style. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

monalisa · 14/50

Brief fidelity (2/10): The image includes the dark dress, neckline, smile, and blue mountains, but lacks the three-quarter view, winding river, distinct parted hair, and clearly folded hands, missing multiple headline elements.
Construction (4/10): The figure is built from clean geometric shapes (circle head, rectangular neck, trapezoidal torso) but lacks a nose, ears, and realistic hands, appearing more like an abstract icon than an assembled portrait.
Composition (3/10): The composition centers a disproportionately large head with a tiny neck and cuts the body at the frame bottom, using the space but with poor relative scale and perspective.
Detail finish (2/10): The artwork is clean and unbroken but nearly devoid of shading, texture, or fine details beyond basic flat shapes.
Visual appeal (3/10): The muted blue-brown-peach palette is coherent and the linework is confident, but the extreme abstraction limits charm and recognizability as the Mona Lisa.

Brief

Redraw the Mona Lisa as a flat vector portrait, bust composition: a seated woman in three-quarter view facing the viewer, dark dress with neckline detail, hands folded at the bottom edge, soft enigmatic smile, hair parted in the middle. Behind her a hazy landscape with a winding river, a bridge on the right, and blue mountains. Muted greens, browns, and smoky blues. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

pelican · 27/50

Brief fidelity (4/10): The pelican, beak, feet, and horizon are present, but the requested long neck is essentially absent and the throat pouch is only minimally suggested, capping fidelity at 4.
Construction (6/10): Clean vector shapes assemble the bird without melting or duplication, though the neck-to-body connection is abrupt and the wing is just a flat color patch.
Composition (6/10): The bird is centered with a clear horizon line, but the frame is dominated by empty sky and the head-to-body scale is stylized rather than naturalistic.
Detail finish (5/10): The artwork uses flat fills with a simple eye and V-toed feet, but lacks shading, texture, or additional finishing touches beyond the basics.
Visual appeal (6/10): The limited brown-and-blue palette is coherent and the linework is confident, but the extreme minimalism limits charm and visual interest.

Brief

Draw a brown pelican standing in side profile, facing left, full body visible: large beak with throat pouch, long neck, folded wing, webbed feet on flat ground. Simple sky background with a hint of sea on the horizon. Flat vector style, clean shapes, limited palette. Output: exactly one self-contained SVG (viewBox "0 0 800 600", no external references) inside a single ```svg fenced block. Short prose outside the fence is allowed.

Compare with

Evidence and notes

Ties best GSM8K-50 on the board: 92.0% (46/50)
Best single clock on the board: 43/50
1 IFEval prompt unrecoverable (persistent AtlasCloud 400), counted as failure
Probes routed to dots-studio/dots-3-note-preview:free via AtlasCloud, confirming router behavior