$132,000 – $300,000estimated from 206 similar other roles
An estimate from salaries disclosed to us for similar roles, not a figure Robots and Pencils published for this role.
Design and build evaluation frameworks for enterprise agents; benchmark models, run controlled experiments, and create automated insights dashboards for client adoption.
AI-verified last checked 2026-11-14
- Remote policy
- fully-remote
- Role
- Other
- Type
- —
- First seen
- 2026-11-06
About this role
We're looking for a Staff Applied Scientist to lead the design and build of an evaluation framework for an enterprise planner-based agent. You will work directly with the client's EA engineering team to benchmark their agent across planner models, executor LLMs and inference optimization levers. Every proposed change, whether a model upgrade, a harness change or a routing strategy, becomes a controlled, statistically sound experiment with a measurable dollar outcome. This is a research-heavy, customer-facing role with a significant build component. You will own both the experimental method and the tooling that makes it repeatable: a scenario-driven user simulator, isolated experiment execution, statistical comparison, and automated insights in a locally hosted dashboard the client can adopt as their own. The work runs in three overlapping streams (evaluation pipeline, grading and statistics, dashboard and experiment campaign) and ends in a recommendation the client can act on.
Why This Role Matters
At Robots & Pencils, we design AI systems for a human world. Our name says it all. Robots and pencils means engineering paired with creativity, because every agent we ship has to work for real people in real workflows. That balance is baked into how we operate.
An excerpt of the employer's posting (the opening of 7,098 characters). Read the full posting and apply on Robots and Pencils's site ↗
- Team
- AI
- Posted
- 2026-11-05
- Last seen alive
- 2026-11-14
- Last verified
- 2026-11-14
- Verification
- AI-verified
- Board
- greenhouse
Report expired / not remote
Check the employer's posting first. If it is gone or not actually remote, tell us via contact with "Staff Applied Scientist at Robots and Pencils"; the next ingest drops it.
All 29 roles at Robots and Pencils ↗