Why we’re building Coop
A successful demo is a starting point. We’re building the physical infrastructure to find out what works, how reliably, and under which conditions.
Read articleReal-world benchmarks for intelligent machines
Coop evaluates robot policies on real hardware, measuring what works, where it fails, and how reliably it performs.
Backed by
Member of
The physical side of AI
Progress in robotics depends on what happens outside the model
Understanding how a robot moves an object, recovers from a mistake, and works in changing conditions takes time with real hardware. Coop turns those questions into repeatable evaluations, with defined conditions, measurable outcomes, and evidence you can inspect.
Benchmarking & evaluations
Know what holds up in the real world
Compare policies, test their limits, and measure progress through controlled experiments on real robots.
Compare policies on common ground
Evaluate compatible policies on the same robot, tasks, and starting conditions. Measure autonomous success, completion time, and interventions, with every trial accounted for.
- The comparison
- Different policies on one documented robot setup
- The evidence
- Per-task results, trial counts, and success and failure recordings
How we benchmark
Trust starts with the protocol
A useful benchmark makes clear what was tested, how it was scored, and what the result means. We build the evaluation around those details from the start.
01Start with a shared protocol
Define the robot, tasks, conditions, and success rules. Check policy compatibility and establish a reference baseline before the scored evaluation begins.
02Make every trial count
Run repeated trials with controlled starting conditions and documented resets. Record failures, interruptions, and human interventions alongside successful runs.
03Connect results to evidence
Review results by task and condition, with trial counts, uncertainty, and recordings. Understand what the comparison supports, where it is limited, and what needs another test.
The Coop journal
Research & insights
The questions, methods, and physical work behind our benchmarks.