Leaderboard

Each evaluation run of a model is one independent repetition. Within a repetition, episode results are averaged per task, and the task means are then averaged with equal weight. Model means and sample variances are computed across repetitions, and error bars show ±1 sample standard deviation. A single repetition has no variance estimate, so its SD and variance are shown as none.

Success rate is the fraction of episodes that meet every success condition of the task. Score is the stage-wise score of an episode on a 0–1 scale, multiplied by 100 in tables and charts; it averages all episodes, including failures. Missing values are shown as none and are never replaced with zero. Hover over a chart value to see its SD, variance, repetition count and, for atomic skills, the number of passed and reached metrics.

Operation families are HP (Handling & Placement), TM (Transfer & Mixing), IO (Instrument Operation) and MD (Measurement & Dosing). A task with two families counts once in each family, so family groups overlap; overall and track results still count each task once. Scope is basic operation or experimental procedure. The four capacity-profile dimensions are Horizon, Precision, Quantitative-target requirement and Reaction-observation requirement.

Atomic-skill rates use the individual metrics of the stages an episode reaches. The first stage is always reached; a later stage is reached when every metric of at least one alternative in the previous stage passes. Each skill a reached metric is credited with adds one to that skill's denominator, and a passing metric adds one to its numerator. Rates are calculated per repetition and then averaged over repetitions with a nonzero denominator.

Test settings: C-only, B-only and I-only vary task configuration, background or instruction alone, and Mixed (C+B+I) varies all three. Demonstrated tasks are tested with 10 episodes per setting (40 per task); zero-shot tasks are tested with 20 Mixed (C+B+I) episodes per task.

Task definitions