Task gallery / Machine learning / Version 1
Task evidence / repo
compressor-fleet-onset-classifier
legacy-onlyChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x1, accept x1
- Aggregate eligibility
- Eligible version
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-3703682fe32fb6c9 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 4.8 · default (unverified) | 7/10 · 70.0% [39.7%, 89.2%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| GPT-5.5 · default (unverified) | 5/10 · 50.0% [23.7%, 76.3%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
28 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__hfwR92coracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__MjVqu6Toracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__eF7MYPAoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__ehNNqHfnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__SP8qpWrClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 71,436 Turns: — Cost: $5.0474 | Not publicly available |
tbench-task__jhaQJLsClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 34,478 Turns: — Cost: $2.0745 | Not publicly available |
tbench-task__RA3qGZ8Claude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 36,472 Turns: — Cost: $2.3927 | Not publicly available |
tbench-task__8wVbCGLClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 28,259 Turns: — Cost: $1.8133 | Not publicly available |
tbench-task__agchsvbClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 37,318 Turns: — Cost: $2.3870 | Not publicly available |
tbench-task__GKKmB9qGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 17,910 Turns: — Cost: $2.8014 | Not publicly available |
tbench-task__D43isuiGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,584 Turns: — Cost: $2.3691 | Not publicly available |
tbench-task__4K6rk6DGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 18,352 Turns: — Cost: $2.9942 | Not publicly available |
tbench-task__jBGiJXGGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 35,892 Turns: — Cost: $5.3619 | Not publicly available |
tbench-task__SaUbjTKGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,509 Turns: — Cost: $2.2874 | Not publicly available |
tbench-task__zRNFZyforacle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__io6ndQgoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__SwjzcVpnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__HKVTDFdoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__2M7ZAocClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 31,553 Turns: — Cost: $1.8902 | Not publicly available |
tbench-task__uEFK4vhClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 29,343 Turns: — Cost: $1.8416 | Not publicly available |
tbench-task__324mTfWClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 31,119 Turns: — Cost: $2.0899 | Not publicly available |
tbench-task__it3BR4JClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 27,494 Turns: — Cost: $1.6832 | Not publicly available |
tbench-task__yNkQPbGClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 36,497 Turns: — Cost: $2.3063 | Not publicly available |
tbench-task__rYuuBxWGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 28,257 Turns: — Cost: $5.2232 | Not publicly available |
tbench-task__3cSr2uNGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 18,811 Turns: — Cost: $2.1850 | Not publicly available |
tbench-task__V9eBnENGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 26,011 Turns: — Cost: $3.9190 | Not publicly available |
tbench-task__vyMyPWPGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 24,221 Turns: — Cost: $4.3566 | Not publicly available |
tbench-task__i3gLMwuGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 17,459 Turns: — Cost: $2.8712 | Not publicly available |