Task gallery / Machine learning / Version 1
Task evidence / repo
sensor-placement-shift-rare-activity
legacy-onlyChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x1, accept x1
- Aggregate eligibility
- Eligible version
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-4b19088f6b926282 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 4.8 · default (unverified) | 3/10 · 30.0% [10.8%, 60.3%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| GPT-5.5 · default (unverified) | 3/5 · 60.0% [23.1%, 88.2%] | 0 / 5 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
23 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__PSRGKEznop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__i4UtAaioracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__YhxbFsPoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__4kKykUboracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__8pnAWLTClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,660 Turns: — Cost: $0.3704 | Not publicly available |
tbench-task__CKZdtpjClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,899 Turns: — Cost: $0.3765 | Not publicly available |
tbench-task__ktH2mUjClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 7,647 Turns: — Cost: $0.3720 | Not publicly available |
tbench-task__3669bmLClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 8,609 Turns: — Cost: $0.3926 | Not publicly available |
tbench-task__hww5izdClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 5,451 Turns: — Cost: $0.2504 | Not publicly available |
tbench-task__7VVokSYoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__NgRp3T9oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__NmzQmYZnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__kS9jXGCoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__ZCknHVwClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 6,772 Turns: — Cost: $0.3282 | Not publicly available |
tbench-task__QHAbxEEClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,321 Turns: — Cost: $0.4605 | Not publicly available |
tbench-task__f7vU6XGClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 6,532 Turns: — Cost: $0.3041 | Not publicly available |
tbench-task__zDQY8iNClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 7,955 Turns: — Cost: $0.3945 | Not publicly available |
tbench-task__TVgdebSClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 6,111 Turns: — Cost: $0.2934 | Not publicly available |
tbench-task__QUuKydbGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 15,905 Turns: — Cost: $1.8423 | Not publicly available |
tbench-task__GaqtyanGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,441 Turns: — Cost: $1.3352 | Not publicly available |
tbench-task__XYYyYcfGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 15,955 Turns: — Cost: $1.8399 | Not publicly available |
tbench-task__fjU5DHBGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 9,232 Turns: — Cost: $0.9912 | Not publicly available |
tbench-task__Qi95mgGGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,992 Turns: — Cost: $1.6369 | Not publicly available |