Task gallery / Scientific computing / Version 1
Task evidence / repo
slope-yield-v2
contested/defectiveOutside aggregate scope- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x2
- Aggregate eligibility
- Excluded / other version
Quality exclusion
- Known task defect recorded
These per-version observations remain visible, but this version does not contribute to aggregate pass rates.
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-d740bc515a9b527e · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 4.8 · default (unverified) | 1/10 · 10.0% [1.8%, 40.4%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| GPT-5.5 · default (unverified) | 1/10 · 10.0% [1.8%, 40.4%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
28 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__ufCr73ioracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__rNmHD3Pnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__8KmHEA4oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__Gd2ytcUoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__aMFoh8BClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 19,339 Turns: — Cost: $0.9121 | Not publicly available |
tbench-task__nwk6CKKClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,901 Turns: — Cost: $0.6640 | Not publicly available |
tbench-task__Ebc8jikClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 18,067 Turns: — Cost: $0.8847 | Not publicly available |
tbench-task__3JTXiDaGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,024 Turns: — Cost: $1.2967 | Not publicly available |
tbench-task__SnfFYGTClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 17,945 Turns: — Cost: $0.8767 | Not publicly available |
tbench-task__CEhWB5SClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 26,276 Turns: — Cost: $1.3345 | Not publicly available |
tbench-task__88kByzVGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,057 Turns: — Cost: $1.9282 | Not publicly available |
tbench-task__dof7LtPGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 8,519 Turns: — Cost: $1.1250 | Not publicly available |
tbench-task__VBo6Y5WGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 15,521 Turns: — Cost: $1.8990 | Not publicly available |
tbench-task__JwMXLj6GPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,444 Turns: — Cost: $1.1053 | Not publicly available |
tbench-task__xckGK7Zoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__WUtzuiHnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__MDiXRsYoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__MTUQzwaoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__P8tFiULGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 8,362 Turns: — Cost: $1.6527 | Not publicly available |
tbench-task__XDdagTNGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,490 Turns: — Cost: $2.0434 | Not publicly available |
tbench-task__X3bH3pbClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 26,024 Turns: — Cost: $1.6911 | Not publicly available |
tbench-task__oTTWpzAGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,616 Turns: — Cost: $2.1455 | Not publicly available |
tbench-task__E4e7V3bGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,935 Turns: — Cost: $1.6121 | Not publicly available |
tbench-task__DPTvAfwClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,448 Turns: — Cost: $0.9024 | Not publicly available |
tbench-task__sZnNdv8Claude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 18,152 Turns: — Cost: $1.2031 | Not publicly available |
tbench-task__QgYb5Z2Claude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,773 Turns: — Cost: $0.9726 | Not publicly available |
tbench-task__DMTaBhDClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 22,390 Turns: — Cost: $1.2889 | Not publicly available |
tbench-task__X9S944qGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 8,743 Turns: — Cost: $1.1115 | Not publicly available |