Task gallery / Software engineering / Version 1
Task evidence / repo
nvpl-nested-lifecycle-coordinator
legacy-onlyChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x2, accept x1
- Aggregate eligibility
- Eligible version
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-3e0c4ece7f2bc1de · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 4.8 · default (unverified) | 11/15 · 73.3% [48.0%, 89.1%] | 0 / 15 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| GPT-5.5 · default (unverified) | 9/15 · 60.0% [35.7%, 80.2%] | 0 / 15 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
42 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__B8UVU7Xoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__gaq4cnEoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__DdtLqmjoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__M7brboknop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__FYNySxWClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,577 Turns: — Cost: $0.6537 | Not publicly available |
tbench-task__s6XjR8wClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 10,365 Turns: — Cost: $0.6825 | Not publicly available |
tbench-task__EJVGVdJClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 9,279 Turns: — Cost: $0.6356 | Not publicly available |
tbench-task__vrtJSdDClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 10,213 Turns: — Cost: $0.6839 | Not publicly available |
tbench-task__MvAoRMtClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 10,976 Turns: — Cost: $0.7410 | Not publicly available |
tbench-task__BoRv2eLGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 5,678 Turns: — Cost: $0.9114 | Not publicly available |
tbench-task__Pvy24XgGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,311 Turns: — Cost: $1.4327 | Not publicly available |
tbench-task__9F8D7wMGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,015 Turns: — Cost: $1.6173 | Not publicly available |
tbench-task__PQBjVXWGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,570 Turns: — Cost: $1.3250 | Not publicly available |
tbench-task__XrUn7bYGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 10,144 Turns: — Cost: $1.3892 | Not publicly available |
tbench-task__va2UHKAoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__SWLUNAknop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__WJM5Vdooracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__7u4ZUzSoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__ziq2CawClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,092 Turns: — Cost: $0.6151 | Not publicly available |
tbench-task__a2X3NDPClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 7,014 Turns: — Cost: $0.4826 | Not publicly available |
tbench-task__H975JjBClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 13,990 Turns: — Cost: $0.8144 | Not publicly available |
tbench-task__qiTyL8CClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 19,233 Turns: — Cost: $0.9969 | Not publicly available |
tbench-task__EP6NYv5Claude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 10,370 Turns: — Cost: $0.6651 | Not publicly available |
tbench-task__AwNgdodGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 6,416 Turns: — Cost: $1.0877 | Not publicly available |
tbench-task__3JVEdwpGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,807 Turns: — Cost: $1.1558 | Not publicly available |
tbench-task__4XTntBeGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 4,761 Turns: — Cost: $0.5858 | Not publicly available |
tbench-task__4yLmo5KGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 6,241 Turns: — Cost: $0.7727 | Not publicly available |
tbench-task__CkYFW9sGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 8,972 Turns: — Cost: $1.2034 | Not publicly available |
tbench-task__dRuXcW7oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__7bczgPcnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__XkGCjscoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__8Mebw76oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__bEQJKQ8Claude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,828 Turns: — Cost: $0.6713 | Not publicly available |
tbench-task__cDpRv9MClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,752 Turns: — Cost: $0.6064 | Not publicly available |
tbench-task__EhsKAMyClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 9,824 Turns: — Cost: $0.6254 | Not publicly available |
tbench-task__sMRSPbCClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 8,408 Turns: — Cost: $0.5396 | Not publicly available |
tbench-task__hVmzgEKClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 7,820 Turns: — Cost: $0.5275 | Not publicly available |
tbench-task__Lck9SgQGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 8,419 Turns: — Cost: $1.1680 | Not publicly available |
tbench-task__HZP8H9xGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 5,001 Turns: — Cost: $0.7599 | Not publicly available |
tbench-task__8Ms4xq8GPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 5,056 Turns: — Cost: $0.8180 | Not publicly available |
tbench-task__Uz6aqkaGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 5,744 Turns: — Cost: $0.8278 | Not publicly available |
tbench-task__hT86PEdGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 5,234 Turns: — Cost: $0.8873 | Not publicly available |