Task gallery / Scientific computing / Version 1
Task evidence / repo
Snorkel-reviewed task review-5500027766d80265
legacy-onlyChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x3
- Aggregate eligibility
- Eligible version
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-5500027766d80265 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| GPT-5.5 · default (unverified) | 2/15 · 13.3% [3.7%, 37.9%] | 0 / 15 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| Claude Opus 4.8 · default (unverified) | 0/15 · 0.0% [0.0%, 20.4%] | 0 / 15 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
42 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__JFmembDoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__gpKU28horacle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__gQZ9mSGoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__4QTaPw3nop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__NNz7a7pGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,834 Turns: — Cost: $1.2075 | Not publicly available |
tbench-task__CbCqgqKClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,950 Turns: — Cost: $0.9526 | Not publicly available |
tbench-task__uessUZeClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 21,390 Turns: — Cost: $1.0562 | Not publicly available |
tbench-task__g5hocYTGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 9,181 Turns: — Cost: $0.8246 | Not publicly available |
tbench-task__kM5vGZTGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,803 Turns: — Cost: $1.7301 | Not publicly available |
tbench-task__zzzzUXEGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 9,459 Turns: — Cost: $1.0840 | Not publicly available |
tbench-task__6bYL9UYClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,771 Turns: — Cost: $0.6519 | Not publicly available |
tbench-task__Kvp25zCClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,900 Turns: — Cost: $0.7579 | Not publicly available |
tbench-task__q6VEczGClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 13,538 Turns: — Cost: $0.7444 | Not publicly available |
tbench-task__6mgw5tfGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,965 Turns: — Cost: $1.1847 | Not publicly available |
tbench-task__vP7pE2Eoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__wjJx56Qnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__hZzJBrmoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__kwEsazboracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__WfJoYk8Claude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 16,739 Turns: — Cost: $0.8293 | Not publicly available |
tbench-task__W8aRxZEClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 15,145 Turns: — Cost: $0.7459 | Not publicly available |
tbench-task__C2PkccrGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,695 Turns: — Cost: $1.0775 | Not publicly available |
tbench-task__65FVkQsClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,264 Turns: — Cost: $0.5728 | Not publicly available |
tbench-task__Gytz4utGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,380 Turns: — Cost: $0.8846 | Not publicly available |
tbench-task__Ca6KeyDClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,163 Turns: — Cost: $0.5313 | Not publicly available |
tbench-task__KR5EFwPGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 9,138 Turns: — Cost: $1.5425 | Not publicly available |
tbench-task__HDByhYHGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,516 Turns: — Cost: $2.0542 | Not publicly available |
tbench-task__ce7srGAGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 8,205 Turns: — Cost: $0.9239 | Not publicly available |
tbench-task__Jv35MnbClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,557 Turns: — Cost: $0.6949 | Not publicly available |
tbench-task__yEC2TjJoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__SUFGAv2oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__yh74Vzdoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__fQ2BUwonop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__48Zq89GGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,262 Turns: — Cost: $1.5597 | Not publicly available |
tbench-task__sHyRPCWGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,378 Turns: — Cost: $1.7574 | Not publicly available |
tbench-task__NAJgnYbClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,569 Turns: — Cost: $0.5940 | Not publicly available |
tbench-task__EaGZP9KGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 8,107 Turns: — Cost: $1.0958 | Not publicly available |
tbench-task__CKyz5kGGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 7,229 Turns: — Cost: $0.9630 | Not publicly available |
tbench-task__nEmC8VTClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,664 Turns: — Cost: $0.6181 | Not publicly available |
tbench-task__GvxrAurClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 17,930 Turns: — Cost: $0.9362 | Not publicly available |
tbench-task__sj42rs7Claude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 16,871 Turns: — Cost: $0.8525 | Not publicly available |
tbench-task__HvXWGNiClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,557 Turns: — Cost: $0.6320 | Not publicly available |
tbench-task__s5uMDDzGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 15,699 Turns: — Cost: $1.8868 | Not publicly available |