Task gallery / Security / cyber / Version 1
Task evidence / repo
chainward-x509-validation
legacy-onlyChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- snorkel review: needs-revision x2, accept x1
- Aggregate eligibility
- Eligible version
The task
An instruction excerpt is not available in the public snapshot.
Excerpt only; not a complete executable task specification. Task ID review-5391cd51fc1d4030 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 4.8 · default (unverified) | 10/15 · 66.7% [41.7%, 84.8%] | 0 / 15 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| GPT-5.5 · default (unverified) | 4/10 · 40.0% [16.8%, 68.7%] | 0 / 10 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Every recorded run
37 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
tbench-task__QNwMiFxnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__nwUmwcdoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__ZhWJsProracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__SNehY95oracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__oahZ3VvClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 15,849 Turns: — Cost: $0.9806 | Not publicly available |
tbench-task__CEPceBoClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 16,320 Turns: — Cost: $1.0493 | Not publicly available |
tbench-task__HEeBwPzClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 21,430 Turns: — Cost: $1.3618 | Not publicly available |
tbench-task__xu5z3xBClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 18,063 Turns: — Cost: $1.0872 | Not publicly available |
tbench-task__2gkYWbyClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 24,370 Turns: — Cost: $1.8440 | Not publicly available |
tbench-task__ebjQC2gGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 8,598 Turns: — Cost: $1.2572 | Not publicly available |
tbench-task__6R2jvhaGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 14,483 Turns: — Cost: $1.9155 | Not publicly available |
tbench-task__brGZvVsGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 12,527 Turns: — Cost: $1.3790 | Not publicly available |
tbench-task__g263MRGGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,962 Turns: — Cost: $1.8330 | Not publicly available |
tbench-task__Tjq6DDNGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 13,108 Turns: — Cost: $1.6430 | Not publicly available |
tbench-task__f9PAkFRoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__FKZALFuoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__8UBBfFWnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__o6xTjYHoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__NEWPiEeClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 16,356 Turns: — Cost: $0.9964 | Not publicly available |
tbench-task__mQGBvRxClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 16,661 Turns: — Cost: $0.9574 | Not publicly available |
tbench-task__DWGRsgRClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 14,367 Turns: — Cost: $0.7970 | Not publicly available |
tbench-task__5a4wzx4Claude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 21,035 Turns: — Cost: $1.4718 | Not publicly available |
tbench-task__rNrrQRYClaude Opus 4.8 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 18,591 Turns: — Cost: $1.0548 | Not publicly available |
tbench-task__fu4GtDjoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__iPWmG8Aoracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__S4Gh5EDnop@@ | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__HV6MzXioracle@@ | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
tbench-task__e9vwKspClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 23,666 Turns: — Cost: $1.3342 | Not publicly available |
tbench-task__BgUU2QeClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 15,342 Turns: — Cost: $0.9262 | Not publicly available |
tbench-task__ANXy5V3Claude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 21,632 Turns: — Cost: $1.2556 | Not publicly available |
tbench-task__6JUY2fpClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 28,729 Turns: — Cost: $1.6121 | Not publicly available |
tbench-task__g6j8eamClaude Opus 4.8 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 26,032 Turns: — Cost: $1.7573 | Not publicly available |
tbench-task__9v2UqcyGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,378 Turns: — Cost: $1.2202 | Not publicly available |
tbench-task__9tfsmqCGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,518 Turns: — Cost: $1.4008 | Not publicly available |
tbench-task__whcFg7aGPT-5.5 · default (unverified) | Not recorded repo | 1 · valid | No diagnostic label derived | Tokens: 14,369 Turns: — Cost: $1.6767 | Not publicly available |
tbench-task__9KBJgWtGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 10,582 Turns: — Cost: $1.3792 | Not publicly available |
tbench-task__Jq547qmGPT-5.5 · default (unverified) | Not recorded repo | 0 · valid | No diagnostic label derived | Tokens: 11,721 Turns: — Cost: $1.5620 | Not publicly available |