Task gallery / Security / cyber / Version 1
Task evidence / console
archive-extract-repair
uncalibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- Not recorded
- Aggregate eligibility
- Eligible version
The task
Repair the archive-extraction guard /app/repo/arcguard/ is the safety check that runs over an archive listing before any of it is written out. Each extraction request carries the entry list, the destination root and a limit profile, and the guard walks it through intake, name canonicalisation, resolution of every entry to a place under the root, link following, the size budgets, the permission bits, and a summary, t…
Excerpt only; not a complete executable task specification. Task ID task_af7650ce24df4abb · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · default (unverified) | 2/3 · 66.7% [20.8%, 93.9%] · preliminary | 0 / 3 | 3 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| Claude Opus 5 · high | 1/2 | 2 / 2 | 0 | GOOD_FAILURE: 1 BAD_FAILURE: 0 GOOD_SUCCESS: 1 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
17 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_05e8e89f83fc4d23Claude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7819f6dc20484d79Claude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_78eff080861b4567Claude Opus 5 · default (unverified) | Oct 8, 2026 console | 0 · excludedharness_error | HARNESS_ERROR frozen-classifier | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_cc4cdf3a99184aadoracle@@ | Oct 8, 2026 console | 1 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_93373ff98afa4441nop@@ | Oct 8, 2026 console | 0 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_c2e2b75e5d954637Claude Opus 5 · default (unverified) | Oct 8, 2026 console | 1 · valid | No diagnostic label derived | Tokens: 29,207 Turns: 36 Cost: $2.6715 | Not publicly available |
trial_8e024ba8c089461eClaude Opus 5 · default (unverified) | Oct 8, 2026 console | 1 · valid | No diagnostic label derived | Tokens: 30,530 Turns: 34 Cost: $2.8224 | Not publicly available |
trial_5197cd5be6c54fcdClaude Opus 5 · default (unverified) | Oct 8, 2026 console | 0 · valid | No diagnostic label derived | Tokens: 33,253 Turns: 42 Cost: $3.4556 | Not publicly available |
trial_067635d3c3084a5eClaude Opus 5 · default (unverified) | Oct 12, 2026 console | 0 · excludedharness_error | HARNESS_ERROR frozen-classifier | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_306f84b22c0945c0Claude Opus 5 · default (unverified) | Oct 12, 2026 console | 0 · excludedharness_error | HARNESS_ERROR frozen-classifier | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_07a025685b27482dClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7791493657044206Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_bfdfa4bb899b4800Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_d7bcd73750b54a9fClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_ec99301973094f68Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_bd8d2a63355f454fClaude Opus 5 · high | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: 44,057 Turns: 46 Cost: $4.2454 | Not publicly available |
trial_c9e9a2d26ef5480aClaude Opus 5 · high | Oct 21, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 37,817 Turns: 44 Cost: $3.3732 | Not publicly available |