etcd-fix-race-berween-read-index-and-lea-v3
hardCurrent calibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- taxonomy: Evaluating; QA: Local QA passed (QA-A v3; 4 redirect probes 0)
- Aggregate eligibility
- Eligible version
The task
Excerpt only; not a complete executable task specification. Task ID task_69bbae703b224f62 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · default (unverified) | 0/7 · 0.0% [0.0%, 35.4%] | 7 / 7 | 3 | GOOD_FAILURE: 7 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Difficulty, with uncertainty
Selected-best difficulty: 100.0% [64.6%, 100.0%]. This is one minus the best measured current-row pass rate among rows with at least five valid runs.
The maximum is selected from the observed rows, so the estimate is optimistic. Its Wilson interval is not adjusted for model selection. “Hard” requires qualifying GOOD_FAILURE evidence; the threshold flag additionally requires the interval and recorded quality gates to meet the documented rule. This is not independent validation. Threshold flag: yes.
Every recorded run
19 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_42121511b59142aforacle@@ | Oct 13, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_9969c434310347a8nop@@ | Oct 13, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_10bcd76dc17642cbClaude Opus 5 · default (unverified) | Oct 13, 2026 console | 0 · excludedharness_error | HARNESS_ERROR independent-review | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_7ca28fb95f5c4f12Claude Opus 5 · default (unverified) | Oct 13, 2026 console | 0 · excludedharness_error | HARNESS_ERROR independent-review | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_e4eab30830b04050Claude Opus 5 · default (unverified) | Oct 13, 2026 console | 0 · excludedharness_error | HARNESS_ERROR independent-review | Tokens: 0 Turns: 1 Cost: $0.0000 | Not publicly available |
trial_d8b2260333d14650Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5254d2ea81eb4badClaude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a3c613dfcce04300Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_87b7fb86fd324377Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_123fd9f4f1e4411dClaude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_fe3f9a24d8204249Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_4a503352a59c4a88Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6ac9bae96dbc4604Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 223,940 Turns: 184 Cost: $29.0062 | Not publicly available |
trial_7b8039a6d2c347b1Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 386,558 Turns: 268 Cost: $59.6007 | Not publicly available |
trial_9e7d1b0293f245abClaude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 218,832 Turns: 210 Cost: $33.0963 | Not publicly available |
trial_c32a862d36ea4f5bClaude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 145,217 Turns: 148 Cost: $18.9154 | Not publicly available |
trial_caee02a91f43453eClaude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 331,612 Turns: 213 Cost: $41.4956 | Not publicly available |
trial_dd68689d506c46e7Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 124,002 Turns: 126 Cost: $14.2065 | Not publicly available |
trial_f1cdb09af87d4f76Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 146,014 Turns: 130 Cost: $16.7453 | Not publicly available |