Task gallery / Software engineering / Version 2
Task evidence / console
rocksdb-fix-compaction-abort-reschedulin
contested/defectiveOutside aggregate scope- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- deepswe review: OUT of delivery (coordinator, 2026-10-22 ~00:55Z)
- Aggregate eligibility
- Excluded / other version
Quality exclusion
- Known task defect recorded
These per-version observations remain visible, but this version does not contribute to aggregate pass rates.
The task
Some operations in this RocksDB build abort instead of completing Teams using this build of RocksDB, a key-value store, have started filing reports with the same shape: an operation that should complete instead brings the process down. No exception they can catch, no error message, just a dead process and whatever the caller was holding lost with it. It compiles cleanly and its own test suite is green, which is exac…
Excerpt only; not a complete executable task specification. Task ID task_758d7c719570445b · version 2. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Tencent HY4 (preview) · think | 4/4 · 100.0% [51.0%, 100.0%] · preliminary | 4 / 4 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 4 BAD_SUCCESS: 0 |
| Claude Opus 5 · high | 0/0 | 0 / 0 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
23 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_663a465f849b4f14nop@@ | Oct 21, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_679dcbe2137648acoracle@@ | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_1073686fb1eb41d3tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_128187f412fc4115tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_133cf37a5bc74c62Tencent HY4 (preview) · think | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS independent-review | Tokens: 433,817 Turns: 530 Cost: $54.4858 | Not publicly available |
trial_256853806ecd4b21tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_35d3b474d94040d5tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_55e9b60fa7ff4a12tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_babdf2c3e0b04df0Tencent HY4 (preview) · think | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS independent-review | Tokens: 932,653 Turns: 369 Cost: $92.2254 | Not publicly available |
trial_be2f9290dc3a4280Tencent HY4 (preview) · think | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS independent-review | Tokens: 305,976 Turns: 419 Cost: $37.8286 | Not publicly available |
trial_814af1d62e794aceClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_88250c5207834da2Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_8ee376dde8ae4b4bClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_af5eb8cca9264e22Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_edbcf486f6024682Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_4a7cb46b0d7c493etencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5ada5792e5ba4bb6tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7611c9bc75f449edTencent HY4 (preview) · think | Oct 21, 2026 console | 1 · valid | GOOD_SUCCESS independent-review | Tokens: 722,558 Turns: 306 Cost: $64.5767 | Not publicly available |
trial_23e73c323aae49b8Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_35c406efdf424ee0Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5636bed6b3bd47deClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_dff0c9bdf83041bdClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_f4a48a480fe04876Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |