Task gallery / DeepSWE (long-horizon SWE) / Version 4
Task evidence / console
timescaledb-fix-invalidation-and-batchin
contested/defectiveOutside aggregate scope- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- deepswe review: EXCLUDED from delivery (coordinator, 2026-10-22 ~02:50Z); runs c
- Aggregate eligibility
- Excluded / other version
Quality exclusion
- Known task defect recorded
These per-version observations remain visible, but this version does not contribute to aggregate pass rates.
The task
Some operations in this TimescaleDB build return the wrong results Teams using this build of TimescaleDB, a time-series extension for PostgreSQL, have started filing reports: queries over hypertables return the wrong rows. There is no crash and no error message. It compiles cleanly and its own test suite is green, which is exactly how this reached them unnoticed. The reports cluster around refreshing continuous aggr…
Excerpt only; not a complete executable task specification. Task ID task_0d0eb0d05b024025 · version 4. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · high | 0/0 | 0 / 0 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
15 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_354cb6106af24a32oracle@@ | Oct 22, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_42db70d0c9ae4533nop@@ | Oct 22, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_1d4158d970154f01Claude Opus 5 · high | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5187099881c244e6Claude Opus 5 · high | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6ae41c0c84634e27Claude Opus 5 · high | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_713019c478724531Claude Opus 5 · high | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a9e6f209e7fc4b10Claude Opus 5 · high | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_c2a4d40dbe0648aatencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_1b4cda00c4b64915tencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5615e71c43bf4e7btencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_70e058c857b14bf8tencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7271d72e84ef4211tencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_8a06cb48ec4b44e2tencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_c970b9cff8aa4dcbtencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_e3f8c81e21cb44e9tencent/hy4-preview@@ | Oct 22, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |