Task gallery / DeepSWE (long-horizon SWE) / Version 1
Task evidence / console
timescaledb-fix-segmentby-crash-in-cagg
uncalibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- Not recorded
- Aggregate eligibility
- Eligible version
The task
Some operations in this TimescaleDB build return the wrong results Teams using this build of TimescaleDB, a time-series extension for PostgreSQL, have started filing reports: queries over hypertables return the wrong rows. There is no crash and no error message. It compiles cleanly and its own test suite is green, which is exactly how this reached them unnoticed. The reports cluster around inserts and deletes on col…
Excerpt only; not a complete executable task specification. Task ID task_cb67d085c2e3483b · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · default (unverified) | 3/3 · 100.0% [43.9%, 100.0%] · preliminary | 0 / 3 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| Claude Opus 5 · high | 0/0 | 0 / 0 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
11 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_04b717e1b9de480bClaude Opus 5 · default (unverified) | Oct 8, 2026 console | 1 · valid | No diagnostic label derived | Tokens: 216,537 Turns: 249 Cost: $34.8971 | Not publicly available |
trial_5fa173f1cc824fcdClaude Opus 5 · default (unverified) | Oct 8, 2026 console | 1 · valid | No diagnostic label derived | Tokens: 147,295 Turns: 188 Cost: $20.3964 | Not publicly available |
trial_6441bde2fc7044ddClaude Opus 5 · default (unverified) | Oct 8, 2026 console | 1 · valid | No diagnostic label derived | Tokens: 48,233 Turns: 61 Cost: $4.4919 | Not publicly available |
trial_10b25d7c8141403doracle@@ | Oct 8, 2026 console | 1 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_e6b3354417614eb0nop@@ | Oct 8, 2026 console | 0 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_d2a9e77998dc4657Claude Opus 5 · default (unverified) | Oct 9, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_0405bac7467545c3Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_45d801cecf784659Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_480c78db1ae54e99Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_cfa26fc38ccc46f4Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_cfedfa7a1ce2499cClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |