Task gallery / Software engineering / Version 1
Task evidence / console
timescaledb-fix-set-returning-functions-v4
uncalibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- taxonomy: Evaluating; QA: Local QA passed (lead, 10-13 02:25): diff-only test.sh
- Aggregate eligibility
- Eligible version
The task
Some operations in this TimescaleDB build return the wrong results Teams using this build of TimescaleDB, a time-series extension for PostgreSQL, have started filing reports: queries over hypertables misbehave. Some return the wrong rows, and some fail with an error where the same query on a plain PostgreSQL table succeeds. Nothing crashes, it compiles cleanly, and its own test suite is green, which is exactly how t…
Excerpt only; not a complete executable task specification. Task ID task_7140c07984dd4e8c · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · default (unverified) | 4/4 · 100.0% [51.0%, 100.0%] · preliminary | 4 / 4 | 2 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 4 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
19 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_110fa940e18c48b6oracle@@ | Oct 13, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_4bf12e8e12bc41a4nop@@ | Oct 13, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: — Turns: — Cost: — | Not publicly available |
trial_9a90c63e838e41beClaude Opus 5 · default (unverified) | Oct 13, 2026 console | 1 · excludedharness_error | HARNESS_ERROR independent-review | Tokens: 83,110 Turns: 114 Cost: $10.8474 | Not publicly available |
trial_c222cc59d78045d1Claude Opus 5 · default (unverified) | Oct 13, 2026 console | 0 · excludedharness_error | HARNESS_ERROR independent-review | Tokens: 61,723 Turns: 89 Cost: $6.6433 | Not publicly available |
trial_edfdf9947e114bcfClaude Opus 5 · default (unverified) | Oct 13, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: 123,042 Turns: 146 Cost: $16.3668 | Not publicly available |
trial_547a640c028b47bdClaude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5a32ddbe869749b6Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_63fb8fce610545a0Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_705017f8597e4b6aClaude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7be0d981af674e8eClaude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a5072a4a6c9d4e42Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_cfd68c95fce04fc5Claude Opus 5 · default (unverified) | Oct 13, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_20df4e2bb6df4907Claude Opus 5 · default (unverified) | Oct 14, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6a64e602335445e1Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: 155,488 Turns: 160 Cost: $21.0340 | Not publicly available |
trial_82ba23cf84334dfdClaude Opus 5 · default (unverified) | Oct 14, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_aa6953902d244299Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: 240,309 Turns: 202 Cost: $34.4603 | Not publicly available |
trial_f0df0c94b26e4e3cClaude Opus 5 · default (unverified) | Oct 14, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_f972eb5100e942a6Claude Opus 5 · default (unverified) | Oct 14, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_fe9be4fb47dc4361Claude Opus 5 · default (unverified) | Oct 14, 2026 console | 1 · valid | GOOD_SUCCESS frozen-classifier | Tokens: 148,704 Turns: 180 Cost: $23.0173 | Not publicly available |