sqlite-distinct-agg-arity

Task gallery / DeepSWE (long-horizon SWE) / Version 1

Reference solution (oracle): Fail
No-op resistance check: Pass
QA verdict: Not recorded
Aggregate eligibility: Excluded / other version

Quality exclusion

Known task defect recorded
Reference solution fails

These per-version observations remain visible, but this version does not contribute to aggregate pass rates.

The task

This SQLite build crashes on some queries This build of SQLite has started crashing on the teams using it: certain statements kill the process outright where they should have come back with a plain, boring complaint that the statement is wrong -- and the same statement with one small change does produce exactly that complaint. The reports come from reporting queries that summarise stored data into groups. It reprodu…

Excerpt only; not a complete executable task specification. Task ID task_2ad9f7dfda1e4f8c · version 1. Selection status: not fully recorded.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effort	Verifier result [95% Wilson]	Diagnostic labels / valid	HARNESS_ERROR · excluded	Outcome classes
Claude Opus 5 · default (unverified)	0/3 · 0.0% [0.0%, 56.1%] · preliminary	0 / 3	0	GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0

Legacy evidence

Model / effort	Verifier result [95% Wilson]	Diagnostic labels / valid	HARNESS_ERROR · excluded	Outcome classes

Every recorded run

7 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / model	Date / source	Reward / eligibility	Diagnostic class / source	Resources	Evidence
`trial_12bc90f5ebbc4d00` Claude Opus 5 · default (unverified)	Oct 8, 2026 console	0 · valid	No diagnostic label derived	Tokens: 190,472 Turns: 179 Cost: $23.2571	Not publicly available
`trial_dc415300d5c7430c` Claude Opus 5 · default (unverified)	Oct 8, 2026 console	0 · valid	No diagnostic label derived	Tokens: 145,272 Turns: 109 Cost: $12.8561	Not publicly available
`trial_e3912e5ad20a4b0d` Claude Opus 5 · default (unverified)	Oct 8, 2026 console	0 · valid	No diagnostic label derived	Tokens: 151,006 Turns: 119 Cost: $14.0173	Not publicly available
`trial_3c21336cbd75480d` oracle@@	Oct 8, 2026 console	0 · excludedsource_evidence_exclusion	No diagnostic label derived	Tokens: — Turns: — Cost: —	Not publicly available
`trial_81cabfb577d545df` nop@@	Oct 8, 2026 console	0 · excludedsource_evidence_exclusion	No diagnostic label derived	Tokens: — Turns: — Cost: —	Not publicly available
`trial_0b21eace18ef474c` oracle@@	Oct 21, 2026 console	1 · valid	GOOD_SUCCESS frozen-classifier	Tokens: — Turns: — Cost: —	Not publicly available
`trial_eb5d531bd2a44515` nop@@	Oct 21, 2026 console	0 · valid	GOOD_FAILURE frozen-classifier	Tokens: — Turns: — Cost: —	Not publicly available