Task gallery / Software engineering / Version 1
Task evidence / console
qdrant-fix-flusher-data-race-in-gridstor
uncalibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- Not recorded
- Aggregate eligibility
- Eligible version
The task
Some searches in this Qdrant build return the wrong results Teams using this build of Qdrant have started filing reports: certain queries come back with results that are simply wrong. There is no panic and no error, the request succeeds and the payload it returns is incorrect. The workspace compiles cleanly and cargo test is green, which is exactly how this reached them unnoticed. The reports do have one thing in co…
Excerpt only; not a complete executable task specification. Task ID task_164a99c5cebc4aff · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · default (unverified) | 0/3 · 0.0% [0.0%, 56.1%] · preliminary | 2 / 3 | 0 | GOOD_FAILURE: 1 BAD_FAILURE: 1 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| Claude Opus 5 · high | 0/0 | 0 / 0 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
20 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_06561a9080f44ec2Claude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_072230581b5d436eClaude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_9aa1633271f84354Claude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_cd299a1c785143fforacle@@ | Oct 8, 2026 console | 1 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_e81d8c39a31040b5nop@@ | Oct 8, 2026 console | 0 · excludedsource_evidence_exclusion | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
trial_2d06b36e7bcf41d4Claude Opus 5 · default (unverified) | Oct 8, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_939b86a5fc7e4d3fClaude Opus 5 · default (unverified) | Oct 9, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: 89,698 Turns: 71 Cost: $8.4702 | Not publicly available |
trial_07a46f12e6954499Claude Opus 5 · default (unverified) | Oct 9, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_02d275b93c4445d7Claude Opus 5 · default (unverified) | Oct 9, 2026 console | 0 · valid | No diagnostic label derived | Tokens: 122,506 Turns: 115 Cost: $14.9538 | Not publicly available |
trial_e27b6ed865d345f0Claude Opus 5 · default (unverified) | Oct 9, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_8167f313b4a944aaClaude Opus 5 · default (unverified) | Oct 9, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_b01759e0a6b94a38Claude Opus 5 · default (unverified) | Oct 12, 2026 console | 0 · valid | GOOD_FAILURE frozen-classifier | Tokens: 102,615 Turns: 72 Cost: $9.1796 | Not publicly available |
trial_c2ba03d894994e17Claude Opus 5 · default (unverified) | Oct 12, 2026 console | 0 · valid | BAD_FAILURE frozen-classifier | Tokens: 133,523 Turns: 148 Cost: $18.0972 | Not publicly available |
trial_6070edd8e31a4b38Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6b9e2a87b3154a86Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a780dbd6a50f4151Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_bbc887065e0546bbClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_cfa8460f9d2e43c4Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_826dc52cda2c4c30Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_e54cdc52278d4d16Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |