Task gallery / DeepSWE (long-horizon SWE) / Version 3
Task evidence / console
qdrant-fix-wrong-multivectors-count-in-s
contested/defectiveOutside aggregate scope- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- Not recorded
- Aggregate eligibility
- Excluded / other version
Quality exclusion
- Known task defect recorded
These per-version observations remain visible, but this version does not contribute to aggregate pass rates.
The task
Some searches in this Qdrant build return the wrong results Teams using this build of Qdrant have started filing reports: certain queries come back with results that are simply wrong. There is no panic and no error, the request succeeds and the payload it returns is incorrect. The workspace compiles cleanly and cargo test is green, which is exactly how this reached them unnoticed. The reports do have one thing in co…
Excerpt only; not a complete executable task specification. Task ID task_efac1df7688e4a98 · version 3. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · high | 0/0 | 0 / 0 | 0 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
30 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_5049a969e8234544oracle@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_7fa8c83aa74b4482nop@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_1845b26f50e04d18tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_42fa4f2861d44e85tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6a3db0c304c64123tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_95b0cc9e5d0a4fb9tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_97fb6020b72e41b6tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_bdd49ebbd50a46dftencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_df983357867f4483tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_ea4c54eeb05f40d8tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_5e4e4f6bbef2438dClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_b99837e4a9c64cbaClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_ce08bb8593a746a6Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_d1d8a5d38c7e4762Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_ead7d621e14d43b1Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_29c6ce3dcf6d434ftencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_354f9459ac0e438atencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_4ba59a4b75614e96tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_6eb2182393f74b60tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a0174cc7e1a04b0etencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_afab1a4ee8314ccctencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_c078eb26d23644d4tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_c20d1b00f8e84a15tencent/hy4-preview@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_18f79d6dd68a4caaClaude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_1e839cc73e454111Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_2df89096a47a41c9Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a5e1cf8397c04e68Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_a899317492b048a3Claude Opus 5 · high | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_37fe1254deda4e9doracle@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |
trial_63c5c001d1ae4323nop@@ | Oct 21, 2026 console | No reward · excludedno_reward | No diagnostic label No diagnostic source | Tokens: — Turns: — Cost: — | Not publicly available |