Task gallery / Security / cyber / Version 1
Task evidence / console

http-smuggle-repair

hardCurrent calibratedChosen benchmark version
Reference solution (oracle)
Pass
No-op resistance check
Pass
QA verdict
Not recorded
Aggregate eligibility
Eligible version

The task

Fix the front-end proxy /app/repo/httpguard/ is an HTTP/1.1 front-end proxy. It runs each connection descriptor through a pipeline (wire intake, line splitting, request-line parsing, header-list building, message framing, upstream routing, forwarding and hop handling, accounting, and an audit summary) and returns one canonical record per request. Several stages carry silent security defects, so some records come bac…

Excerpt only; not a complete executable task specification. Task ID task_c1fec78f2b0b49e0 · version 1. Selection status: not fully recorded.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes
Claude Opus 5 · default (unverified)0/10 · 0.0% [0.0%, 27.8%]5 / 100
GOOD_FAILURE: 5
BAD_FAILURE: 0
GOOD_SUCCESS: 0
BAD_SUCCESS: 0

Legacy evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes

Difficulty, with uncertainty

Selected-best difficulty: 100.0% [72.2%, 100.0%]. This is one minus the best measured current-row pass rate among rows with at least five valid runs.

The maximum is selected from the observed rows, so the estimate is optimistic. Its Wilson interval is not adjusted for model selection. “Hard” requires qualifying GOOD_FAILURE evidence; the threshold flag additionally requires the interval and recorded quality gates to meet the documented rule. This is not independent validation. Threshold flag: yes.

Every recorded run

14 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / modelDate / sourceReward / eligibilityDiagnostic class / sourceResourcesEvidence
trial_d06cb75ef47f459b
oracle@@		Oct 8, 2026
console		1 · excludedsource_evidence_exclusionNo diagnostic label
derived		Tokens: —
Turns: —
Cost: —		Not publicly available
trial_b4a6adf755c848c3
nop@@		Oct 8, 2026
console		0 · excludedsource_evidence_exclusionNo diagnostic label
derived		Tokens: —
Turns: —
Cost: —		Not publicly available
trial_5a4b1e153e2e4420
Claude Opus 5 · default (unverified)		Oct 8, 2026
console		0 · validNo diagnostic label
derived		Tokens: 29,187
Turns: 31
Cost: $2.0668		Not publicly available
trial_8408affe3aa04318
Claude Opus 5 · default (unverified)		Oct 8, 2026
console		0 · validNo diagnostic label
derived		Tokens: 31,775
Turns: 32
Cost: $2.1183		Not publicly available
trial_2e7db1c90540412f
Claude Opus 5 · default (unverified)		Oct 8, 2026
console		0 · validNo diagnostic label
derived		Tokens: 29,681
Turns: 31
Cost: $2.0032		Not publicly available
trial_d3f06dcc17d04636
Claude Opus 5 · default (unverified)		Oct 8, 2026
console		0 · validNo diagnostic label
derived		Tokens: 29,616
Turns: 33
Cost: $1.9680		Not publicly available
trial_ec6de42ea9214b9b
Claude Opus 5 · default (unverified)		Oct 8, 2026
console		0 · validNo diagnostic label
derived		Tokens: 26,814
Turns: 11
Cost: $1.5693		Not publicly available
trial_21f539c8a80245c7
oracle@@		Oct 10, 2026
console		1 · validNo diagnostic label
derived		Tokens: —
Turns: —
Cost: —		Not publicly available
trial_4aef7ff6be3a41ca
nop@@		Oct 10, 2026
console		0 · validNo diagnostic label
derived		Tokens: —
Turns: —
Cost: —		Not publicly available
trial_ae0cae6d4e324cb7
Claude Opus 5 · default (unverified)		Oct 10, 2026
console		0 · validGOOD_FAILURE
frozen-classifier		Tokens: 30,474
Turns: 32
Cost: $1.9467		Not publicly available
trial_d326ee391354474f
Claude Opus 5 · default (unverified)		Oct 10, 2026
console		0 · validGOOD_FAILURE
frozen-classifier		Tokens: 27,393
Turns: 34
Cost: $1.7980		Not publicly available
trial_e807fc1a86464853
Claude Opus 5 · default (unverified)		Oct 10, 2026
console		0 · validGOOD_FAILURE
frozen-classifier		Tokens: 37,997
Turns: 35
Cost: $2.4624		Not publicly available
trial_e85bf27aad4841a1
Claude Opus 5 · default (unverified)		Oct 10, 2026
console		0 · validGOOD_FAILURE
frozen-classifier		Tokens: 28,447
Turns: 31
Cost: $1.8928		Not publicly available
trial_f432c147ad5d46f0
Claude Opus 5 · default (unverified)		Oct 10, 2026
console		0 · validGOOD_FAILURE
frozen-classifier		Tokens: 29,204
Turns: 38
Cost: $2.6233		Not publicly available