Task gallery / Security / cyber / Version 1
Reference solution (oracle)
Unverified
No-op resistance check
Unverified
QA verdict
Not recorded
Aggregate eligibility
Eligible version

The task

Rebuild weft from its observable behaviour /app/oraclebinary/weft is a working program you may run but may not read. Your job is to write a second program that behaves the same way, byte for byte, on every input the grader tries. What weft does It reads directives from standard input, one per physical line, and maintains a table of nodes, strands and motifs plus a list of spans built from a weave text. Directives de…

Excerpt only; not a complete executable task specification. Task ID task_f001a4acb02b458a · version 1. Selection status: not fully recorded.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes
Claude Opus 5 · default (unverified)3/5 · 60.0% [23.1%, 88.2%]0 / 50
GOOD_FAILURE: 0
BAD_FAILURE: 0
GOOD_SUCCESS: 0
BAD_SUCCESS: 0

Legacy evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes

Difficulty, with uncertainty

Selected-best difficulty: 40.0% [11.8%, 76.9%]. This is one minus the best measured current-row pass rate among rows with at least five valid runs.

The maximum is selected from the observed rows, so the estimate is optimistic. Its Wilson interval is not adjusted for model selection. “Hard” requires qualifying GOOD_FAILURE evidence; the threshold flag additionally requires the interval and recorded quality gates to meet the documented rule. This is not independent validation. Threshold flag: no.

Every recorded run

5 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / modelDate / sourceReward / eligibilityDiagnostic class / sourceResourcesEvidence
trial_3f6d3b060cff4d56
Claude Opus 5 · default (unverified)
Oct 9, 2026
console
0 · validNo diagnostic label
derived
Tokens: 157,562
Turns: 146
Cost: $20.4853
Not publicly available
trial_7f3b791dfa144a50
Claude Opus 5 · default (unverified)
Oct 9, 2026
console
1 · validNo diagnostic label
derived
Tokens: 205,024
Turns: 185
Cost: $28.9616
Not publicly available
trial_9fd9fdda2e98464b
Claude Opus 5 · default (unverified)
Oct 9, 2026
console
1 · validNo diagnostic label
derived
Tokens: 155,667
Turns: 118
Cost: $17.0245
Not publicly available
trial_d636cf2bce824a99
Claude Opus 5 · default (unverified)
Oct 9, 2026
console
1 · validNo diagnostic label
derived
Tokens: 183,236
Turns: 150
Cost: $22.1229
Not publicly available
trial_df086749ed2c4d9d
Claude Opus 5 · default (unverified)
Oct 9, 2026
console
0 · validNo diagnostic label
derived
Tokens: 202,918
Turns: 185
Cost: $29.7766
Not publicly available