Task gallery / Cloud operations / Version 1
Reference solution (oracle)
Pass
No-op resistance check
Pass
QA verdict
taxonomy: Evaluating; QA: QA-H sweep (budget fairness)
Aggregate eligibility
Excluded / other version
Quality exclusion
  • Known task defect recorded

These per-version observations remain visible, but this version does not contribute to aggregate pass rates.

The task

The local backup service at /app/bin/snapshotctl uses Tcl 8.6 and the real /usr/bin/rsync to maintain incremental snapshots. Its create, audit, and recover commands are usable, but mixed-change generations, interrupted promotion, and chronological retention can leave reports or repository state inconsistent. Restore the controller without changing its public interface or replacing Tcl or rsync. The authoritative CLI…

Excerpt only; not a complete executable task specification. Task ID task_a822cbe2fcfa498f · version 1. Selection status: not fully recorded.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes
Claude Opus 5 · default (unverified)2/4 · 50.0% [15.0%, 85.0%] · preliminary4 / 46
GOOD_FAILURE: 0
BAD_FAILURE: 2
GOOD_SUCCESS: 2
BAD_SUCCESS: 0

Legacy evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes

Every recorded run

12 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / modelDate / sourceReward / eligibilityDiagnostic class / sourceResourcesEvidence
trial_73230b3c4c374db7
oracle@@
Oct 13, 2026
console
1 · validGOOD_SUCCESS
frozen-classifier
Tokens: —
Turns: —
Cost: —
Not publicly available
trial_9e85dd4454744e82
nop@@
Oct 13, 2026
console
0 · validGOOD_FAILURE
frozen-classifier
Tokens: —
Turns: —
Cost: —
Not publicly available
trial_b48242a48c7940dd
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
1 · validGOOD_SUCCESS
frozen-classifier
Tokens: 142,051
Turns: 123
Cost: $14.4233
Not publicly available
trial_e5ebdc03e7444eb3
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · validBAD_FAILURE
frozen-classifier
Tokens: 130,880
Turns: 116
Cost: $12.5165
Not publicly available
trial_8a6efbc655814c79
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
1 · validGOOD_SUCCESS
frozen-classifier
Tokens: 135,903
Turns: 110
Cost: $12.7895
Not publicly available
trial_07a0a98c6dc04be4
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · validBAD_FAILURE
frozen-classifier
Tokens: 152,972
Turns: 81
Cost: $11.3199
Not publicly available
trial_3ea8f510973d4a18
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_5d878e1bac014b25
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_66b8e382be5a4fbd
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_a410bdec0cbb4624
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_ed09b65b436743a4
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_eda1c871ba364c8c
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available