openssh-certificate-authorization

Task gallery / Security / cyber / Version 1

Reference solution (oracle): Unverified
No-op resistance check: Unverified
QA verdict: EXCLUDE
Aggregate eligibility: Excluded / other version

Quality exclusion

Known task defect recorded
Task fault on at least half of labelled current-model failures

These per-version observations remain visible, but this version does not contribute to aggregate pass rates.

The task

An instruction excerpt is not available in the public snapshot.

Excerpt only; not a complete executable task specification. Task ID task_4faf88d8421f4bf7 · version 1. Selection status: known curated delivery cohort.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effort	Verifier result [95% Wilson]	Diagnostic labels / valid	HARNESS_ERROR · excluded	Outcome classes
Claude Opus 5 · high	3/5 · 60.0% [23.1%, 88.2%]	5 / 5	0	GOOD_FAILURE: 1 BAD_FAILURE: 1 GOOD_SUCCESS: 3 BAD_SUCCESS: 0
Tencent HY4 (preview) · think	3/8 · 37.5% [13.7%, 69.4%]	8 / 8	0	GOOD_FAILURE: 2 BAD_FAILURE: 3 GOOD_SUCCESS: 3 BAD_SUCCESS: 0

Legacy evidence

Model / effort	Verifier result [95% Wilson]	Diagnostic labels / valid	HARNESS_ERROR · excluded	Outcome classes

Every recorded run

13 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / model	Date / source	Reward / eligibility	Diagnostic class / source	Resources	Evidence
`trial_3f53090604ec4026` Claude Opus 5 · high	Oct 22, 2026 delivery	0 · valid	BAD_FAILURE independent-review	Tokens: — Turns: 20 Cost: —	Not publicly available
`trial_440d4f89a0dc4287` Claude Opus 5 · high	Oct 22, 2026 delivery	0 · valid	GOOD_FAILURE independent-review	Tokens: — Turns: 20 Cost: —	Not publicly available
`trial_979d62f4ec5444fc` Claude Opus 5 · high	Oct 22, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 23 Cost: —	Not publicly available
`trial_d7566a99572445ed` Claude Opus 5 · high	Oct 22, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 23 Cost: —	Not publicly available
`trial_ecf044c7ede8452a` Claude Opus 5 · high	Oct 22, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 22 Cost: —	Not publicly available
`trial_09718e36332b4b76` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	0 · valid	BAD_FAILURE independent-review	Tokens: — Turns: 38 Cost: —	Not publicly available
`trial_098b437ae6164de9` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	0 · valid	BAD_FAILURE independent-review	Tokens: — Turns: 21 Cost: —	Not publicly available
`trial_38b78adfceb647e1` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 28 Cost: —	Not publicly available
`trial_6d9f53d781a14300` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	0 · valid	GOOD_FAILURE independent-review	Tokens: — Turns: 46 Cost: —	Not publicly available
`trial_9812690361684090` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	0 · valid	BAD_FAILURE independent-review	Tokens: — Turns: 32 Cost: —	Not publicly available
`trial_dfe217d8f78748b5` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	0 · valid	GOOD_FAILURE independent-review	Tokens: — Turns: 48 Cost: —	Not publicly available
`trial_6c1f5650b5534104` Tencent HY4 (preview) · think	Oct 21, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 30 Cost: —	Not publicly available
`trial_54b32232380340e2` Tencent HY4 (preview) · think	Oct 22, 2026 delivery	1 · valid	GOOD_SUCCESS independent-review	Tokens: — Turns: 47 Cost: —	Not publicly available