Task gallery / Media / Version 1
Reference solution (oracle)
Pass
No-op resistance check
Pass
QA verdict
taxonomy: Evaluating; QA: QA-I READY: oracle root/uid1000 1 (420/420); nop 0; wr
Aggregate eligibility
Eligible version

The task

Implement a TrueType bytecode hinting (grid-fitting) interpreter for ttf-parser. ttf-parser reads glyph outlines, but it discards the TrueType instruction stream: glyf parsing skips the instruction bytes, and the fpgm, prep and cvt tables are never parsed. There is no way to run a glyph's hinting program and obtain grid-fitted points. Add a new public module ttfparser::hint that executes the instruction set and retu…

Excerpt only; not a complete executable task specification. Task ID task_d9d893f549e64959 · version 1. Selection status: not fully recorded.

Results by model and effort

This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.

Current evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes
Claude Opus 5 · default (unverified)0/22 / 21
GOOD_FAILURE: 2
BAD_FAILURE: 0
GOOD_SUCCESS: 0
BAD_SUCCESS: 0

Legacy evidence

Model / effortVerifier result [95% Wilson]Diagnostic labels / validHARNESS_ERROR · excludedOutcome classes

Every recorded run

5 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.

Run / modelDate / sourceReward / eligibilityDiagnostic class / sourceResourcesEvidence
trial_830df33a6f52477c
oracle@@
Oct 14, 2026
console
1 · validGOOD_SUCCESS
frozen-classifier
Tokens: —
Turns: —
Cost: —
Not publicly available
trial_e14c8833bc98483a
nop@@
Oct 14, 2026
console
0 · validGOOD_FAILURE
frozen-classifier
Tokens: —
Turns: —
Cost: —
Not publicly available
trial_1de92f5e12394fdc
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · validGOOD_FAILURE
frozen-classifier
Tokens: 327,858
Turns: 18
Cost: $9.0946
Not publicly available
trial_c65dff1eea0f440b
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · excludedharness_errorHARNESS_ERROR
independent-review
Tokens: 0
Turns: 1
Cost: $0.0000
Not publicly available
trial_d145b164e0b04211
Claude Opus 5 · default (unverified)
Oct 14, 2026
console
0 · validGOOD_FAILURE
frozen-classifier
Tokens: 449,566
Turns: 110
Cost: $30.2328
Not publicly available