Task gallery / Machine learning / Version 1
Task evidence / console
landmark-competing-risk-scorecard
uncalibratedChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- Not recorded
- Aggregate eligibility
- Eligible version
The task
Landmark prediction scorecard The survival-model report uses a calibration cohort to account for censoring in a separate evaluation cohort. Implement the landmark population, competing-event semantics and time-indexed accuracy summaries. /app/repo/survival/score.py loads the two cohort exports. /app/repo/CONTRACT.md defines left limits, support failures, weighting and revision selection. Preserve strata in the outpu…
Excerpt only; not a complete executable task specification. Task ID task_b48d044b6fcc43c7 · version 1. Selection status: not fully recorded.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| Claude Opus 5 · xhigh | 0/0 | 0 / 0 | 3 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
| Tencent HY4 (preview) · think | 0/1 | 1 / 1 | 3 | GOOD_FAILURE: 1 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
14 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
trial_c49a55548cbf41e1Claude Opus 5 · xhigh | Oct 21, 2026 console | 0 · excludedsource_evidence_exclusion | GOOD_FAILURE independent-review | Tokens: 56,590 Turns: 37 Cost: $3.3091 | Not publicly available |
trial_aaefe01d19b44ee0Tencent HY4 (preview) · think | Oct 21, 2026 console | 0 · valid | GOOD_FAILURE independent-review | Tokens: 52,684 Turns: 59 Cost: $3.7950 | Not publicly available |
trial_46eb7d709d5f460fClaude Opus 5 · xhigh | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
trial_6cb74f3983144e69Tencent HY4 (preview) · think | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
trial_0031bf109db547ebClaude Opus 5 · xhigh | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
trial_53eea1e4565c4c11Tencent HY4 (preview) · think | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
trial_86127a458e964f53Claude Opus 5 · xhigh | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
trial_4b6b624d492d4c2fTencent HY4 (preview) · think | Oct 21, 2026 console | No reward · excludedno_reward | HARNESS_ERROR independent-review | Tokens: — Turns: 0 Cost: — | Not publicly available |
landmark-competing-risk-scorecard-oracle-1-d184b510oracle@@ | Oct 21, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
landmark-competing-risk-scorecard-nop-6df447efnop@@ | Oct 21, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
landmark-competing-risk-scorecard-oracle-2-3efff38boracle@@ | Oct 21, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
landmark-competing-risk-scorecard-oracle-1-0575b833oracle@@ | Oct 21, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
landmark-competing-risk-scorecard-nop-f6135a96nop@@ | Oct 21, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
landmark-competing-risk-scorecard-oracle-2-97667823oracle@@ | Oct 21, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |