Task gallery / Data processing / data science / Version 1
Task evidence / delivery
datasciresearch/pyfixest-picchetti-ri
uncalibratedKnown curated cohortChosen benchmark version- Reference solution (oracle)
- Pass
- No-op resistance check
- Pass
- QA verdict
- in-house design (no vendor PR); see docs/reports/datasci-research-v5
- Aggregate eligibility
- Eligible version
The task
Randomization test with an autoregressive instrument Users of pyfixest run local projections with instruments such as weather, cloud cover or commodity prices, which are persistent over time. Picchetti (2026, arXiv:2610.19075) proposes randomization tests for short panels in which the instrument follows a first-order autoregression with unknown persistence and only its innovations are as good as randomly assigned. T…
Excerpt only; not a complete executable task specification. Task ID sfdata2/pyfixest-picchetti-ri · version 1. Selection status: known curated delivery cohort.
Results by model and effort
This exact task version only. Every rate is passes / valid runs with a Wilson 95% interval; n≤2 remains counts only. Macro aggregation does not apply within a single task. Baselines are summarized in the quality gates, not ranked. A passed no-op resistance check means that doing nothing did not solve the task.
Current evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|---|---|---|---|
| GPT-6 Astra · medium | 2/2 | 0 / 2 | 2 | GOOD_FAILURE: 0 BAD_FAILURE: 0 GOOD_SUCCESS: 0 BAD_SUCCESS: 0 |
Legacy evidence
| Model / effort | Verifier result [95% Wilson] | Diagnostic labels / valid | HARNESS_ERROR · excluded | Outcome classes |
|---|
Every recorded run
28 recorded trials, including controls. Diagnostic labels do not replace verifier reward. Labels and quality checks are source declarations, not independently attested here. Internal evidence remains private; a missing public URL is shown explicitly. Records with invalid or absent rewards are not scored.
| Run / model | Date / source | Reward / eligibility | Diagnostic class / source | Resources | Evidence |
|---|---|---|---|---|---|
pyfixest-picchetti-ri__3A89VBpnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__D6LQQYqnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__PPGRgvworacle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__XqL9jFNoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__Z9msopsoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__evoHLVtnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__3FhzBmboracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__Ew62dAQoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__GAhMQKRnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__dxZDuWonop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__r88itvyoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__sHdXGWrnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__CBqXr7jGPT-6 Astra · medium | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: 22,266 Turns: 40 Cost: — | Not publicly available |
pyfixest-picchetti-ri__xq4S5G9GPT-6 Astra · medium | Oct 18, 2026 delivery | No reward · excludedno_reward | HARNESS_ERROR derived | Tokens: 6,144 Turns: 7 Cost: — | Not publicly available |
pyfixest-picchetti-ri__URrnazMnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__YLUgvSUoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__c5Ep57Bnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__hGsLRH4nop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__rq9Uy6goracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__sACPma5oracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__5oSBAvHoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__9X6EBaBoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__sevxeXknop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__tMvNdCpnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__uF8jZDWnop@@ | Oct 18, 2026 delivery | 0 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__zrEzKJWoracle@@ | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: — Turns: — Cost: — | Not publicly available |
pyfixest-picchetti-ri__Fdi2RJJGPT-6 Astra · medium | Oct 18, 2026 delivery | 1 · valid | No diagnostic label derived | Tokens: 15,555 Turns: 26 Cost: — | Not publicly available |
pyfixest-picchetti-ri__knfBNFaGPT-6 Astra · medium | Oct 18, 2026 delivery | No reward · excludedno_reward | HARNESS_ERROR derived | Tokens: 15,868 Turns: 27 Cost: — | Not publicly available |