tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.Explore app structure
ls -la && echo "---SUBMISSION---" && ls -la submission/ && echo "---DATA---" && ls -la data/ && echo "---run_checks---" && wc -l run_checks.py test.sh
total 117 drwxr-xr-x 4 user user 4096 Jun 30 23:56 . drwxr-xr-x 21 root root 4096 Jun 30 23:56 .. -rw-r--r-- 1 user user 15364 Jun 30 23:56 PROMPT.txt -rw-r--r-- 1 user user 5 Jun 30 23:56 agent.pid -rw-r--r-- 1 user user 0 Jun 30 23:56 agent.stderr -rw-r--r-- 1 user user 4609 Jun 30 23:56 agent_trajectory.jsonl drwxr-xr-x 2 user user 60 Jun 30 23:56 data -rw-r--r-- 1 user user 0 Jun 30 23:56 run.log -rwxr-xr-x 1 user user 76773 Jun 30 10:02 run_checks.py drwxr-xr-x 2 user user 128 Jun 30 23:56 submission -rwxr-xr-x 1 user user 804 Jun 30 10:02 test.sh -rwxr-xr-x 1 user user 629 Jun 30 10:02 timer.sh ---SUBMISSION--- total 112 drwxr-xr-x 2 user user 128 Jun 30 23:56 . drwxr-xr-x 4 user user 4096 Jun 30 23:56 .. -rw-r--r-- 1 user user 50472 Jun 30 10:02 autograd.py -rw-r--r-- 1 user user 1661 Jun 30 10:02 data.py -rw-r--r-- 1 user user 33295 Jun 30 10:02 nn.py -rw-r--r-- 1 user user 11509 Jun 30 10:02 optim.py -rw-r--r-- 1 user user 2827 Jun 30 10:02 train.py ---DATA--- total 1900 drwxr-xr-x 2 user user 60 Jun 30 23:56 . drwxr-xr-x 4 user user 4096 Jun 30 23:56 .. -rw-r--r-- 1 user user 1940490 Jun 30 10:02 train.npz ---run_checks--- 1522 run_checks.py 18 test.sh 1540 total
Read test.sh and inspect data
cat test.sh && echo "===TIMER===" && cat timer.sh && echo "===DATA===" && python3 -c "import numpy as np; d=np.load('data/train.npz'); [print(k, d[k].shape, d[k].dtype, d[k].min(), d[k].max()) for k in d.files]"#!/usr/bin/env bash
# Smoke test (NOT the grader): confirms the environment is wired -- numpy imports, the train split
# loads, and the submission stubs are present. It does NOT check correctness (the stubs raise
# NotImplementedError until you implement them).
set -uo pipefail
python3 - <<'PY'
import numpy as np, os
assert os.path.exists("/app/data/train.npz"), "missing train.npz"
d = np.load("/app/data/train.npz")
X, y = d["X"], d["y"]
print(f"train: X={X.shape} {X.dtype}, y={y.shape}, classes={len(set(y.tolist()))}")
for f in ("autograd.py", "nn.py", "optim.py", "data.py", "train.py"):
assert os.path.exists(f"/app/submission/{f}"), f"missing /app/submission/{f}"
print("submission stubs present:", os.listdir("/app/submission"))
print("numpy", np.__version__)
PY
echo "app_test smoke OK"
===TIMER===
#!/bin/bash
# Reports remaining agent time (mirrors task.toml [agent] timeout_sec).
TIMEOUT_SEC=36000
ELAPSED=$(ps -o etimes= -p 1 2>/dev/null | tr -d ' ')
if ! [[ "$ELAPSED" =~ ^[0-9]+$ ]]; then
START_FILE="$(dirname "$0")/.timer_start"
if [ ! -f "$START_FILE" ]; then
date +%s >"$START_FILE"
fi
START_DATE=$(cat "$START_FILE")
NOW=$(date +%s)
ELAPSED=$((NOW - START_DATE))
fi
REMAINING=$((TIMEOUT_SEC - ELAPSED))
if [ $REMAINING -le 0 ]; then
echo "Timer expired!"
else
echo "Remaining time (hours:minutes)":
HOURS=$((REMAINING / 3600))
MINUTES=$(((REMAINING % 3600) / 60))
printf "%d:%02d\n" $HOURS $MINUTES
fi
===DATA===
X (2500, 3, 16, 16) uint8 0 255
y (2500,) in
…[truncated 7 chars]/app/submission/autograd.py
1 """Reverse-mode automatic differentiation over numpy arrays. 2 3 YOU MUST IMPLEMENT EVERYTHING MARKED ``# TODO`` / ``raise NotImplementedError`` using numpy 4 ONLY. No torch/jax/tensorflow/autograd/etc. The grader's conformance suite (a) differentiates the 5 differentiable ops below with central finite differences and compares against YOUR analytic 6 backward, and (b) checks the LSQ ``fake_quant`` STE + scale gradient against the reference 7 definition, so every backward must be correct (a single wrong reduction / STE / Jacobian fails 8 the gradient checks and zeroes the reward). 9 10 A ``Tensor`` wraps one ``numpy.ndarray`` (stored as ``float64`` in ``self.data``) and records the 11 op that produced it so ``.backward()`` can accumulate ``.grad`` into every leaf. 12 13 CONTRACT (the grader builds its own reference against this exact API): 14 * ``Tensor(data, requires_grad=False)`` -- ``self.data`` is ``float64``; ``self.grad`` is None 15 until backward, then a same-shape ``float64`` array. requires_grad leaves accumulate grad. 16 * Every op returns a NEW Tensor whose ``requires_grad`` is the OR of its differentiable inputs, 17 and whose ``_backward`` closure does ``input._accum(local_grad)`` (so multiple paths add up). 18 * ``t.backward(grad=None)`` -- seeds ``grad`` (all-ones for a scalar) and walks the graph in 19 reverse topological order, calling each node's ``_backward``. Repeated backward accumulates. 20 * BROADCASTING: binary ops broadcast like numpy; the ba …[truncated 54775 chars]
/app/submission/nn.py
1 """QAT layers + a small INT2 CNN, built on YOUR autograd engine.
2
3 Implement every ``# TODO`` forward (the parameters + ``named_params``/``buffers`` naming are
4 already wired for you; you compose the autograd ops). The grader checks each layer's forward
5 against its OWN reference AND finite-difference-checks the gradients that flow through your
6 autograd, so the composition must be exactly right.
7
8 CHECKPOINT NAMING CONTRACT (the grader loads YOUR ``qat_model.npz`` into its own reference
9 QuantCNN by these exact keys/shapes -- do not rename):
10 stem.conv.weight (W0,3,3,3) | stem.conv.wq.scale (1,) | stem.bn.gamma/.beta (W0,) | stem.aq.scale (1,)
11 blocks.{i}.conv.weight (Cout,Cin,3,3) | blocks.{i}.conv.wq.scale (1,)
12 blocks.{i}.bn.gamma/.beta (Cout,) | blocks.{i}.aq.scale (1,)
13 head.weight (Cfeat,n_classes) | head.wq.scale (1,) | head.bias (n_classes,)
14 BN running stats are BUFFERS (saved alongside params, NOT trained by the optimizer, NOT audited):
15 stem.bn.running_mean/.running_var (W0,) | blocks.{i}.bn.running_mean/.running_var (Cout,)
16
17 Quantization: every conv/head WEIGHT is LSQ-quantized to a signed 2-bit grid {-2,-1,0,1} via a
18 per-tensor learnable step before use; every block activation (post-relu) is LSQ-quantized to an
19 unsigned 2-bit grid {0,1,2,3}. Conv = 3x3 same-padding cross-correlation. Linear head = x@weight+bias.
20 """
21 from __future__ import annotations
22
23 from typing import Dict, List
24
25 import numpy as n
…[truncated 35167 chars]/app/submission/optim.py
1 """Optimizers over autograd ``Tensor`` parameters.
2
3 Implement the ``# TODO`` update math. Both optimizers update ``param.data`` IN PLACE using
4 ``param.grad``. The grader runs one (and several) steps and compares your updated params against
5 its reference within a tight tolerance, so the math must be exact.
6
7 SGD (with momentum ``mu`` and coupled weight decay ``wd``):
8 g = grad + wd * param; v = mu * v + g; param -= lr * v
9
10 AdamW (DECOUPLED weight decay -- the decay is applied to the PARAM, not folded into the moments):
11 m = b1*m + (1-b1)*g; v = b2*v + (1-b2)*g^2
12 mhat = m / (1 - b1^t); vhat = v / (1 - b2^t) # t = 1-based step count
13 param -= lr * ( mhat / (sqrt(vhat) + eps) + wd * param )
14
15 PARAM GROUPS: both optimizers must accept EITHER a flat param list OR a list of
16 ``{"params": [...], "weight_decay": wd}`` dicts, so weight decay can be applied to SOME params
17 and not others. This matters for low-bit LSQ QAT: the per-tensor quantizer ``scale`` params,
18 BatchNorm gamma/beta, and biases must be EXCLUDED from weight decay. ``_normalize_groups`` and
19 ``split_decay_params`` are provided; honour the per-group weight decay in ``step``.
20 """
21 from __future__ import annotations
22
23 import math
24
25 from typing import Dict, List
26
27 import numpy as np
28
29 from autograd import Tensor
30
31
32 def split_decay_params(model) -> Dict[str, List[Tensor]]:
33 """Partition a model's named params into weight
…[truncated 11129 chars]/app/submission/data.py
1 """Image data loading + normalization + batching.
2
3 The dataset is a procedurally-generated CIFAR-style set: ``X`` is ``uint8`` (N,3,S,S) in [0,255],
4 ``y`` is the integer class label in [0, n_classes). The shipped training split is at
5 ``/app/data/train.npz``; the SEALED held-out split is regenerated by the verifier (never shipped).
6
7 Implement the ``# TODO`` bodies (numpy only). The verifier normalizes the held-out split with the
8 SAME transform, so match it exactly.
9
10 Contract:
11 load_npz(path) -> (X uint8 (N,3,S,S), y int64 (N,)) [provided]
12 normalize(X) -> float64: x = (X/255 - 0.5) / 0.25 [per-channel, the grader's transform]
13 iter_minibatches(Xn, y, batch_size, rng, shuffle=True) -> yields (xb, yb)
14 """
15 from __future__ import annotations
16
17 from typing import Tuple
18
19 import numpy as np
20
21 NORM_MEAN = 0.5
22 NORM_STD = 0.25
23
24
25 def load_npz(path: str) -> Tuple[np.ndarray, np.ndarray]:
26 d = np.load(path)
27 X = np.asarray(d["X"]).astype(np.uint8)
28 y = np.asarray(d["y"]).astype(np.int64).reshape(-1)
29 return X, y
30
31
32 def normalize(X: np.ndarray) -> np.ndarray:
33 """uint8 (N,3,S,S) -> float64, x = (X/255 - NORM_MEAN) / NORM_STD."""
34 # TODO: cast to float, scale to [0,1], subtract NORM_MEAN, divide by NORM_STD.
35 raise NotImplementedError("normalize")
36
37
38 def iter_minibatches(Xn: np.ndarray, y: np.ndarray, batch_size: int,
39 rng: np.rando
…[truncated 281 chars]/app/submission/train.py
1 """Training entry point. Implement the QAT loop, train your INT2 CNN on /app/data/train.npz, and 2 SAVE the checkpoint to /app/submission/qat_model.npz (numpy .npz of the model state_dict: every 3 param + every BN running buffer, by the nn.QuantCNN naming contract). 4 5 The grader (verify.py) will: 6 (1) run the conformance suite over your autograd/nn/optim, 7 (2) BIT-AUDIT every conv/head weight tensor in your checkpoint to <= 4 distinct float values, 8 (3) load your weights into its OWN QuantCNN and score held-out top-1 on a SEALED split. 9 10 So at SAVE time you must BAKE the quantized weights back into each ``weight`` tensor (replace it 11 with round(clip(w/s, -W_NEG, W_POS)) * s using the learned wq.scale) so the saved weights already 12 lie on the 2-bit grid and pass the audit. Reaching the top-1 floor at INT-2 needs a real recipe: 13 sensible LSQ STEP-SIZE INITIALIZATION (a default scale of 1.0 quantizes the small init weights to 14 all-zeros -> a dead network at chance), per-tensor learnable scales, weight decay EXCLUDED from the 15 scale/BN/bias params, gradient clipping, a decaying LR, and enough epochs. Naive post-training 16 rounding or under-training lands at chance. 17 18 This file is a skeleton; the loop body is yours to write. 19 """ 20 from __future__ import annotations 21 22 import argparse 23 import os 24 import sys 25 26 import numpy as np 27 28 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) 29 sys.path.insert(0, "/app") 30 3 …[truncated 1513 chars]
Scan run_checks structure
grep -n "def \|class \|milestone\|tol\|seed\|REF\|reference\|np.random" /app/run_checks.py | head -120
8: python3 /app/run_checks.py --only autograd_grads # run one milestone 17:This runner is a DEBUGGING AID, not the grader. It checks YOUR framework against references it can 33:The REAL grader (the verifier) runs the SAME categories but with its OWN reference framework, 34:DIFFERENT random seeds and shapes, and additional cases. So: 36: held-out cases use different seeds/shapes, so your implementation must be GENERICALLY correct. 37: * The grader ALSO bit-audits + loads your trained qat_model.npz into its own reference QuantCNN 40:reward = 1 from the grader iff (every component milestone fully passes on the held-out cases) AND 60:PUBLIC_SEED_BASE = 1000 # the GRADER uses a DIFFERENT base; do not curve-fit to these seeds. 64:def _load_module(name, path, inject=None): 84:def load_framework(root): 93:def _num_grad_input(forward, x, eps=EPS): 108:def _max_abs(a, b): 112:def _bsum(it): 119:def _imax(a, b): 124:def check_autograd_grads(fw, sb): 130: def case(name, build, shape, seed, upstream=None): 132: rng = np.random.default_rng(sb + seed) 135: def wrapped(t): 193: def two_path(): 195: rng = np.random.default_rng(sb + 41) 204: def loss_grad(name, lossfn, shape, seed, inttgt): 206: rng = np.random.default_rng(sb + seed) 218: def norm_param(name, which, seed): 220: rng = np.random.default_rng(sb + seed) 237: def conv_grad(name, which, seed): 239: rng = np.random.default_rng(sb + seed) 265: …[truncated 4883 chars]
/app/run_checks.py
55 TOL_PROP = 1e-6
56 TOL_OPT = 1e-7
57 TOL_SCHED = 1e-9
58 EPS = 1e-5
59
60 PUBLIC_SEED_BASE = 1000 # the GRADER uses a DIFFERENT base; do not curve-fit to these seeds.
61
62
63 # --------------------------------------------------------------------------- module loading
64 def _load_module(name, path, inject=None):
65 saved = {}
66 if inject:
67 for k, v in inject.items():
68 saved[k] = sys.modules.get(k)
69 sys.modules[k] = v
70 try:
71 spec = importlib.util.spec_from_file_location(name, str(path))
72 mod = importlib.util.module_from_spec(spec)
73 spec.loader.exec_module(mod)
74 return mod
75 finally:
76 if inject:
77 for k, old in saved.items():
78 if old is None:
79 sys.modules.pop(k, None)
80 else:
81 sys.modules[k] = old
82
83
84 def load_framework(root):
85 ag = _load_module("agent_autograd", root / "autograd.py")
86 nn = _load_module("agent_nn", root / "nn.py", inject={"autograd": ag})
87 opt = _load_module("agent_optim", root / "optim.py", inject={"autograd": ag})
88 data = _load_module("agent_data", root / "data.py")
89 return {"autograd": ag, "nn": nn, "optim": opt, "data": data}
90
91
92 # --------------------------------------------------------------------------- numeric helpers
93 def _num_grad_input(forward, x, eps=EPS):
94 g = np.zeros_like(x)
95 flat = x
…[truncated 871 chars]/app/run_checks.py
130 def case(name, build, shape, seed, upstream=None):
131 try:
132 rng = np.random.default_rng(sb + seed)
133 x0 = rng.standard_normal(shape)
134
135 def wrapped(t):
136 o = build(t)
137 return ag.mul(o, T(upstream)) if upstream is not None else o
138 t = T(x0.copy(), requires_grad=True)
139 ag.sum(wrapped(t)).backward()
140 gan = np.asarray(t.grad, np.float64)
141 gnum = _num_grad_input(lambda xv: wrapped(T(xv)).data, x0.copy())
142 err = _max_abs(gan, gnum)
143 results.append((f"grad/{name}", err < TOL_GRAD, f"max_abs={err:.2e}"))
144 except Exception as e: # noqa: BLE001
145 results.append((f"grad/{name}", False, f"raised {type(e).__name__}: {e}"))
146
147 # element-wise binary
148 case("add_bcast", lambda t: ag.add(t, T(np.arange(t.shape[-1], dtype=float) + 1.0)), (4, 5), 1)
149 case("sub_bcast", lambda t: ag.sub(t, T(np.arange(t.shape[-1], dtype=float) + 1.0)), (4, 5), 2)
150 case("mul_bcast", lambda t: ag.mul(t, T(np.arange(t.shape[-1], dtype=float) + 1.0)), (4, 5), 3)
151 case("div", lambda t: ag.div(t, T(np.full((4, 5), 2.0))), (4, 5), 4)
152 case("div_denom", lambda t: ag.div(T(np.full((4, 5), 3.0)), ag.add(t, T(5.0))), (4, 5), 5)
153 case("power2", lambda t: ag.power(t, 2.0), (4, 5), 6)
154 case("power3", lambda t: ag.power(t, 3.0), (4, 5), 7)
155 case("maxi
…[truncated 4100 chars]Find TOL_GRAD and optim check refs
grep -n "TOL_GRAD\s*=\|TOL_GRAD " /app/run_checks.py | head; echo "---NAdam/sched check sections---"; grep -n "nadam\|adamax\|adadelta\|nesterov\|polynomial\|multistep\|cosine_restarts\|linear_lr\|split_decay\|def check_optim\|def check_integration" /app/run_checks.py
54:TOL_GRAD = 2e-3
---NAdam/sched check sections---
1106:def check_optim(fw, sb):
1196: def adamax_step(lr, b1, b2, eps, wd):
1206: def nadam_step(lr, b1, b2, eps, wd):
1218: def adadelta_step(lr, rho, eps, wd):
1230: def nesterov_step(lr, mu, wd):
1240: adamax_step(2e-3, 0.9, 0.99, 1e-8, 0.02), 5, sb + 20)
1242: nadam_step(2e-3, 0.9, 0.999, 1e-8, 0.0), 10, sb + 21)
1244: adadelta_step(1.0, 0.9, 1e-6, 0.01), 5, sb + 22)
1246: nesterov_step(0.05, 0.9, 1e-2), 5, sb + 23)
1277: groups = opt.split_decay_params(m)
1288: results.append(("optim/split_decay_params", ok, ""))
1290: results.append(("optim/split_decay_params", False, f"raised {type(e).__name__}: {e}"))
1331: sched("sched_polynomial", lambda s: opt.polynomial_lr(s, 50, 1e-2, power=2.0, min_lr=1e-4), ref_poly)
1332: sched("sched_linear", lambda s: opt.linear_lr(s, 40, 1e-2, min_lr=1e-4),
1334: sched("sched_multistep", lambda s: opt.multistep_lr(s, 1e-2, (10, 25, 40), gamma=0.5),
1340: sched("sched_cosine_restarts", lambda s: opt.cosine_restarts_lr(s, 15, 1e-2, min_lr=1e-4), ref_cos_restart)
1346:def check_integration(fw, sb):
1365: g = opt.split_decay_params(m)/app/run_checks.py
1106 def check_optim(fw, sb):
1107 ag = fw["autograd"]
1108 opt = fw["optim"]
1109 T = ag.Tensor
1110 results = []
1111
1112 def mk(seed):
1113 rng = np.random.default_rng(seed)
1114 return [T(rng.standard_normal((4, 3)), requires_grad=True), T(rng.standard_normal(3), requires_grad=True)]
1115
1116 def run(name, opt_fn, ref_step, nsteps, gseed):
1117 try:
1118 ps = mk(sb + 500)
1119 ref = [p.data.copy() for p in ps]
1120 o = opt_fn(ps)
1121 st = {}
1122 for s in range(nsteps):
1123 grng = np.random.default_rng(gseed + s)
1124 grads = [grng.standard_normal(p.data.shape) for p in ps]
1125 for p, g in zip(ps, grads):
1126 p.grad = g.copy()
1127 o.step()
1128 ref = [ref_step(i, ref[i], grads[i].copy(), s + 1, st) for i in range(len(ps))]
1129 err = max(_max_abs(p.data, ref[i]) for i, p in enumerate(ps))
1130 results.append((f"optim/{name}", err < TOL_OPT, f"max_abs={err:.1e}"))
1131 except Exception as e: # noqa: BLE001
1132 results.append((f"optim/{name}", False, f"raised {type(e).__name__}: {e}"))
1133
1134 def sgd_step(lr, mu, wd):
1135 def f(i, p, g, t, st):
1136 v = st.get(("v", i), np.zeros_like(p))
1137 g = g + wd * p
1138 v = mu * v + g
1139 st[("v", i)] = v
114
…[truncated 11079 chars]/app/run_checks.py
584 def check_quant_grads(fw, sb):
585 ag = fw["autograd"]
586 results = []
587
588 def grads(v0, s0, Qn, Qp, up):
589 vt = ag.Tensor(v0.copy(), requires_grad=True)
590 st = ag.Tensor(np.array([s0]), requires_grad=True)
591 out = ag.fake_quant(vt, st, Qn, Qp)
592 out.backward(up.copy())
593 return np.asarray(vt.grad, np.float64), np.asarray(st.grad, np.float64), out.data.copy()
594
595 def ste_indicator(name, seed, Qn, Qp, s0):
596 try:
597 rng = np.random.default_rng(sb + seed)
598 v0 = rng.standard_normal((5, 5)) * 2.0
599 up = rng.standard_normal((5, 5))
600 gv, _, _ = grads(v0, s0, Qn, Qp, up)
601 r = v0 / s0
602 middle = (r >= -Qn) & (r <= Qp)
603 expected = up * middle
604 ok = _max_abs(gv, expected) < 1e-9 and np.all(gv[~middle] == 0.0)
605 results.append((f"quant/{name}", ok, f"err={_max_abs(gv, expected):.1e}"))
606 except Exception as e: # noqa: BLE001
607 results.append((f"quant/{name}", False, f"raised {type(e).__name__}: {e}"))
608 ste_indicator("ste_indicator_w", 60, 2, 1, 0.7)
609 ste_indicator("ste_indicator_a", 61, 0, 3, 0.5)
610
611 def scale_clip(name, seed, Qn, Qp, s0, fill, bound):
612 try:
613 v0 = np.full((4, 4), fill)
614 up = np.ones((4, 4))
615 _, gs, _ = grads(v0, s0, Qn, Qp, up)
616 gscale
…[truncated 10053 chars]/app/run_checks.py
1345 # ======================================================================= integration
1346 def check_integration(fw, sb):
1347 ag = fw["autograd"]
1348 nn = fw["nn"]
1349 opt = fw["optim"]
1350 results = []
1351
1352 def init_scales(m):
1353 nps = m.named_params()
1354 for nm, p in nps.items():
1355 if nm.endswith("wq.scale"):
1356 wkey = nm[:-len("wq.scale")] + "weight"
1357 p.data[...] = _imax(2.0 * float(np.mean(np.abs(nps[wkey].data))), 1e-2)
1358 elif nm.endswith("aq.scale"):
1359 p.data[...] = 1.0
1360
1361 def overfit_loss(m, opt_build, X, y, nsteps):
1362 # repeatedly train on ONE fixed (X, y) batch: a correct forward+backward+optimizer drives
1363 # the loss DOWN on a memorizable batch. A broken backward keeps it flat/NaN.
1364 m.set_training(True)
1365 g = opt.split_decay_params(m)
1366 o = opt_build(g)
1367 losses = []
1368 for _ in range(nsteps):
1369 loss = ag.cross_entropy(m(X), y)
1370 o.zero_grad(); loss.backward(); opt.clip_grad_norm(m.params(), 2.0); o.step()
1371 losses.append(float(loss.data))
1372 return losses
1373
1374 def qcnn_run():
1375 m = nn.QuantCNN(6, widths=(8, 16), seed=sb + 5)
1376 init_scales(m)
1377 rng = np.random.default_rng(sb)
1378 X = rng.standard_normal((8, 3, 16, 16)); y = rng.intege
…[truncated 4701 chars]/app/run_checks.py
880 def fakequant_grid_w():
881 x = np.random.default_rng(sb + 21).standard_normal((6, 6)) * 2.0
882 s = 0.5
883 out = ag.fake_quant(T(x), T([s]), 2, 1).data
884 levels = np.round(out / s)
885 in_grid = np.all((levels >= -2) & (levels <= 1))
886 on_grid = _max_abs(out, levels * s) < 1e-9
887 return bool(in_grid and on_grid), f"unique={np.unique(out).size}"
888
889 def fakequant_grid_a():
890 x = np.random.default_rng(sb + 22).standard_normal((6, 6)) * 2.0
891 s = 0.4
892 out = ag.fake_quant(T(x), T([s]), 0, 3).data
893 levels = np.round(out / s)
894 return bool(np.all((levels >= 0) & (levels <= 3))), f"unique={np.unique(out).size}"
895
896 def linear_def():
897 rng = np.random.default_rng(sb + 23)
898 lin = nn.Linear(5, 3, np.random.default_rng(sb + 90))
899 x = rng.standard_normal((4, 5))
900 return _max_abs(lin(T(x)).data, x @ lin.weight.data + lin.bias.data) < 1e-6, ""
901
902 def mlp_shape():
903 y = nn.MLP(8, 16, np.random.default_rng(sb + 91))(T(np.random.default_rng(sb + 24).standard_normal((2, 3, 8)))).data
904 return y.shape == (2, 3, 8), f"shape={y.shape}"
905
906 def qcnn_shape():
907 m = nn.QuantCNN(10, widths=(8, 16), seed=sb + 25)
908 m.set_training(False)
909 return m(np.random.default_rng(sb + 26).standard_normal((3, 3, 16, 16))).data.shape == (3, 10), ""
910
911 def
…[truncated 10817 chars]/app/submission/autograd.py
return grad.reshape(shape) class Tensor:
return grad.reshape(shape)
def _expand(g, axis, keepdims, shape):
"""Broadcast a reduced gradient ``g`` back to ``shape`` (re-inserting reduced axes if the
reduction was done with keepdims=False)."""
g = np.asarray(g, dtype=np.float64)
if (not keepdims) and (axis is not None):
g = np.expand_dims(g, axis)
return np.broadcast_to(g, shape).astype(np.float64).copy()
def _reduce_count(shape, axis):
if axis is None:
n = 1
for s in shape:
n *= s
return n
axes = axis if isinstance(axis, tuple) else (axis,)
n = 1
for ax in axes:
n *= shape[ax]
return n
class Tensor:The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
TODO: implement the topological walk + seeding. (The per-op ``_backward`` closures do the
local accumulation; here you only orchestrate the order + seed ``self.grad``.)
"""
raise NotImplementedError("Tensor.backward") TODO: implement the topological walk + seeding. (The per-op ``_backward`` closures do the
local accumulation; here you only orchestrate the order + seed ``self.grad``.)
"""
topo: List[Tensor] = []
visited = set()
def build(t: "Tensor"):
if id(t) in visited:
return
visited.add(id(t))
for p in t._parents:
build(p)
topo.append(t)
build(self)
if grad is None:
seed = np.ones_like(self.data)
else:
seed = _as_f64(grad)
self._accum(seed)
for t in reversed(topo):
t._backward()The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("add backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sub backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(-g, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mul backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g * b.data, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * a.data, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("div backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g / b.data, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(-g * a.data / (b.data * b.data), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("power backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * p * a.data ** (p - 1))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("maximum backward") # TODO def _bw():
g = out.grad
am = (a.data > b.data).astype(np.float64) + 0.5 * (a.data == b.data)
if a.requires_grad:
a._accum(_unbroadcast(g * am, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * (1.0 - am), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("minimum backward") # TODO def _bw():
g = out.grad
am = (a.data < b.data).astype(np.float64) + 0.5 * (a.data == b.data)
if a.requires_grad:
a._accum(_unbroadcast(g * am, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * (1.0 - am), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("where backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g * c, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * (~c), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("exp backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * out.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("log backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad / a.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sqrt backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * 0.5 / out.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("abs backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * np.sign(a.data))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("clip backward") # TODO def _bw():
if a.requires_grad:
mask = (a.data > lo) & (a.data < hi)
a._accum(out.grad * mask)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sum backward") # TODO def _bw():
if a.requires_grad:
a._accum(_expand(out.grad, axis, keepdims, a.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mean backward") # TODO def _bw():
if a.requires_grad:
n = _reduce_count(a.data.shape, axis)
a._accum(_expand(out.grad, axis, keepdims, a.data.shape) / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("max backward") # TODO def _bw():
if a.requires_grad:
outd = _expand(out.data, axis, keepdims, a.data.shape)
mask = (a.data == outd).astype(np.float64)
counts = mask.sum(axis=axis, keepdims=True)
g = _expand(out.grad, axis, keepdims, a.data.shape)
a._accum(g * mask / counts)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("min backward") # TODO def _bw():
if a.requires_grad:
outd = _expand(out.data, axis, keepdims, a.data.shape)
mask = (a.data == outd).astype(np.float64)
counts = mask.sum(axis=axis, keepdims=True)
g = _expand(out.grad, axis, keepdims, a.data.shape)
a._accum(g * mask / counts)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("concat backward") # TODO def _bw():
g = out.grad
ax = axis if axis >= 0 else axis + g.ndim
offset = 0
for t in ts:
sz = t.data.shape[ax]
sl = [slice(None)] * g.ndim
sl[ax] = slice(offset, offset + sz)
if t.requires_grad:
t._accum(g[tuple(sl)])
offset += szThe file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("stack backward") # TODO def _bw():
g = out.grad
ax = axis if axis >= 0 else axis + g.ndim
for i, t in enumerate(ts):
if t.requires_grad:
sl = [slice(None)] * g.ndim
sl[ax] = i
t._accum(g[tuple(sl)])The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("transpose backward") # TODO def _bw():
if a.requires_grad:
if axes is None:
a._accum(np.transpose(out.grad))
else:
inv = np.argsort(list(axes))
a._accum(np.transpose(out.grad, inv))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("reshape backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad.reshape(a.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("getitem backward") # TODO def _bw():
if a.requires_grad:
gz = np.zeros_like(a.data)
np.add.at(gz, idx, out.grad)
a._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("matmul backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
da = g @ np.swapaxes(b.data, -1, -2)
a._accum(_unbroadcast(da, a.data.shape))
if b.requires_grad:
db = np.swapaxes(a.data, -1, -2) @ g
b._accum(_unbroadcast(db, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("relu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (a.data > 0.0))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("leaky_relu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * np.where(a.data > 0.0, 1.0, slope))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sigmoid backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * out.data * (1.0 - out.data))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("tanh backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (1.0 - out.data * out.data))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("gelu backward") # TODO def _bw():
if a.requires_grad:
pdf = np.exp(-0.5 * a.data * a.data) / np.sqrt(2.0 * np.pi)
a._accum(out.grad * (cdf + a.data * pdf))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("softmax backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
da = s * (g - (g * s).sum(axis=axis, keepdims=True))
a._accum(da)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("log_softmax backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
sm = np.exp(out.data)
a._accum(g - sm * g.sum(axis=axis, keepdims=True))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("cross_entropy backward") # TODO def _bw():
if logits.requires_grad:
sm = np.exp(logp)
oh = np.zeros_like(sm)
oh[np.arange(n), t] = 1.0
logits._accum(out.grad * (sm - oh) / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mse_loss backward") # TODO def _bw():
if pred.requires_grad:
n = pred.data.size
pred._accum(out.grad * (2.0 / n) * (pred.data - tgt))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("layernorm backward") # TODO def _bw():
g = out.grad
axes = tuple(range(g.ndim - 1))
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=axes))
if beta.requires_grad:
beta._accum(g.sum(axis=axes))
if x.requires_grad:
gx = g * gamma.data
dx = inv / D * (D * gx - gx.sum(axis=-1, keepdims=True)
- xhat * (gx * xhat).sum(axis=-1, keepdims=True))
x._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("conv2d backward") # TODO (dW, dbias, dx via _col2im) def _bw():
dout = out.grad.reshape(N, Cout, OH * OW)
if weight.requires_grad:
dW = np.einsum("nop,ncp->oc", dout, cols).reshape(Cout, Cin, KH, KW)
weight._accum(dW)
if has_bias and bias.requires_grad:
bias._accum(dout.sum(axis=(0, 2)))
if x.requires_grad:
dcols = np.einsum("oc,nop->ncp", Wm, dout)
dxp = _col2im(dcols, xp.shape, KH, KW, stride, OH, OW)
if pad > 0:
dxp = dxp[:, :, pad:xp.shape[2] - pad, pad:xp.shape[3] - pad]
x._accum(dxp)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("avgpool2d backward") # TODO def _bw():
if x.requires_grad:
g = out.grad / (k * k)
dexp = np.broadcast_to(g[:, :, :, None, :, None], (N, C, OH, k, OW, k))
dx = np.zeros_like(x.data)
dx[:, :, :OH * k, :OW * k] = dexp.reshape(N, C, OH * k, OW * k)
x._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("maxpool2d backward") # TODO def _bw():
if x.requires_grad:
xr = x.data[:, :, :OH * k, :OW * k].reshape(N, C, OH, k, OW, k)
outb = out.data[:, :, :, None, :, None]
mask = (xr == outb).astype(np.float64)
counts = mask.sum(axis=(3, 5), keepdims=True)
g = out.grad[:, :, :, None, :, None]
dexp = mask / counts * g
dx = np.zeros_like(x.data)
dx[:, :, :OH * k, :OW * k] = dexp.reshape(N, C, OH * k, OW * k)
x._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("batchnorm2d backward") # TODO def _bw():
g = out.grad
ax = (0, 2, 3)
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=ax))
if beta.requires_grad:
beta._accum(g.sum(axis=ax))
if x.requires_grad:
gx = g * g_
if training:
sum_gx = gx.sum(axis=ax, keepdims=True)
sum_gxxhat = (gx * xhat).sum(axis=ax, keepdims=True)
dx = inv / M * (M * gx - sum_gx - xhat * sum_gxxhat)
else:
dx = gx * inv
x._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("fake_quant backward (STE + LSQ scale gradient)") # TODO def _bw():
mask = (r >= Qn) & (r <= Qp)
if v.requires_grad:
v._accum(out.grad * mask)
if scale.requires_grad:
d = np.where(r < Qn, Qn, np.where(r > Qp, Qp, np.round(r) - r))
ds = float((out.grad * d).sum()) * grad_scale
scale._accum(np.asarray(ds, dtype=np.float64).reshape(scale.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("square backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * 2.0 * a.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("rsqrt backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (-0.5) * a.data ** (-1.5))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("reciprocal backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (-1.0) / (a.data * a.data))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("var backward") # TODO def _bw():
if a.requires_grad:
n = _reduce_count(a.data.shape, axis)
g = _expand(out.grad, axis, keepdims, a.data.shape)
a._accum(g * (2.0 / n) * xc)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("std backward") # TODO def _bw():
if a.requires_grad:
n = _reduce_count(a.data.shape, axis)
g = _expand(out.grad, axis, keepdims, a.data.shape)
a._accum(g * xc / (n * sd))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("gather backward") # TODO def _bw():
if a.requires_grad:
gz = np.zeros_like(a.data)
np.put_along_axis(gz, idx, 0.0, axis=axis)
np.add.at(gz, np.ogrid_placeholder if False else Ellipsis, 0) # noop
# scatter-add via take/put with accumulation
_scatter_add_along_axis(gz, idx, out.grad, axis)
a._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
if a.requires_grad:
gz = np.zeros_like(a.data)
np.put_along_axis(gz, idx, 0.0, axis=axis)
np.add.at(gz, np.ogrid_placeholder if False else Ellipsis, 0) # noop
# scatter-add via take/put with accumulation
_scatter_add_along_axis(gz, idx, out.grad, axis)
a._accum(gz) def _bw():
if a.requires_grad:
gz = np.zeros_like(a.data)
ax = axis % gz.ndim
grids = np.indices(idx.shape)
idxs = [grids[d] for d in range(gz.ndim)]
idxs[ax] = idx
np.add.at(gz, tuple(idxs), out.grad)
a._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("pad2d backward") # TODO def _bw():
if x.requires_grad:
g = out.grad
if pad > 0:
g = g[:, :, pad:g.shape[2] - pad, pad:g.shape[3] - pad]
x._accum(g)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("softplus backward") # TODO def _bw():
if a.requires_grad:
with np.errstate(over="ignore"):
sig = 1.0 / (1.0 + np.exp(-bx))
a._accum(out.grad * sig)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("silu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (sig + a.data * sig * (1.0 - sig)))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mish backward") # TODO def _bw():
if a.requires_grad:
th = np.tanh(sp)
with np.errstate(over="ignore"):
sig = 1.0 / (1.0 + np.exp(-x))
a._accum(out.grad * (th + x * (1.0 - th * th) * sig))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("elu backward") # TODO def _bw():
if a.requires_grad:
grad_local = np.where(x > 0.0, 1.0, alpha * np.exp(np.minimum(x, 0.0)))
a._accum(out.grad * grad_local)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("hardtanh backward") # TODO def _bw():
if a.requires_grad:
mask = (a.data > lo) & (a.data < hi)
a._accum(out.grad * mask)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("hardsigmoid backward") # TODO def _bw():
if a.requires_grad:
mask = (z > 0.0) & (z < 1.0)
a._accum(out.grad * mask / 6.0)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("groupnorm backward") # TODO def _bw():
g = out.grad
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=(0, 2, 3)))
if beta.requires_grad:
beta._accum(g.sum(axis=(0, 2, 3)))
if x.requires_grad:
M = cg * H * W
gx = g * gamma.data.reshape(1, C, 1, 1)
gxg = gx.reshape(N, G, M)
xhatg = xhat.reshape(N, G, M)
sum_gx = gxg.sum(axis=2, keepdims=True)
sum_gxxhat = (gxg * xhatg).sum(axis=2, keepdims=True)
dxg = inv / M * (M * gxg - sum_gx - xhatg * sum_gxxhat)
x._accum(dxg.reshape(N, C, H, W))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("fake_quant_per_channel backward (STE + per-channel scale grad)") # TODO def _bw():
mask = (r >= Qn) & (r <= Qp)
if v.requires_grad:
v._accum(out.grad * mask)
if scale.requires_grad:
d = np.where(r < Qn, Qn, np.where(r > Qp, Qp, np.round(r) - r))
contrib = out.grad * d
axc = axis % v.data.ndim
axes = tuple(i for i in range(v.data.ndim) if i != axc)
ds = contrib.sum(axis=axes) * grad_scale
scale._accum(ds.reshape(scale.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("fake_quant_affine backward (STE + scale grad on shifted grid)") # TODO def _bw():
lower = r < Qn
upper = r > Qp
middle = ~(lower | upper)
if v.requires_grad:
v._accum(out.grad * middle)
if scale.requires_grad:
d = np.where(middle, (q - z) - (r - z), np.where(lower, Qn - z, Qp - z))
ds = float((out.grad * d).sum()) * grad_scale
scale._accum(np.asarray(ds, dtype=np.float64).reshape(scale.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("cumsum backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
a._accum(np.flip(np.cumsum(np.flip(g, axis=axis), axis=axis), axis=axis))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("flip backward") # TODO def _bw():
if a.requires_grad:
a._accum(np.flip(out.grad, axis=axis))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("logsumexp backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
if not keepdims:
g = np.expand_dims(g, axis)
a._accum(sm * g)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("logaddexp backward") # TODO def _bw():
g = out.grad
wa = np.exp(a.data - out_data)
wb = np.exp(b.data - out_data)
if a.requires_grad:
a._accum(_unbroadcast(g * wa, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * wb, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("l2_normalize backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
yg = (y * g).sum(axis=axis, keepdims=True)
a._accum((g - y * yg) / nrm)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("rms_norm backward") # TODO def _bw():
g = out.grad
if gamma.requires_grad:
axes = tuple(range(g.ndim - 1))
gamma._accum((g * xhat).sum(axis=axes))
if x.requires_grad:
gg = g * gamma.data
s = (gg * xd).sum(axis=-1, keepdims=True)
dx = inv * gg - (xd * inv ** 3 / D) * s
x._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("instance_norm backward") # TODO def _bw():
g = out.grad
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=(0, 2, 3)))
if beta.requires_grad:
beta._accum(g.sum(axis=(0, 2, 3)))
if x.requires_grad:
gx = g * gamma.data.reshape(1, C, 1, 1)
gxg = gx.reshape(N, C, M)
xhatg = xhat.reshape(N, C, M)
sum_gx = gxg.sum(axis=2, keepdims=True)
sum_gxxhat = (gxg * xhatg).sum(axis=2, keepdims=True)
dxg = inv / M * (M * gxg - sum_gx - xhatg * sum_gxxhat)
x._accum(dxg.reshape(N, C, H, W))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("huber_loss backward") # TODO def _bw():
if pred.requires_grad:
local = np.where(quad, diff, delta * np.sign(diff))
pred._accum(out.grad * local / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("kl_div backward") # TODO def _bw():
if log_p.requires_grad:
log_p._accum(out.grad * (-q) / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("embedding backward") # TODO def _bw():
if weight.requires_grad:
gz = np.zeros_like(weight.data)
np.add.at(gz, idx, out.grad)
weight._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("conv2d_gen backward (grouped/dilated dW/db/dx)") # TODO def _bw():
dout = out.grad.reshape(N, Cout, OH * OW)
dout_g = dout.reshape(N, groups, cog, OH * OW)
if weight.requires_grad:
dW = np.einsum("ngop,ngcp->goc", dout_g, cols_g).reshape(Cout, cig, KH, KW)
weight._accum(dW)
if has_bias and bias.requires_grad:
bias._accum(dout.sum(axis=(0, 2)))
if x.requires_grad:
dcols_g = np.einsum("goc,ngop->ngcp", Wm, dout_g)
dcols = dcols_g.reshape(N, Cin * KH * KW, OH * OW)
dxp = _col2im_dil(dcols, xp.shape, KH, KW, stride, dilation, OH, OW)
if pad > 0:
dxp = dxp[:, :, pad:xp.shape[2] - pad, pad:xp.shape[3] - pad]
x._accum(dxp)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("conv_transpose2d backward") # TODO def _bw():
g = out.grad
if has_bias and bias.requires_grad:
bias._accum(g.sum(axis=(0, 2, 3)))
if pad > 0:
gfull = np.zeros((N, Cout, OHf, OWf), dtype=np.float64)
gfull[:, :, pad:OHf - pad, pad:OWf - pad] = g
else:
gfull = g
if x.requires_grad or weight.requires_grad:
gcontrib = np.zeros((N, Cout, H, W, KH, KW), dtype=np.float64)
for i in range(KH):
for j in range(KW):
gcontrib[:, :, :, :, i, j] = gfull[:, :, i:i + stride * H:stride, j:j + stride * W:stride]
if x.requires_grad:
x._accum(np.einsum("noijKL,coKL->ncij", gcontrib, Wm))
if weight.requires_grad:
weight._accum(np.einsum("noijKL,ncij->coKL", gcontrib, xd))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("avgpool2d_s backward") # TODO def _bw():
if x.requires_grad:
g = out.grad
dxp = np.zeros((N, C, Hp, Wp), dtype=np.float64)
for oi in range(OH):
for oj in range(OW):
dxp[:, :, oi * stride:oi * stride + k, oj * stride:oj * stride + k] += \
(g[:, :, oi, oj] / (k * k))[:, :, None, None]
if pad > 0:
dxp = dxp[:, :, pad:Hp - pad, pad:Wp - pad]
x._accum(dxp)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("maxpool2d_s backward") # TODO def _bw():
if x.requires_grad:
g = out.grad
dxp = np.zeros((N, C, Hp, Wp), dtype=np.float64)
nidx = np.arange(N)[:, None]
cidx = np.arange(C)[None, :]
for oi in range(OH):
for oj in range(OW):
rows = oi * stride + argi[:, :, oi, oj]
cols = oj * stride + argj[:, :, oi, oj]
np.add.at(dxp, (nidx, cidx, rows, cols), g[:, :, oi, oj])
if pad > 0:
dxp = dxp[:, :, pad:Hp - pad, pad:Wp - pad]
x._accum(dxp)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("fake_quant_lsq_plus backward (STE v-grad + beta-grad + LSQ scale grad)") # TODO def _bw():
mid = (r >= Qn) & (r <= Qp)
if v.requires_grad:
v._accum(out.grad * mid)
if beta.requires_grad:
db = float((out.grad * (~mid)).sum())
beta._accum(np.asarray(db, dtype=np.float64).reshape(beta.data.shape))
if scale.requires_grad:
d = np.where(mid, q - r, np.where(r < Qn, Qn, Qp))
ds = float((out.grad * d).sum()) * grad_scale
scale._accum(np.asarray(ds, dtype=np.float64).reshape(scale.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
Check autograd complete and imports
cd /app/submission && grep -n "NotImplementedError\|# TODO" autograd.py; echo "---import test---"; python3 -c "import autograd; print('autograd OK')"3:YOU MUST IMPLEMENT EVERYTHING MARKED ``# TODO`` / ``raise NotImplementedError`` using numpy ---import test--- autograd OK
/app/submission/data.py
# TODO: cast to float, scale to [0,1], subtract NORM_MEAN, divide by NORM_STD.
raise NotImplementedError("normalize")return (np.asarray(X, dtype=np.float64) / 255.0 - NORM_MEAN) / NORM_STD
The file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/data.py
# TODO: optionally shuffle an index array, then yield contiguous batch_size slices of it.
raise NotImplementedError("iter_minibatches") n = Xn.shape[0]
idx = np.arange(n)
if shuffle:
rng.shuffle(idx)
for start in range(0, n, batch_size):
sel = idx[start:start + batch_size]
yield Xn[sel], y[sel]The file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
"""Optimizers over autograd ``Tensor`` parameters (numpy only)."""
from __future__ import annotations
import math
from typing import Dict, List
import numpy as np
from autograd import Tensor
def split_decay_params(model) -> Dict[str, List[Tensor]]:
"""Partition a model's named params into weight-decay vs no-decay groups.
DECAY = conv/head ``weight`` tensors; NO-DECAY = ``scale`` params, BN gamma/beta, biases."""
decay, no_decay = [], []
for name, p in model.named_params().items():
lname = name.lower()
if lname.endswith("scale") or "bn." in lname or lname.endswith("gamma") \
or lname.endswith("beta") or lname.endswith("bias"):
no_decay.append(p)
else:
decay.append(p)
return {"decay": decay, "no_decay": no_decay}
def _normalize_groups(params):
"""Accept a flat param list (single group, optimizer-default wd) OR a list of
{'params': [...], 'weight_decay': wd} dicts. Returns [(params, wd_override_or_None), ...]."""
plist = list(params)
if plist and isinstance(plist[0], dict):
return [(list(g["params"]), g.get("weight_decay", None)) for g in plist]
return [(plist, None)]
class Optimizer:
def __init__(self, params):
self._groups = _normalize_groups(params)
self.params: List[Tensor] = [p for g, _ in self._groups for p in g]
def zero_grad(self):
for p in self.params:
p.grad = None
def _iter(self):
"""Yield (
…[truncated 9376 chars]The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: y = x @ weight (+ bias). Use ag.matmul / ag.add.
raise NotImplementedError("Linear.forward") y = ag.matmul(x, self.weight)
if self.bias is not None:
y = ag.add(y, self.bias)
return yThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: out = x + fc2(gelu(fc1(x)))
raise NotImplementedError("MLP.forward")return ag.add(x, self.fc2(ag.gelu(self.fc1(x))))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.layernorm(x, self.gamma, self.beta, self.eps)
raise NotImplementedError("LayerNorm.forward")return ag.layernorm(x, self.gamma, self.beta, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.conv2d(x, self.weight, bias=self.bias, stride=self.stride, pad=self.padding)
raise NotImplementedError("Conv2d.forward")return ag.conv2d(x, self.weight, bias=self.bias, stride=self.stride, pad=self.padding)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.maxpool2d(x, self.k)
raise NotImplementedError("MaxPool2d.forward")return ag.maxpool2d(x, self.k)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.avgpool2d(x, self.k)
raise NotImplementedError("AvgPool2d.forward")return ag.avgpool2d(x, self.k)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.fake_quant(w, self.scale, W_NEG, W_POS)
raise NotImplementedError("WeightQuant.quant")return ag.fake_quant(w, self.scale, W_NEG, W_POS)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.fake_quant(x, self.scale, A_NEG, A_POS)
raise NotImplementedError("ActQuant.forward")return ag.fake_quant(x, self.scale, A_NEG, A_POS)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: ag.batchnorm2d(x, gamma, beta, eps, running_mean, running_var, momentum, training)
raise NotImplementedError("BatchNorm2d.forward") return ag.batchnorm2d(x, self.gamma, self.beta, self.eps,
running_mean=self.running_mean, running_var=self.running_var,
momentum=self.momentum, training=self.training)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
Check batchnorm1d existence
grep -n "batchnorm1d\|def batchnorm" autograd.py
850:def batchnorm2d(x, gamma, beta, eps: float = 1e-5,
/app/run_checks.py
281 def bn_grad(name, dim, which, seed):
282 try:
283 rng = np.random.default_rng(sb + seed)
284 C = 3 if dim == 2 else 4
285 shape = (4, C, 5, 5) if dim == 2 else (6, C)
286 op = ag.batchnorm2d if dim == 2 else ag.batchnorm1d
287 x = rng.standard_normal(shape)
288 g0 = rng.standard_normal(C) * 0.5 + 1.0
289 b0 = rng.standard_normal(C) * 0.3
290 if which == "x":
291 xt = T(x.copy(), requires_grad=True)
292 ag.sum(op(xt, T(g0), T(b0), training=True)).backward()
293 gnum = _num_grad_input(lambda xv: op(T(xv), T(g0), T(b0), training=True).data, x.copy())
294 gan = xt.grad
295 elif which == "gamma":
296 gt = T(g0.copy(), requires_grad=True)
297 ag.sum(op(T(x), gt, T(b0), training=True)).backward()
298 gnum = _num_grad_input(lambda gv: op(T(x), T(gv), T(b0), training=True).data, g0.copy())
299 gan = gt.grad
300 else:
301 bt = T(b0.copy(), requires_grad=True)
302 ag.sum(op(T(x), T(g0), bt, training=True)).backward()
303 gnum = _num_grad_input(lambda bv: op(T(x), T(g0), T(bv), training=True).data, b0.copy())
304 gan = bt.grad
305 results.append((f"grad/{name}", _max_abs(np.asarray(gan), gnum) < TOL_GRAD, ""))
306 except Exception as e: # noqa:
…[truncated 681 chars]/app/submission/autograd.py
# ============================================================ LSQ fake-quant (DOMAIN; the crux) def fake_quant(v, scale, n_levels_neg: int, n_levels_pos: int,
def batchnorm1d(x, gamma, beta, eps: float = 1e-5,
running_mean=None, running_var=None, momentum: float = 0.1,
training: bool = True) -> Tensor:
"""BatchNorm over the batch axis N per feature C. x:(N,C); gamma,beta:(C,). Same convention as
batchnorm2d (population variance, unbiased running var)."""
x, gamma, beta = _ensure(x), _ensure(gamma), _ensure(beta)
N, C = x.data.shape
M = N
xd = x.data
if training:
mu = xd.mean(axis=0, keepdims=True)
xc = xd - mu
var = (xc * xc).mean(axis=0, keepdims=True)
inv = 1.0 / np.sqrt(var + eps)
xhat = xc * inv
if running_mean is not None and running_var is not None:
ubvar = var * (M / _bi_max(1, M - 1))
running_mean[:] = (1 - momentum) * running_mean + momentum * mu.reshape(-1)
running_var[:] = (1 - momentum) * running_var + momentum * ubvar.reshape(-1)
else:
mu = running_mean.reshape(1, C)
var = running_var.reshape(1, C)
inv = 1.0 / np.sqrt(var + eps)
xhat = (xd - mu) * inv
g_ = gamma.data.reshape(1, C)
out = _mk(g_ * xhat + beta.data.reshape(1, C), (x, gamma, beta),
x.requires_grad or gamma.requires_grad or beta.requires_grad)
def _bw():
g = out.grad
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=0))
if beta.requires_grad:
beta._accum(g.sum(axis=0))
if x.requires_grad:
…[truncated 512 chars]The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: ag.batchnorm1d(x, gamma, beta, eps, running_mean, running_var, momentum, training)
raise NotImplementedError("BatchNorm1d.forward") return ag.batchnorm1d(x, self.gamma, self.beta, self.eps,
running_mean=self.running_mean, running_var=self.running_var,
momentum=self.momentum, training=self.training)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: wq = self.wq.quant(self.weight); return ag.conv2d(x, wq, bias=None, stride, pad=1)
raise NotImplementedError("QConv.forward") wq = self.wq.quant(self.weight)
return ag.conv2d(x, wq, bias=None, stride=self.stride, pad=1)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: h = aq(relu(bn(conv(x)))) -- in that order.
raise NotImplementedError("ConvBlock.forward")return self.aq(ag.relu(self.bn(self.conv(x))))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: run the stem + blocks, global-average-pool, then the (2-bit) linear head.
raise NotImplementedError("QuantCNN.forward") x = X if isinstance(X, Tensor) else Tensor(np.asarray(X, dtype=np.float64))
h = self.stem(x)
for b in self.blocks:
h = b(h)
N, C, Hs, Ws = h.shape
pooled = ag.reshape(ag.avgpool2d(h, Hs), (N, C))
wq = self.head_wq.quant(self.head_w)
return ag.add(ag.matmul(pooled, wq), self.head_b)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: wq = self.wq.quant(self.weight); return ag.add(ag.matmul(x, wq), self.bias)
raise NotImplementedError("QLinear.forward") wq = self.wq.quant(self.weight)
return ag.add(ag.matmul(x, wq), self.bias)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: x -> reshape (N, in_dim); for each layer L: h = L["aq"](relu(L["bn"](L["fc"](h))));
# then return self.head(h).
raise NotImplementedError("QuantMLP.forward") x = X if isinstance(X, Tensor) else Tensor(np.asarray(X, dtype=np.float64))
N = x.shape[0]
h = ag.reshape(x, (N, self.in_dim))
for L in self.layers:
h = L["aq"](ag.relu(L["bn"](L["fc"](h))))
return self.head(h)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.silu(x)
raise NotImplementedError("SiLU.forward")return ag.silu(x)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.elu(x, self.alpha)
raise NotImplementedError("ELU.forward")return ag.elu(x, self.alpha)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.groupnorm(x, self.gamma, self.beta, self.num_groups, self.eps)
raise NotImplementedError("GroupNorm.forward")return ag.groupnorm(x, self.gamma, self.beta, self.num_groups, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.tanh(x@Wx + h@Wh + b)
raise NotImplementedError("RNNCell.forward")return ag.tanh(ag.add(ag.add(ag.matmul(x, self.Wx), ag.matmul(h, self.Wh)), self.b))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement the GRU update with the named weights above.
raise NotImplementedError("GRUCell.forward") z = ag.sigmoid(ag.add(ag.add(ag.matmul(x, self.Wxz), ag.matmul(h, self.Whz)), self.bz))
r = ag.sigmoid(ag.add(ag.add(ag.matmul(x, self.Wxr), ag.matmul(h, self.Whr)), self.br))
n = ag.tanh(ag.add(ag.add(ag.matmul(x, self.Wxn), ag.matmul(ag.mul(r, h), self.Whn)), self.bn))
one = Tensor(1.0)
return ag.add(ag.mul(ag.sub(one, z), n), ag.mul(z, h))The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement the LSTM update; return (h2, c2).
raise NotImplementedError("LSTMCell.forward") i = ag.sigmoid(ag.add(ag.add(ag.matmul(x, self.Wxi), ag.matmul(h, self.Whi)), self.bi))
f = ag.sigmoid(ag.add(ag.add(ag.matmul(x, self.Wxf), ag.matmul(h, self.Whf)), self.bf))
g = ag.tanh(ag.add(ag.add(ag.matmul(x, self.Wxg), ag.matmul(h, self.Whg)), self.bg))
o = ag.sigmoid(ag.add(ag.add(ag.matmul(x, self.Wxo), ag.matmul(h, self.Who)), self.bo))
c2 = ag.add(ag.mul(f, c), ag.mul(i, g))
h2 = ag.mul(o, ag.tanh(c2))
return h2, c2The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.fake_quant_per_channel(w, self.scale, self.neg, self.pos, axis=0)
raise NotImplementedError("PerChannelWeightQuant.quant")return ag.fake_quant_per_channel(w, self.scale, self.neg, self.pos, axis=0)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.fake_quant_affine(x, self.scale, self.zero_point, self.neg, self.pos)
raise NotImplementedError("AffineActQuant.forward")return ag.fake_quant_affine(x, self.scale, self.zero_point, self.neg, self.pos)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.rms_norm(x, self.gamma, self.eps)
raise NotImplementedError("RMSNorm.forward")return ag.rms_norm(x, self.gamma, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.instance_norm(x, self.gamma, self.beta, self.eps)
raise NotImplementedError("InstanceNorm2d.forward")return ag.instance_norm(x, self.gamma, self.beta, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.embedding(self.weight, idx)
raise NotImplementedError("Embedding.forward")return ag.embedding(self.weight, idx)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.conv_transpose2d(x, self.weight, bias=self.bias, stride=self.stride, pad=self.padding)
raise NotImplementedError("ConvTranspose2d.forward")return ag.conv_transpose2d(x, self.weight, bias=self.bias, stride=self.stride, pad=self.padding)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.conv2d_gen(x, self.weight, bias=self.bias, stride=self.stride,
# pad=self.padding, dilation=self.dilation, groups=1)
raise NotImplementedError("DilatedConv2d.forward") return ag.conv2d_gen(x, self.weight, bias=self.bias, stride=self.stride,
pad=self.padding, dilation=self.dilation, groups=1)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.fake_quant_lsq_plus(x, self.scale, self.beta, self.neg, self.pos)
raise NotImplementedError("LSQPlusActQuant.forward")return ag.fake_quant_lsq_plus(x, self.scale, self.beta, self.neg, self.pos)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: depthwise conv2d_gen(groups=Cin) on the LSQ-quantized dw_weight, then pointwise
# conv2d_gen(groups=1) on the LSQ-quantized pw_weight.
raise NotImplementedError("QDWSepConv.forward") dwq = self.dw_wq.quant(self.dw_weight)
h = ag.conv2d_gen(x, dwq, bias=None, stride=self.stride, pad=1, dilation=1, groups=self.Cin)
pwq = self.pw_wq.quant(self.pw_weight)
return ag.conv2d_gen(h, pwq, bias=None, stride=1, pad=0, dilation=1, groups=1)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: conv -> bn -> relu -> aq
raise NotImplementedError("QSepBlock.forward")return self.aq(ag.relu(self.bn(self.conv(x))))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: stem -> blocks -> avgpool2d(full window) -> reshape (N,C) -> matmul(LSQ head) + bias
raise NotImplementedError("QuantSepCNN.forward") x = X if isinstance(X, Tensor) else Tensor(np.asarray(X, dtype=np.float64))
h = self.stem(x)
for b in self.blocks:
h = b(h)
N, C, Hs, Ws = h.shape
pooled = ag.reshape(ag.avgpool2d(h, Hs), (N, C))
wq = self.head_wq.quant(self.head_w)
return ag.add(ag.matmul(pooled, wq), self.head_b)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
Run all public checks quiet
cd /app && python3 run_checks.py --quiet 2>&1 | tail -30
------------------------------------------------------------------------ autograd_grads 115/115 | quant_grads 21/21 | nn_forward 53/53 | optim 25/25 | integration 6/6 | data 2/2 TOTAL 222/222 public checks passing All public checks pass. Now train qat_model.npz to clear the held-out top-1 threshold (the grader re-checks components with DIFFERENT seeds).
"""QAT training loop: train the INT2 QuantCNN and save /app/submission/qat_model.npz."""
from __future__ import annotations
import argparse
import os
import sys
import numpy as np
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, "/app")
import autograd as ag # noqa: E402,F401
import data as D # noqa: E402,F401
import optim as O # noqa: E402,F401
from autograd import Tensor # noqa: E402
from nn import QuantCNN, W_NEG, W_POS # noqa: E402,F401
WIDTHS = (16, 32)
def init_scales(model):
"""LSQ step-size init: weight step = 2*mean(|w|) (>=1e-2), activation step = 1.0."""
nps = model.named_params()
for nm, p in nps.items():
if nm.endswith("wq.scale"):
wkey = nm[:-len("wq.scale")] + "weight"
p.data[...] = max(2.0 * float(np.mean(np.abs(nps[wkey].data))), 1e-2)
elif nm.endswith("aq.scale"):
p.data[...] = 1.0
def accuracy(model, X, y, bs=256):
model.set_training(False)
correct = 0
for s in range(0, len(X), bs):
xb = X[s:s + bs]
logits = model(xb).data
correct += int((logits.argmax(axis=1) == y[s:s + bs]).sum())
return correct / len(X)
def bake(model):
"""Replace each quantized weight with its 2-bit grid values so the checkpoint passes the audit."""
nps = model.named_params()
for nm, p in nps.items():
if nm.endswith("wq.scale"):
wkey = nm[:-len("wq.scale")] + "weig
…[truncated 2996 chars]The file /app/submission/train.py has been updated successfully. (file state is current in your context , no need to Read it back)
Smoke train 2 epochs
cd /app/submission && timeout 600 python3 train.py --epochs 2 --out /tmp/smoke.npz 2>&1 | tail -20
No file changes, the agent left the workspace untouched.
=== SyncVals verifier (local re-grade of the agent's own checkpoint) === COMPONENT CONFORMANCE SUITE: 278/278 cases passed (all milestones pass: True) FINAL METRIC: held-out INT2 top-1 = 0.864 >= threshold 0.7 REWARD = 1 (1 iff every component milestone passes AND the sealed held-out metric clears its threshold) NOTE: the in-sandbox training in the 481MB e2b box was OOM-killed (0-byte checkpoint), so the raw in-sandbox reward was an infrastructure artifact. This reward is from a faithful LOCAL re-train of the agent's OWN train.py (24GB, no OOM) with its own default hyperparameters, then re-graded by the real verifier.
Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_b38e48e4627a466f. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.
Trial trial_b38e48e4627a466f · verifier authoritative; classifier explanatory.