tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.List app structure
ls -la && echo "---SUBMISSION---" && ls -la submission/ && echo "---DATA---" && ls -la data/
total 89 drwxr-xr-x 4 user user 4096 Jun 30 22:41 . drwxr-xr-x 21 root root 4096 Jun 30 22:41 .. -rw-r--r-- 1 user user 12853 Jun 30 22:41 PROMPT.txt -rw-r--r-- 1 user user 5 Jun 30 22:41 agent.pid -rw-r--r-- 1 user user 0 Jun 30 22:41 agent.stderr -rw-r--r-- 1 user user 4497 Jun 30 22:41 agent_trajectory.jsonl drwxr-xr-x 2 user user 60 Jun 30 22:41 data -rw-r--r-- 1 user user 0 Jun 30 22:41 run.log -rwxr-xr-x 1 user user 47612 Jun 30 11:01 run_checks.py drwxr-xr-x 3 user user 128 Jun 30 22:41 submission -rwxr-xr-x 1 user user 1376 Jun 30 11:01 test.sh -rwxr-xr-x 1 user user 629 Jun 30 11:01 timer.sh ---SUBMISSION--- total 100 drwxr-xr-x 3 user user 128 Jun 30 22:41 . drwxr-xr-x 4 user user 4096 Jun 30 22:41 .. drwxr-xr-x 2 user user 4096 Jun 30 22:41 __pycache__ -rw-r--r-- 1 user user 27741 Jun 30 11:01 autograd.py -rw-r--r-- 1 user user 2625 Jun 30 11:01 data.py -rw-r--r-- 1 user user 21670 Jun 30 11:01 nn.py -rw-r--r-- 1 user user 6253 Jun 30 11:01 optim.py -rw-r--r-- 1 user user 20833 Jun 30 11:01 ssl.py -rw-r--r-- 1 user user 3551 Jun 30 11:01 train.py ---DATA--- total 1804 drwxr-xr-x 2 user user 60 Jun 30 22:41 . drwxr-xr-x 4 user user 4096 Jun 30 22:41 .. -rw-r--r-- 1 user user 59874 Jun 30 11:01 labeled.npz -rw-r--r-- 1 user user 1778166 Jun 30 11:01 unlabeled.npz
Line counts of stub files
wc -l submission/*.py run_checks.py test.sh 2>/dev/null
836 submission/autograd.py
54 submission/data.py
542 submission/nn.py
162 submission/optim.py
402 submission/ssl.py
86 submission/train.py
1008 run_checks.py
25 test.sh
3115 total/app/submission/autograd.py
1 """Reverse-mode automatic differentiation over numpy arrays. 2 3 YOU MUST IMPLEMENT EVERYTHING MARKED ``# TODO`` / ``raise NotImplementedError`` using numpy 4 ONLY. No torch/jax/tensorflow/autograd/etc. The grader's conformance suite differentiates each 5 op below with central finite differences and compares against YOUR analytic backward, so every 6 backward must be correct (a single wrong reduction/Jacobian fails the gradient checks). 7 8 A ``Tensor`` wraps one ``numpy.ndarray`` (stored as ``float64`` in ``self.data``) and records 9 the op that produced it so ``.backward()`` can accumulate ``.grad`` into every leaf. 10 11 CONTRACT (the grader builds its own reference against this exact API): 12 * ``Tensor(data, requires_grad=False)`` -- ``self.data`` is ``float64``; ``self.grad`` is None 13 until backward, then a same-shape ``float64`` array. requires_grad leaves accumulate grad. 14 * Every op returns a NEW Tensor whose ``requires_grad`` is the OR of its differentiable inputs, 15 and whose ``_backward`` closure does ``input._accum(local_grad)`` (so multiple paths add up). 16 * ``t.backward(grad=None)`` -- seeds ``grad`` (all-ones for a scalar) and walks the graph in 17 reverse topological order, calling each node's ``_backward``. Repeated backward accumulates. 18 * BROADCASTING: binary ops broadcast like numpy; the backward MUST reduce (sum) the upstream 19 gradient back to each input's ORIGINAL shape (use the provided ``_unbroadcast`` helper). 20 * Numerically …[truncated 29481 chars]
/app/submission/nn.py
1 """Neural-network layers + several from-scratch models, built on YOUR autograd engine.
2
3 Implement every ``# TODO`` forward (the parameters + the ``named_params`` naming are already
4 wired for you; you compose the autograd ops). The grader checks each layer's forward against its
5 OWN reference AND finite-difference-checks the gradients that flow through your autograd, so the
6 composition must be exactly right.
7
8 LAYER / MODEL CHECKLIST (forward + grad checked):
9 Linear Embedding LayerNorm BatchNorm1d Dropout Conv2d MaxPool2d AvgPool2d
10 MultiHeadSelfAttention MLP TransformerBlock
11 RNNCell LSTMCell GRUCell
12 HiddenBlock + MLPClassifier (THE METRIC MODEL) | CNNClassifier (images)
13 | SeqClassifier (rnn/lstm/gru sequence)
14
15 CHECKPOINT NAMING CONTRACT for the METRIC model (model.npz; the grader loads YOUR ``.npz`` into
16 its own reference MLPClassifier by these exact keys/shapes -- do not rename):
17 MLPClassifier (model.npz):
18 layers.{i}.fc.weight (d_in_i, d_hidden) | layers.{i}.fc.bias (d_hidden,)
19 layers.{i}.ln.gamma/.beta (d_hidden,)
20 head.weight (d_hidden, num_classes) | head.bias (num_classes,)
21 (layer 0 has d_in_0 = in_dim; layers 1.. have d_in_i = d_hidden.)
22 CNNClassifier:
23 conv1.weight (C,1,3,3) | conv1.bias (C,) | conv2.weight (C,C,3,3) | conv2.bias (C,)
24 fc.weight (C*(img//4)*(img//4), n_classes) | fc.bias (n_classes,)
25 SeqClassifier, cell in {rnn (G=1), lstm (G=4), gru (G=3)}:
26 embed
…[truncated 22234 chars]/app/submission/optim.py
1 """Optimizers + LR schedulers over a flat list of autograd ``Tensor`` parameters. 2 3 Implement the ``# TODO`` update math + schedulers. Every optimizer updates ``param.data`` IN 4 PLACE using ``param.grad``. The grader runs one (and several) step(s) and compares your updated 5 params against its reference within a tight tolerance, so the math must be exact. ``t`` is the 6 1-based step count (track it yourself where bias correction is needed). 7 8 SGD (momentum ``mu``, coupled weight decay ``wd``, optional Nesterov): 9 g = grad + wd * param 10 v = mu * v + g 11 update = (g + mu * v) if nesterov else v 12 param -= lr * update 13 14 Adagrad: g = grad + wd*param ; s += g^2 ; param -= lr * g / (sqrt(s) + eps) 15 RMSprop: g = grad + wd*param ; s = alpha*s + (1-alpha)*g^2 ; param -= lr*g/(sqrt(s)+eps) 16 17 Adam (COUPLED / L2 weight decay folded into the gradient): 18 g = grad + wd*param 19 m = b1*m + (1-b1)*g ; v = b2*v + (1-b2)*g^2 20 mhat = m/(1-b1^t) ; vhat = v/(1-b2^t) 21 param -= lr * mhat / (sqrt(vhat) + eps) 22 23 AdamW (DECOUPLED weight decay -- decay the PARAM, not the moments): 24 m = b1*m + (1-b1)*g ; v = b2*v + (1-b2)*g^2 25 mhat = m/(1-b1^t) ; vhat = v/(1-b2^t) 26 param -= lr * ( mhat/(sqrt(vhat)+eps) + wd*param ) 27 28 Lion (sign of an interpolated momentum; decoupled weight decay): 29 update = sign( b1*m + (1-b1)*g ) 30 param -= lr * ( update + wd*param ) 31 m = b2*m + (1-b2)*g …[truncated 5297 chars]
/app/submission/data.py
1 """Data loading + minibatch iteration for the FixMatch task, plus tiny synthetic dataset builders
2 the component suite uses to exercise the CNN / sequence models.
3
4 Implement every ``# TODO`` body. ``/app/data/labeled.npz`` ships the FEW labeled examples (arrays
5 ``X`` (N_lab, dim) standardized features + ``y`` (N_lab,) integer labels) and
6 ``/app/data/unlabeled.npz`` ships the large UNLABELED pool (array ``X`` only).
7
8 Contract:
9 load_npz(path) -> (X float64 (N, dim), y int64 (N,) or None if the file has no ``y``)
10 n_classes(y) -> int (max label + 1)
11 epoch_batches(n, batch_size, rng) -> yields arrays of shuffled indices (last batch may be short)
12 get_batch(X, y, idx) -> (Xb float64 (B, dim), yb int64 (B,) or None, idx int64 (B,))
13 make_image_dataset(n, seed, img, n_classes) -> (X (n,1,img,img), y (n,)) for the CNN model
14 make_seq_dataset(n, seed, T, vocab, n_classes) -> (X (n,T) int, y (n,)) for the seq model
15 """
16 from __future__ import annotations
17
18 from typing import Optional, Tuple
19
20 import numpy as np
21
22
23 def load_npz(path: str) -> Tuple[np.ndarray, Optional[np.ndarray]]:
24 # TODO: np.load the file; return (X as float64, y as int64 reshaped to 1-D or None if absent).
25 raise NotImplementedError("load_npz")
26
27
28 def n_classes(y: np.ndarray) -> int:
29 # TODO: return int(max(y)) + 1
30 raise NotImplementedError("n_classes")
31
32
33 def epoch_batches(n: int, batch_size: int, rng: np.random.G
…[truncated 1281 chars]/app/submission/ssl.py
1 """FixMatch (+ related MixMatch / UDA) semi-supervised-learning machinery, on top of YOUR numpy 2 autograd engine. THIS IS THE DOMAIN DELIVERABLE -- implement every ``# TODO`` body. 3 4 Each differentiable loss below is an autograd OP: it must build a scalar Tensor whose ``_backward`` 5 adds the correct analytic gradient into the input logits (the grader finite-difference gradient- 6 checks each one). The augmentation / gating / alignment / EMA pieces are pure numpy recursions the 7 grader checks for exactness. A wrong gradient, gate, alignment, sharpen, mixup, interleave, or EMA 8 update fails the domain milestone -> reward 0. 9 10 CHECKLIST (each is a separate graded check): 11 AUGMENTATION (numpy, label-preserving, no grad): 12 weak_augment strong_augment cutout mixup interleave / de_interleave 13 PSEUDO-LABELING / DISTRIBUTION CONTROL (numpy, no grad): 14 pseudo_label confidence_mask sharpen DistributionAligner EMAScalar 15 DIFFERENTIABLE LOSSES (autograd ops, hand-written analytic backward): 16 soft_cross_entropy consistency_loss mse_consistency kl_divergence 17 entropy_loss class_balance_loss fixmatch_loss 18 EMA TEACHER (numpy recursion): 19 EMATeacher 20 21 NUMERICAL CONVENTIONS (MUST match): 22 * soft_cross_entropy / consistency: per-row loss ``-sum_c target[c]*logp[c]`` weighted by 23 ``weight[n]`` and AVERAGED OVER THE FULL BATCH N (divide by N, not by the number of confident 24 rows). Backward wrt logits: `` …[truncated 20837 chars]
/app/submission/train.py
1 """FixMatch semi-supervised training loop using YOUR from-scratch numpy framework + YOUR SSL 2 machinery in ssl.py. 3 4 Implement the ``# TODO`` training loop. Per step: 5 * supervised: cross-entropy on a WEAKLY-augmented labeled batch; 6 * unlabeled: forward a WEAKLY-augmented view -> softmax -> (distribution-align) -> confidence-gated 7 pseudo-labels; then minimize the masked consistency loss between the STRONGLY-augmented view and 8 those (stop-gradient) pseudo-labels, weighted by lambda_u; 9 * backward -> grad-clip -> optimizer step (an LR warmup + cosine schedule is a good choice); 10 * EMA-update a teacher copy of the weights. 11 12 Save an ``.npz`` checkpoint of the EMA-averaged classifier state_dict so the grader can load your 13 weights into ITS reference classifier and score top-1 on the SEALED held-out TEST split. 14 15 PIN the architecture to match the grader (it rebuilds exactly this graph and loads your weights): 16 in_dim = X.shape[1], num_classes = max(y)+1, d_hidden = 256, n_layer = 3. 17 18 NOTE: a supervised-only run on the few labels overfits them and generalizes near the supervised 19 floor; a naive / collapsed FixMatch run (no distribution alignment / no supervised warmup / 20 mis-tuned gate) collapses to a constant predictor near chance. You need the full FixMatch recipe, 21 with correct component gradients, to clear the threshold. 22 """ 23 from __future__ import annotations 24 25 import argparse 26 import json 27 import math 2 …[truncated 2303 chars]
/app/run_checks.py
1 #!/usr/bin/env python3 2 """PUBLIC component check runner for the from-scratch numpy DL framework + FixMatch SSL machinery. 3 4 Run this AS OFTEN AS YOU LIKE while you implement to get a per-check progress signal: 5 6 python3 /app/run_checks.py # check the framework in /app/submission 7 python3 /app/run_checks.py --submission /some/dir 8 python3 /app/run_checks.py --only fixmatch # run one milestone 9 python3 /app/run_checks.py --quiet # summary line only 10 11 It imports YOUR modules from /app/submission (autograd.py / nn.py / optim.py / data.py / ssl.py), 12 exactly as the grader does, and runs a PUBLIC portion of the component conformance suite, printing 13 a clear per-check report so you can implement -> run -> read failures -> fix -> repeat. 14 15 HOW THIS RELATES TO YOUR GRADE 16 ------------------------------ 17 This runner is a DEBUGGING AID, not the grader. It checks your framework against SELF-CONSISTENT 18 references it computes on its own: 19 * autograd_grads : central FINITE-DIFFERENCE of YOUR forward vs YOUR analytic backward, for every 20 op and every layer/cell/model. (A wrong backward fails here.) 21 * nn_forward : mathematical PROPERTIES your forward must satisfy (softmax rows sum to 1, 22 log_softmax == log(softmax), layernorm/batchnorm row stats, the attention mask 23 is causal, sigmoid/tanh/relu/gelu definitions, shape inva …[truncated 50050 chars]
Inspect data shapes
python3 -c "
import numpy as np
d=np.load('data/labeled.npz'); print('labeled X',d['X'].shape, 'y',d['y'].shape, 'classes', sorted(set(d['y'].tolist())))
import collections; print(collections.Counter(d['y'].tolist()))
u=np.load('data/unlabeled.npz'); print('unlabeled keys',u.files,'X',u['X'].shape)
print('X stats',d['X'].mean(),d['X'].std())
" && echo "---TEST.SH---" && cat test.shlabeled X (200, 80) y (200,) classes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Counter({7: 20, 0: 20, 9: 20, 2: 20, 6: 20, 5: 20, 1: 20, 8: 20, 4: 20, 3: 20})
unlabeled keys ['X'] X (6000, 80)
X stats -0.013586603 0.9973769
---TEST.SH---
#!/usr/bin/env bash
# Smoke test (NOT the grader): confirms the environment is wired -- numpy imports, the few labeled
# split + the unlabeled pool load, and the submission stubs are present. It does NOT check
# correctness (the stubs raise NotImplementedError until you implement them).
set -uo pipefail
python3 - <<'PY'
import numpy as np, os
lp, up = "/app/data/labeled.npz", "/app/data/unlabeled.npz"
assert os.path.exists(lp), "missing labeled.npz"
assert os.path.exists(up), "missing unlabeled.npz"
dl = np.load(lp); du = np.load(up)
print(f"labeled.npz: X{dl['X'].shape} {dl['X'].dtype} y{dl['y'].shape} classes={len(set(dl['y'].tolist()))}")
print(f"unlabeled.npz: X{du['X'].shape} {du['X'].dtype} (no labels)")
for f in ("autograd.py", "nn.py", "optim.py", "data.py", "ssl.py", "train.py"):
assert os.path.exists(f"/app/submission/{f}"), f"missing /app/submission/{f}"
assert os.path.exists("/app/run_checks.py"), "missing /app/run_checks.py"
print("submission stubs present:", sorted(os.listdir("/app/submission")))
print("numpy", np.__version__)
PY
# The visible check runner is present + runnable (it exits non-zero on the unfilled stub, which is
# expected -- the stubs raise NotImplementedError until you implement them).
python3 /app/run_checks.py -
…[truncated 102 chars]/app/submission/autograd.py
TODO: implement the topological walk + seeding. (The per-op ``_backward`` closures do
the local accumulation; here you only orchestrate the order + seed ``self.grad``.)
"""
raise NotImplementedError("Tensor.backward") TODO: implement the topological walk + seeding. (The per-op ``_backward`` closures do
the local accumulation; here you only orchestrate the order + seed ``self.grad``.)
"""
topo: List["Tensor"] = []
visited = set()
def build(t: "Tensor"):
if id(t) in visited:
return
visited.add(id(t))
for p in t._parents:
build(p)
topo.append(t)
build(self)
if grad is None:
grad = np.ones_like(self.data)
self._accum(_as_f64(grad))
for node in reversed(topo):
node._backward()The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("add backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sub backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(-g, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mul backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g * b.data, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * a.data, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("div backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(g / b.data, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(-g * a.data / (b.data ** 2), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("power backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * pf * (a.data ** (pf - 1.0)))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("maximum backward") # TODO def _bw():
g = out.grad
am = (a.data > b.data).astype(np.float64) + 0.5 * (a.data == b.data)
if a.requires_grad:
a._accum(_unbroadcast(g * am, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * (1.0 - am), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("minimum backward") # TODO def _bw():
g = out.grad
am = (a.data < b.data).astype(np.float64) + 0.5 * (a.data == b.data)
if a.requires_grad:
a._accum(_unbroadcast(g * am, a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(g * (1.0 - am), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("where backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(_unbroadcast(np.where(c, g, 0.0), a.data.shape))
if b.requires_grad:
b._accum(_unbroadcast(np.where(c, 0.0, g), b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("exp backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * out.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("log backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad / a.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sqrt backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * 0.5 / out.data)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("abs backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * np.sign(a.data))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("clip backward") # TODO def _bw():
if a.requires_grad:
mask = ((a.data > lo) & (a.data < hi)).astype(np.float64)
a._accum(out.grad * mask)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sum backward") # TODO def _bw():
if not a.requires_grad:
return
g = out.grad
if axis is not None and not keepdims:
g = np.expand_dims(g, axis)
a._accum(np.broadcast_to(g, a.data.shape).copy())The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mean backward") # TODO def _bw():
if not a.requires_grad:
return
g = out.grad
if axis is None:
n = a.data.size
elif isinstance(axis, tuple):
n = int(np.prod([a.data.shape[ax] for ax in axis]))
else:
n = a.data.shape[axis]
if axis is not None and not keepdims:
g = np.expand_dims(g, axis)
a._accum(np.broadcast_to(g, a.data.shape).copy() / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("max backward") # TODO def _bw():
if not a.requires_grad:
return
outd = out.data
g = out.grad
if axis is not None and not keepdims:
outd = np.expand_dims(outd, axis)
g = np.expand_dims(g, axis)
mask = (a.data == outd).astype(np.float64)
counts = mask.sum(axis=axis, keepdims=True)
a._accum(mask * g / counts)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("min backward") # TODO def _bw():
if not a.requires_grad:
return
outd = out.data
g = out.grad
if axis is not None and not keepdims:
outd = np.expand_dims(outd, axis)
g = np.expand_dims(g, axis)
mask = (a.data == outd).astype(np.float64)
counts = mask.sum(axis=axis, keepdims=True)
a._accum(mask * g / counts)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("transpose backward") # TODO def _bw():
if not a.requires_grad:
return
if axes is None:
a._accum(np.transpose(out.grad))
else:
inv = np.argsort(axes)
a._accum(np.transpose(out.grad, tuple(inv)))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("reshape backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad.reshape(a.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("getitem backward") # TODO def _bw():
if not a.requires_grad:
return
gz = np.zeros_like(a.data)
np.add.at(gz, idx, out.grad)
a._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("concat backward") # TODO def _bw():
g = out.grad
offset = 0
for t in ts:
sz = t.data.shape[axis]
if t.requires_grad:
sl = [slice(None)] * g.ndim
sl[axis] = slice(offset, offset + sz)
t._accum(g[tuple(sl)])
offset += szThe file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("stack backward") # TODO def _bw():
g = out.grad
for i, t in enumerate(ts):
if t.requires_grad:
sl = [slice(None)] * g.ndim
sl[axis] = i
t._accum(g[tuple(sl)])The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("matmul backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
da = g @ np.swapaxes(b.data, -1, -2)
a._accum(_unbroadcast(da, a.data.shape))
if b.requires_grad:
db = np.swapaxes(a.data, -1, -2) @ g
b._accum(_unbroadcast(db, b.data.shape))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("relu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (a.data > 0.0))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("leaky_relu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * np.where(a.data > 0.0, 1.0, sl))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("sigmoid backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * s * (1.0 - s))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("tanh backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (1.0 - t * t))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("softplus backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * (1.0 / (1.0 + np.exp(-bx))))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("silu backward") # TODO def _bw():
if a.requires_grad:
a._accum(out.grad * s * (1.0 + x * (1.0 - s)))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mish backward") # TODO def _bw():
if a.requires_grad:
sig = 1.0 / (1.0 + np.exp(-x))
a._accum(out.grad * (t + x * (1.0 - t * t) * sig))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("elu backward") # TODO def _bw():
if a.requires_grad:
grad_local = np.where(pos, 1.0, al * np.exp(np.minimum(x, 0.0)))
a._accum(out.grad * grad_local)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("logsumexp backward") # TODO def _bw():
if not a.requires_grad:
return
g = out.grad
if not keepdims:
g = np.expand_dims(g, axis)
sm = e / s
a._accum(g * sm)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("var backward") # TODO def _bw():
if not a.requires_grad:
return
if axis is None:
n = x.size
elif isinstance(axis, tuple):
n = int(np.prod([x.shape[ax] for ax in axis]))
else:
n = x.shape[axis]
g = out.grad
if axis is not None and not keepdims:
g = np.expand_dims(g, axis)
a._accum(g * (2.0 / n) * xc)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("std backward") # TODO def _bw():
if not a.requires_grad:
return
if axis is None:
n = x.size
elif isinstance(axis, tuple):
n = int(np.prod([x.shape[ax] for ax in axis]))
else:
n = x.shape[axis]
g = out.grad
if axis is not None and not keepdims:
g = np.expand_dims(g, axis)
a._accum(g * xc / (n * sd))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("cumsum backward") # TODO def _bw():
if not a.requires_grad:
return
g = out.grad
rev = np.flip(np.cumsum(np.flip(g, axis=axis), axis=axis), axis=axis)
a._accum(rev)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("outer backward") # TODO def _bw():
g = out.grad
if a.requires_grad:
a._accum(g @ b.data.reshape(-1))
if b.requires_grad:
b._accum(a.data.reshape(-1) @ g)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("gelu backward") # TODO def _bw():
if a.requires_grad:
pdf = np.exp(-0.5 * x * x) / np.sqrt(2.0 * np.pi)
a._accum(out.grad * (cdf + x * pdf))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("softmax backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
a._accum(s * (g - (g * s).sum(axis=axis, keepdims=True)))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("log_softmax backward") # TODO def _bw():
if a.requires_grad:
g = out.grad
sm = np.exp(out.data)
a._accum(g - sm * g.sum(axis=axis, keepdims=True))The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("cross_entropy backward") # TODO def _bw():
if not logits.requires_grad:
return
sm = np.exp(logp)
oh = np.zeros_like(sm)
oh[np.arange(n), t] = 1.0
logits._accum(out.grad * (sm - oh) / n)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("mse_loss backward") # TODO def _bw():
if pred.requires_grad:
pred._accum(out.grad * 2.0 * (pred.data - tgt) / pred.data.size)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("layernorm backward") # TODO def _bw():
g = out.grad
D = x.shape[-1]
if gamma.requires_grad:
axes = tuple(range(g.ndim - 1))
gamma._accum((g * xhat).sum(axis=axes))
if beta.requires_grad:
axes = tuple(range(g.ndim - 1))
beta._accum(g.sum(axis=axes))
if a.requires_grad:
gx = g * gamma.data
dx = inv / D * (D * gx - gx.sum(axis=-1, keepdims=True)
- xhat * (gx * xhat).sum(axis=-1, keepdims=True))
a._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("batchnorm backward") # TODO def _bw():
g = out.grad
N = x.shape[0]
if gamma.requires_grad:
gamma._accum((g * xhat).sum(axis=0))
if beta.requires_grad:
beta._accum(g.sum(axis=0))
if a.requires_grad:
gx = g * gamma.data
dx = inv / N * (N * gx - gx.sum(axis=0, keepdims=True)
- xhat * (gx * xhat).sum(axis=0, keepdims=True))
a._accum(dx)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw():
raise NotImplementedError("embedding backward") # TODO def _bw():
if not weight.requires_grad:
return
gz = np.zeros_like(weight.data)
np.add.at(gz, idx, out.grad)
weight._accum(gz)The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
def _bw_id():
raise NotImplementedError("dropout (eval) backward") # TODO
out._backward = _bw_id
return out
raise NotImplementedError("dropout (training) -- optional") def _bw_id():
if a.requires_grad:
a._accum(out.grad)
out._backward = _bw_id
return out
r = rng if rng is not None else np.random.default_rng()
mask = (r.random(a.data.shape) >= p).astype(np.float64) / (1.0 - p)
out = _mk(a.data * mask, (a,), a.requires_grad)
def _bw():
if a.requires_grad:
a._accum(out.grad * mask)
out._backward = _bw
return outThe file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
TODO: implement the FORWARD and the backward. (Unlike the other ops, the forward is NOT
pre-filled for you -- build the output, then differentiate it.)
"""
raise NotImplementedError("conv2d") TODO: implement the FORWARD and the backward. (Unlike the other ops, the forward is NOT
pre-filled for you -- build the output, then differentiate it.)
"""
x = _ensure(x)
weight = _ensure(weight)
has_bias = bias is not None
if has_bias:
bias = _ensure(bias)
N, Cin, H, W = x.data.shape
Cout, Cin2, KH, KW = weight.data.shape
s = int(stride)
pad = int(padding)
xp = _pad2d(x.data, pad)
Hp, Wp = xp.shape[2], xp.shape[3]
OH = (Hp - KH) // s + 1
OW = (Wp - KW) // s + 1
# im2col: (N, Cin, KH, KW, OH, OW)
cols = np.empty((N, Cin, KH, KW, OH, OW), dtype=np.float64)
for i in range(KH):
for j in range(KW):
cols[:, :, i, j, :, :] = xp[:, :, i:i + s * OH:s, j:j + s * OW:s]
cols_r = cols.reshape(N, Cin * KH * KW, OH * OW)
w_r = weight.data.reshape(Cout, Cin * KH * KW)
out_data = np.einsum("oc,ncp->nop", w_r, cols_r).reshape(N, Cout, OH, OW)
if has_bias:
out_data = out_data + bias.data.reshape(1, Cout, 1, 1)
rg = x.requires_grad or weight.requires_grad or (has_bias and bias.requires_grad)
parents = (x, weight, bias) if has_bias else (x, weight)
out = _mk(out_data, parents, rg)
def _bw():
g = out.grad # (N, Cout, OH, OW)
g_r = g.reshape(N, Cout, OH * OW)
if has_bias and bias.requires_grad:
bias._accum(g.sum(axis=(0, 2, 3)))
if weight.requires_grad:
dw_r = np.einsum("nop,ncp->oc", g_r, cols_r)
…[truncated 580 chars]The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
TODO: implement the FORWARD and the backward.
"""
raise NotImplementedError("maxpool2d") TODO: implement the FORWARD and the backward.
"""
x = _ensure(x)
N, C, H, W = x.data.shape
k = int(kernel)
s = int(stride)
OH = (H - k) // s + 1
OW = (W - k) // s + 1
win = np.empty((N, C, k, k, OH, OW), dtype=np.float64)
for i in range(k):
for j in range(k):
win[:, :, i, j, :, :] = x.data[:, :, i:i + s * OH:s, j:j + s * OW:s]
win_r = win.reshape(N, C, k * k, OH, OW)
out_data = win_r.max(axis=2)
out = _mk(out_data, (x,), x.requires_grad)
def _bw():
if not x.requires_grad:
return
g = out.grad # (N, C, OH, OW)
mask = (win_r == out_data[:, :, None, :, :]).astype(np.float64)
counts = mask.sum(axis=2, keepdims=True)
contrib = mask / counts * g[:, :, None, :, :]
contrib = contrib.reshape(N, C, k, k, OH, OW)
dx = np.zeros_like(x.data)
for i in range(k):
for j in range(k):
dx[:, :, i:i + s * OH:s, j:j + s * OW:s] += contrib[:, :, i, j, :, :]
x._accum(dx)
out._backward = _bw
return outThe file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/autograd.py
TODO: implement the FORWARD and the backward.
"""
raise NotImplementedError("avgpool2d") TODO: implement the FORWARD and the backward.
"""
x = _ensure(x)
N, C, H, W = x.data.shape
k = int(kernel)
s = int(stride)
OH = (H - k) // s + 1
OW = (W - k) // s + 1
acc = np.zeros((N, C, OH, OW), dtype=np.float64)
for i in range(k):
for j in range(k):
acc += x.data[:, :, i:i + s * OH:s, j:j + s * OW:s]
out_data = acc / (k * k)
out = _mk(out_data, (x,), x.requires_grad)
def _bw():
if not x.requires_grad:
return
g = out.grad / (k * k)
dx = np.zeros_like(x.data)
for i in range(k):
for j in range(k):
dx[:, :, i:i + s * OH:s, j:j + s * OW:s] += g
x._accum(dx)
out._backward = _bw
return outThe file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: matmul x@weight then add bias (if present). Use ag.matmul / ag.add.
raise NotImplementedError("Linear.forward") out = ag.matmul(x, self.weight)
if self.bias is not None:
out = ag.add(out, self.bias)
return outThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.embedding(self.weight, idx)
raise NotImplementedError("Embedding.forward")return ag.embedding(self.weight, idx)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.layernorm(x, self.gamma, self.beta, self.eps)
raise NotImplementedError("LayerNorm.forward")return ag.layernorm(x, self.gamma, self.beta, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: ms = mean(x*x, -1, keepdims); inv = (ms+eps)**-0.5; return x*inv*gamma (autograd ops)
raise NotImplementedError("RMSNorm.forward") ms = ag.mean(ag.mul(x, x), axis=-1, keepdims=True)
inv = ag.power(ag.add(ms, self.eps), -0.5)
return ag.mul(ag.mul(x, inv), self.gamma)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: g = ag.silu(self.w_gate(x)); u = self.w_up(x); return self.w_down(ag.mul(g, u))
raise NotImplementedError("SwiGLU.forward") g = ag.silu(self.w_gate(x))
u = self.w_up(x)
return self.w_down(ag.mul(g, u))The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.batchnorm(x, self.gamma, self.beta, self.eps)
raise NotImplementedError("BatchNorm1d.forward")return ag.batchnorm(x, self.gamma, self.beta, self.eps)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.dropout(x, self.p, training, rng)
raise NotImplementedError("Dropout.forward")return ag.dropout(x, self.p, training, rng)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.conv2d(x, self.weight, self.bias, self.stride, self.padding)
raise NotImplementedError("Conv2d.forward")return ag.conv2d(x, self.weight, self.bias, self.stride, self.padding)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.maxpool2d(x, self.kernel, self.stride)
raise NotImplementedError("MaxPool2d.forward")return ag.maxpool2d(x, self.kernel, self.stride)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: return ag.avgpool2d(x, self.kernel, self.stride)
raise NotImplementedError("AvgPool2d.forward")return ag.avgpool2d(x, self.kernel, self.stride)
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement causal multi-head self-attention as described in the docstring.
raise NotImplementedError("MultiHeadSelfAttention.forward") B, Tn, _ = x.shape
H, dh = self.n_head, self.d_head
qkv = self.qkv(x) # (B, T, 3*d_model)
qkv = ag.reshape(qkv, (B, Tn, 3, H, dh))
qkv = ag.transpose(qkv, (2, 0, 3, 1, 4)) # (3, B, H, T, dh)
q = qkv[0]
k = qkv[1]
v = qkv[2]
kt = ag.transpose(k, (0, 1, 3, 2)) # (B, H, dh, T)
scores = ag.mul(ag.matmul(q, kt), 1.0 / np.sqrt(dh)) # (B, H, T, T)
mask = np.triu(np.ones((Tn, Tn), dtype=np.float64), k=1) * (-1e9)
scores = ag.add(scores, Tensor(mask.reshape(1, 1, Tn, Tn)))
att = ag.softmax(scores, axis=-1)
out = ag.matmul(att, v) # (B, H, T, dh)
out = ag.transpose(out, (0, 2, 1, 3)) # (B, T, H, dh)
out = ag.reshape(out, (B, Tn, self.d_model))
return self.proj(out)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: fc2(gelu(fc1(x)))
raise NotImplementedError("MLP.forward")return self.fc2(ag.gelu(self.fc1(x)))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: pre-norm residual wiring (see docstring).
raise NotImplementedError("TransformerBlock.forward") x = ag.add(x, self.attn(self.ln1(x)))
x = ag.add(x, self.mlp(self.ln2(x)))
return xThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: tanh(x@W_ih + b_ih + h@W_hh + b_hh)
raise NotImplementedError("RNNCell.forward") pre = ag.add(ag.add(ag.matmul(x, self.W_ih), self.b_ih),
ag.add(ag.matmul(h, self.W_hh), self.b_hh))
return ag.tanh(pre)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement the LSTM cell update; return (h_new, c_new).
raise NotImplementedError("LSTMCell.forward") h, c = state
H = self.hidden
z = ag.add(ag.add(ag.matmul(x, self.W_ih), self.b_ih),
ag.add(ag.matmul(h, self.W_hh), self.b_hh))
i = ag.sigmoid(z[:, 0 * H:1 * H])
f = ag.sigmoid(z[:, 1 * H:2 * H])
g = ag.tanh(z[:, 2 * H:3 * H])
o = ag.sigmoid(z[:, 3 * H:4 * H])
c_new = ag.add(ag.mul(f, c), ag.mul(i, g))
h_new = ag.mul(o, ag.tanh(c_new))
return h_new, c_newThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement the GRU cell update; return h_new.
raise NotImplementedError("GRUCell.forward") H = self.hidden
gi = ag.add(ag.matmul(x, self.W_ih), self.b_ih)
gh = ag.add(ag.matmul(h, self.W_hh), self.b_hh)
r = ag.sigmoid(ag.add(gi[:, 0 * H:1 * H], gh[:, 0 * H:1 * H]))
z = ag.sigmoid(ag.add(gi[:, 1 * H:2 * H], gh[:, 1 * H:2 * H]))
n = ag.tanh(ag.add(gi[:, 2 * H:3 * H], ag.mul(r, gh[:, 2 * H:3 * H])))
h_new = ag.add(ag.mul(ag.sub(1.0, z), n), ag.mul(z, h))
return h_newThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: relu(layernorm(linear(x))) -- compose self.fc, self.ln and ag.relu.
raise NotImplementedError("HiddenBlock.forward")return ag.relu(self.ln(self.fc(x)))
The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: h = Tensor(X) if X is a raw array; for blk in self.layers: h = blk(h);
# return self.head(h)
raise NotImplementedError("MLPClassifier.forward") h = X if isinstance(X, Tensor) else Tensor(np.asarray(X, dtype=np.float64))
for blk in self.layers:
h = blk(h)
return self.head(h)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: implement the conv->relu->maxpool x2 -> flatten -> fc pipeline (see docstring).
# Wrap a raw ndarray x in a Tensor first; reshape with the batch size N as dim 0.
raise NotImplementedError("CNNClassifier.forward") h = x if isinstance(x, Tensor) else Tensor(np.asarray(x, dtype=np.float64))
N = h.shape[0]
h = ag.maxpool2d(ag.relu(self.conv1(h)), 2, 2)
h = ag.maxpool2d(ag.relu(self.conv2(h)), 2, 2)
h = ag.reshape(h, (N, self.feat))
return self.fc(h)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
# TODO: embed idx, loop the cell over time from a zero state, classify the final hidden.
raise NotImplementedError("SeqClassifier.forward") idx = np.asarray(idx)
N, T = idx.shape
emb = self.embed(idx) # (N, T, d_model)
h = Tensor(np.zeros((N, self.hidden)))
if self.cell_kind == "lstm":
c = Tensor(np.zeros((N, self.hidden)))
for t in range(T):
h, c = self.cell(emb[:, t, :], (h, c))
else:
for t in range(T):
h = self.cell(emb[:, t, :], h)
return self.fc(h)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
Run autograd grad checks
python3 run_checks.py --only autograd_grads 2>&1 | tail -25
grad/cross_entropy OK grad/embedding OK grad/nn.Linear_x OK grad/nn.RMSNorm_x OK grad/nn.SwiGLU_x OK grad/nn.LayerNorm_x OK grad/nn.BatchNorm1d_x OK grad/nn.HiddenBlock_x OK grad/nn.MLP_x OK grad/nn.MHA_x OK grad/nn.Conv2d_x OK grad/nn.MaxPool2d_x OK grad/nn.AvgPool2d_x OK grad/nn.RNNCell_x OK grad/nn.LSTMCell_x OK grad/nn.GRUCell_x OK grad/clf_head.bias OK grad/clf_layers.0.fc.weight OK grad/clf_layers.1.ln.gamma OK grad/clf_layers.0.fc.bias OK ---------------------------------------------------------------------- autograd_grads 77/77 TOTAL 77/77 public checks passing All public checks pass. Now make sure model.npz clears the held-out test threshold, and remember the grader re-checks with DIFFERENT seeds/shapes.
Run nn_forward checks
python3 run_checks.py --only nn_forward 2>&1 | tail -8
fwd/MLPClassifier_shape OK fwd/CNN_shape OK fwd/Seq_shape OK ---------------------------------------------------------------------- nn_forward 20/20 TOTAL 20/20 public checks passing All public checks pass. Now make sure model.npz clears the held-out test threshold, and remember the grader re-checks with DIFFERENT seeds/shapes.
/app/submission/optim.py
def step(self):
# TODO: SGD(+momentum, +coupled weight decay, +optional Nesterov) update.
raise NotImplementedError("SGD.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
v = self.momentum * self._v[i] + g
self._v[i] = v
upd = (g + self.momentum * v) if self.nesterov else v
p.data -= self.lr * updThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
# TODO: Adagrad update (accumulate squared grads).
raise NotImplementedError("Adagrad.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._s[i] = self._s[i] + g * g
p.data -= self.lr * g / (np.sqrt(self._s[i]) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
# TODO: RMSprop update (EMA of squared grads).
raise NotImplementedError("RMSprop.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._s[i] = self.alpha * self._s[i] + (1 - self.alpha) * (g * g)
p.data -= self.lr * g / (np.sqrt(self._s[i]) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
# TODO: Adam update with bias correction + COUPLED (L2) weight decay.
raise NotImplementedError("Adam.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._v[i] = self.b2 * self._v[i] + (1 - self.b2) * (g * g)
mhat = self._m[i] / (1 - self.b1 ** self.t)
vhat = self._v[i] / (1 - self.b2 ** self.t)
p.data -= self.lr * mhat / (np.sqrt(vhat) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
# TODO: AdamW update with bias correction + DECOUPLED weight decay.
raise NotImplementedError("AdamW.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._v[i] = self.b2 * self._v[i] + (1 - self.b2) * (g * g)
mhat = self._m[i] / (1 - self.b1 ** self.t)
vhat = self._v[i] / (1 - self.b2 ** self.t)
p.data -= self.lr * (mhat / (np.sqrt(vhat) + self.eps) + self.weight_decay * p.data)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
# TODO: Lion update (sign of interpolated momentum; momentum updated AFTER the step).
raise NotImplementedError("Lion.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad
upd = np.sign(self.b1 * self._m[i] + (1 - self.b1) * g)
p.data -= self.lr * (upd + self.weight_decay * p.data)
self._m[i] = self.b2 * self._m[i] + (1 - self.b2) * gThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
# TODO: compute the global L2 norm over all param grads; scale in place if it exceeds max_norm.
raise NotImplementedError("clip_grad_norm") total = math.sqrt(sum(float((p.grad ** 2).sum()) for p in params if p.grad is not None))
if total > max_norm:
sc = max_norm / (total + 1e-6)
for p in params:
if p.grad is not None:
p.grad = p.grad * sc
return totalThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
# TODO
raise NotImplementedError("cosine_lr") if warmup > 0 and step < warmup:
return base_lr * (step + 1) / warmup
prog = min(1.0, max(0.0, (step - warmup) / max(1, total - warmup)))
return min_lr + 0.5 * (base_lr - min_lr) * (1 + math.cos(math.pi * prog))The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
# TODO
raise NotImplementedError("step_lr")return base_lr * gamma ** (step // step_size)
The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
# TODO
raise NotImplementedError("exponential_lr")return base_lr * gamma ** step
The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
# TODO
raise NotImplementedError("warmup_lr") if step < warmup:
return base_lr * (step + 1) / warmup
return base_lrThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
Run optim checks
python3 run_checks.py --only optim 2>&1 | tail -25
[optim] 16/16 optim/SGD_1step OK (max_abs=0.0e+00) optim/SGD_momentum_5step OK (max_abs=0.0e+00) optim/SGD_nesterov_5step OK (max_abs=0.0e+00) optim/Adagrad_5step OK (max_abs=0.0e+00) optim/RMSprop_5step OK (max_abs=0.0e+00) optim/Adam_1step OK (max_abs=0.0e+00) optim/Adam_10step_wd OK (max_abs=0.0e+00) optim/AdamW_1step OK (max_abs=0.0e+00) optim/AdamW_10step_wd OK (max_abs=0.0e+00) optim/Lion_5step OK (max_abs=0.0e+00) optim/clip_grad_norm_clips OK (max_abs=0.0e+00) optim/clip_grad_norm_noop OK (max_abs=0.0e+00) optim/sched_cosine OK (max_abs=0.0e+00) optim/sched_warmup OK (max_abs=0.0e+00) optim/sched_step OK (max_abs=0.0e+00) optim/sched_exponential OK (max_abs=0.0e+00) ---------------------------------------------------------------------- optim 16/16 TOTAL 16/16 public checks passing All public checks pass. Now make sure model.npz clears the held-out test threshold, and remember the grader re-checks with DIFFERENT seeds/shapes.
/app/submission/data.py
def load_npz(path: str) -> Tuple[np.ndarray, Optional[np.ndarray]]:
# TODO: np.load the file; return (X as float64, y as int64 reshaped to 1-D or None if absent).
raise NotImplementedError("load_npz")
def n_classes(y: np.ndarray) -> int:
# TODO: return int(max(y)) + 1
raise NotImplementedError("n_classes")
def epoch_batches(n: int, batch_size: int, rng: np.random.Generator):
# TODO: yield successive slices of a single rng.permutation(n) of size batch_size.
raise NotImplementedError("epoch_batches")
def get_batch(X: np.ndarray, y: Optional[np.ndarray], idx: np.ndarray):
# TODO: gather X[idx] (float64) and y[idx] (int64, or None if y is None); return (Xb, yb, idx).
raise NotImplementedError("get_batch")def load_npz(path: str) -> Tuple[np.ndarray, Optional[np.ndarray]]:
d = np.load(path)
X = np.asarray(d["X"], dtype=np.float64)
y = np.asarray(d["y"], dtype=np.int64).reshape(-1) if "y" in d.files else None
return X, y
def n_classes(y: np.ndarray) -> int:
return int(np.max(y)) + 1
def epoch_batches(n: int, batch_size: int, rng: np.random.Generator):
perm = rng.permutation(n)
for start in range(0, n, batch_size):
yield perm[start:start + batch_size]
def get_batch(X: np.ndarray, y: Optional[np.ndarray], idx: np.ndarray):
idx = np.asarray(idx, dtype=np.int64)
Xb = np.asarray(X[idx], dtype=np.float64)
yb = np.asarray(y[idx], dtype=np.int64) if y is not None else None
return Xb, yb, idxThe file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/data.py
# TODO: build per-class templates, pick labels, return template[y] + noise and the labels.
raise NotImplementedError("make_image_dataset") rng = np.random.default_rng(seed)
templates = rng.standard_normal((n_classes, 1, img, img))
y = rng.integers(0, n_classes, size=n)
X = templates[y] + rng.standard_normal((n, 1, img, img)) * 0.5
return np.asarray(X, dtype=np.float64), np.asarray(y, dtype=np.int64)The file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/data.py
# TODO: sample (n, T) token ids in [0, vocab); label = (row sum) % n_classes.
raise NotImplementedError("make_seq_dataset") rng = np.random.default_rng(seed)
X = rng.integers(0, vocab, size=(n, T))
y = X.sum(axis=1) % n_classes
return np.asarray(X, dtype=np.int64), np.asarray(y, dtype=np.int64)The file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: return X + rng.normal(0, sigma, X.shape) (X as float64)
raise NotImplementedError("weak_augment") X = np.asarray(X, dtype=np.float64)
return X + rng.normal(0.0, sigma, X.shape)The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: add jitter, then for each row zero round(mask_frac*dim) randomly-chosen coordinates.
raise NotImplementedError("strong_augment") X = np.asarray(X, dtype=np.float64)
out = X + rng.normal(0.0, sigma, X.shape)
n, dim = out.shape
k = int(round(mask_frac * dim))
if k > 0:
for i in range(n):
cols = rng.choice(dim, size=k, replace=False)
out[i, cols] = 0.0
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: copy X; for each row pick a random start and zero [start:start+length].
raise NotImplementedError("cutout") out = np.asarray(X, dtype=np.float64).copy()
n, dim = out.shape
for i in range(n):
start = int(rng.integers(0, dim))
out[i, start:start + length] = 0.0
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: implement the convex combination described above; return (X_mix, y_mix, lam, perm).
raise NotImplementedError("mixup") X = np.asarray(X, dtype=np.float64)
onehot = np.asarray(onehot, dtype=np.float64)
lam = float(rng.beta(alpha, alpha))
lam = builtins_max(lam, 1.0 - lam)
n = X.shape[0]
perm = rng.permutation(n)
X_mix = lam * X + (1.0 - lam) * X[perm]
y_mix = lam * onehot + (1.0 - lam) * onehot[perm]
return X_mix, y_mix, lam, permThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: stack to (K, M, ...), swap the leading block axis with K over full blocks, return list.
raise NotImplementedError("interleave") arrs = [np.asarray(a) for a in arrays]
K = len(arrs)
stacked = np.stack(arrs, axis=0) # (K, M, *rest)
M = stacked.shape[1]
rest = stacked.shape[2:]
nb = M // batch_size
a = stacked.reshape((K, nb, batch_size) + rest)
b = a.swapaxes(0, 1) # (nb, K, batch_size, *rest)
out = b.reshape((K, M) + rest)
return [out[i] for i in range(K)]The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: invert the interleave permutation.
raise NotImplementedError("de_interleave") arrs = [np.asarray(a) for a in arrays]
K = len(arrs)
stacked = np.stack(arrs, axis=0) # (K, M, *rest)
M = stacked.shape[1]
rest = stacked.shape[2:]
nb = M // batch_size
c = stacked.reshape((nb, K, batch_size) + rest)
d = c.swapaxes(0, 1) # (K, nb, batch_size, *rest)
out = d.reshape((K, M) + rest)
return [out[i] for i in range(K)]The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: compute hard/conf/mask/onehot as above.
raise NotImplementedError("pseudo_label") weak_probs = np.asarray(weak_probs, dtype=np.float64)
n, c = weak_probs.shape
hard = weak_probs.argmax(axis=1)
conf = weak_probs.max(axis=1)
mask = (conf >= tau).astype(np.float64)
onehot = np.zeros((n, c), dtype=np.float64)
onehot[np.arange(n), hard] = 1.0
return onehot, mask, hardThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: return (probs.max(axis=1) >= tau).astype(float)
raise NotImplementedError("confidence_mask") probs = np.asarray(probs, dtype=np.float64)
return (probs.max(axis=1) >= tau).astype(np.float64)The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: pw = probs ** (1/temperature); return pw / pw.sum(axis=1, keepdims=True)
raise NotImplementedError("sharpen") probs = np.asarray(probs, dtype=np.float64)
pw = probs ** (1.0 / temperature)
return pw / pw.sum(axis=1, keepdims=True)The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: update self.value with the EMA recursion and return it.
raise NotImplementedError("EMAScalar.update") self.value = self.decay * self.value + (1.0 - self.decay) * float(new)
return self.valueThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: update self.running, rescale by prior/(running+1e-6), renormalize rows, return.
raise NotImplementedError("DistributionAligner.align") weak_probs = np.asarray(weak_probs, dtype=np.float64)
self.running = self.momentum * self.running + (1.0 - self.momentum) * weak_probs.mean(axis=0)
aligned = weak_probs * (self.prior / (self.running + 1e-6))
aligned = aligned / aligned.sum(axis=1, keepdims=True)
return alignedThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: compute sigma (per-class confident-argmax counts), set self.sigma, return beta*tau
# with denom = max(sigma.max(), C - sigma.sum()) (guard against 0).
raise NotImplementedError("FlexMatchThresholder.update")
def flex_mask(self, weak_probs) -> np.ndarray:
# TODO: thr = self.update(weak_probs); return (max prob per row >= thr[argmax]) as 0/1 float.
raise NotImplementedError("FlexMatchThresholder.flex_mask") weak_probs = np.asarray(weak_probs, dtype=np.float64)
hard = weak_probs.argmax(axis=1)
conf = weak_probs.max(axis=1)
adm = conf >= self.tau
sigma = np.array([float(np.sum(adm & (hard == c))) for c in range(self.C)], dtype=np.float64)
self.sigma = sigma
denom = builtins_max(float(sigma.max()), float(self.C - sigma.sum()))
denom = denom if denom > 0 else 1.0
return (sigma / denom) * self.tau
def flex_mask(self, weak_probs) -> np.ndarray:
weak_probs = np.asarray(weak_probs, dtype=np.float64)
thr = self.update(weak_probs)
hard = weak_probs.argmax(axis=1)
conf = weak_probs.max(axis=1)
return (conf >= thr[hard]).astype(np.float64)The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: clip(step/length, 0, 1) (handle length<=0 -> 1.0).
raise NotImplementedError("linear_rampup") if length <= 0:
return 1.0
return float(np.clip(step / length, 0.0, 1.0))The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: per the docstring (1.0 once step>=length).
raise NotImplementedError("cosine_rampup") if length <= 0:
return 1.0
x = float(np.clip(step / length, 0.0, 1.0))
return 0.5 * (1.0 - np.cos(np.pi * x))The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: per the docstring.
raise NotImplementedError("exp_rampup") if length <= 0:
return 1.0
x = float(np.clip(step / length, 0.0, 1.0))
return float(np.exp(-5.0 * (1.0 - x) ** 2))The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: build the (N,C) complementary one-hot on argmin for confident rows only.
raise NotImplementedError("complement_distribution") weak_probs = np.asarray(weak_probs, dtype=np.float64)
n, c = weak_probs.shape
conf = weak_probs.max(axis=1)
least = weak_probs.argmin(axis=1)
sel = conf >= tau
comp = np.zeros((n, c), dtype=np.float64)
comp[np.arange(n)[sel], least[sel]] = 1.0
return compThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward value via _stable_log_softmax_np; build out via _mk(val, (logits,)); set out._backward.
raise NotImplementedError("soft_cross_entropy") logp = _stable_log_softmax_np(x)
per_row = -(tgt * logp).sum(axis=1)
val = float((w * per_row).sum() / n)
out = _mk(val, (logits,))
def _bw():
if not logits.requires_grad:
return
sm = np.exp(logp)
tsum = tgt.sum(axis=1, keepdims=True)
grad = (w[:, None] / n) * (sm * tsum - tgt)
logits._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward (sum of squared diffs, weighted, /N); backward through the softmax (see docstring).
raise NotImplementedError("mse_consistency") s = _stable_softmax_np(x)
diff = s - tgt
val = float((w * (diff * diff).sum(axis=1)).sum() / n)
out = _mk(val, (student_logits,))
def _bw():
if not student_logits.requires_grad:
return
v = (2.0 * w[:, None] / n) * diff
grad = s * (v - (v * s).sum(axis=1, keepdims=True))
student_logits._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward (include the teacher-entropy term so KL is reported); backward as above.
raise NotImplementedError("kl_divergence") logp = _stable_log_softmax_np(x)
log_t = np.log(np.clip(tgt, 1e-12, None))
per_row = (tgt * (log_t - logp)).sum(axis=1)
val = float((w * per_row).sum() / n)
out = _mk(val, (student_logits,))
def _bw():
if not student_logits.requires_grad:
return
sm = np.exp(logp)
tsum = tgt.sum(axis=1, keepdims=True)
grad = (w[:, None] / n) * (sm * tsum - tgt)
student_logits._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward mean entropy; backward per the docstring.
raise NotImplementedError("entropy_loss") logp = _stable_log_softmax_np(x)
p = np.exp(logp)
H = -(p * logp).sum(axis=1) # (N,)
val = float(H.sum() / n)
out = _mk(val, (logits,))
def _bw():
if not logits.requires_grad:
return
grad = (1.0 / n) * p * (-logp - H[:, None])
logits._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward (-H of the batch marginal); backward per the docstring.
raise NotImplementedError("class_balance_loss") p = _stable_softmax_np(x)
pbar = p.mean(axis=0) # (C,)
val = float((pbar * np.log(np.clip(pbar, 1e-12, None))).sum())
out = _mk(val, (logits,))
def _bw():
if not logits.requires_grad:
return
a = 1.0 + np.log(np.clip(pbar, 1e-12, None)) # (C,)
ap = (p * a[None, :]).sum(axis=1, keepdims=True)
grad = (1.0 / n) * p * (a[None, :] - ap)
logits._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: stable sigmoid-BCE forward via _mk(val, (z,)); backward (w/N)*(sigmoid(z)-t).
raise NotImplementedError("bce_consistency") per_elem = np.maximum(x, 0.0) - x * t + np.log1p(np.exp(-np.abs(x)))
val = float((w * per_elem.sum(axis=1)).sum() / n)
out = _mk(val, (z,))
def _bw():
if not z.requires_grad:
return
s = 1.0 / (1.0 + np.exp(-x))
grad = (w[:, None] / n) * (s - t)
z._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward sum-squared-diff averaged over N; backward (2w/N)*(x-t).
raise NotImplementedError("pi_consistency") diff = x - t
val = float((w * (diff * diff).sum(axis=1)).sum() / n)
out = _mk(val, (z,))
def _bw():
if not z.requires_grad:
return
grad = (2.0 * w[:, None] / n) * diff
z._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward via _stable_log_softmax_np; backward through softmax per the docstring.
raise NotImplementedError("focal_consistency") logp = _stable_log_softmax_np(x)
p = np.exp(logp)
neglog = -logp
om = 1.0 - p
om_g = om ** gm
phi = t * om_g * neglog
val = float((w * phi.sum(axis=1)).sum() / n)
out = _mk(val, (z,))
def _bw():
if not z.requires_grad:
return
om_gm1 = om ** (gm - 1.0)
u = t * (-gm * om_gm1 * neglog - om_g / p)
up = (u * p).sum(axis=1, keepdims=True)
grad = (w[:, None] / n) * p * (u - up)
z._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward JS (clip probs to 1e-12 before log); backward through softmax per the docstring.
raise NotImplementedError("js_divergence") p = _stable_softmax_np(x)
m = 0.5 * (p + t)
pc = np.clip(p, 1e-12, None)
tc = np.clip(t, 1e-12, None)
mc = np.clip(m, 1e-12, None)
js = 0.5 * (p * (np.log(pc) - np.log(mc))).sum(axis=1) \
+ 0.5 * (t * (np.log(tc) - np.log(mc))).sum(axis=1)
val = float((w * js).sum() / n)
out = _mk(val, (z,))
def _bw():
if not z.requires_grad:
return
u = (w[:, None] / n) * 0.5 * (np.log(pc) - np.log(mc))
up = (u * p).sum(axis=1, keepdims=True)
grad = p * (u - up)
z._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: forward -(comp*log(1-p)) (clip 1-p to 1e-12); backward through softmax per the docstring.
raise NotImplementedError("negative_consistency") p = _stable_softmax_np(x)
om = np.clip(1.0 - p, 1e-12, None)
val = float(-(w * (comp * np.log(om)).sum(axis=1)).sum() / n)
out = _mk(val, (z,))
def _bw():
if not z.requires_grad:
return
u = comp / om
up = (u * p).sum(axis=1, keepdims=True)
grad = (w[:, None] / n) * p * (u - up)
z._accum(out.grad * grad)
out._backward = _bw
return outThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: onehot, mask = pseudo_label(...); loss = soft_cross_entropy(...); scale by lambda_u.
raise NotImplementedError("consistency_loss") weak_probs = np.asarray(weak_probs, dtype=np.float64)
onehot, mask, _ = pseudo_label(weak_probs, tau)
loss = soft_cross_entropy(strong_logits, onehot, weight=mask)
if lambda_u != 1.0:
loss = ag.mul(loss, lambda_u)
mask_rate = float(mask.mean())
return loss, mask_rateThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
# TODO: sup = ag.cross_entropy(...); cons, mr = consistency_loss(...); return (ag.add(sup, cons), mr).
raise NotImplementedError("fixmatch_loss") sup = ag.cross_entropy(sup_logits, sup_y)
cons, mr = consistency_loss(strong_logits, weak_probs, tau, lambda_u=lambda_u)
return ag.add(sup, cons), mrThe file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/ssl.py
def update(self, model):
# TODO: for each named param, shadow <- decay*shadow + (1-decay)*param.data (in place).
raise NotImplementedError("EMATeacher.update")
def copy_to(self, model):
# TODO: write self.shadow[k] into each model param's data.
raise NotImplementedError("EMATeacher.copy_to") def update(self, model):
d = self.decay
for k, v in model.named_params().items():
self.shadow[k] = d * self.shadow[k] + (1.0 - d) * v.data
def copy_to(self, model):
for k, v in model.named_params().items():
v.data[...] = self.shadow[k]The file /app/submission/ssl.py has been updated successfully. (file state is current in your context , no need to Read it back)
Run fixmatch checks
python3 run_checks.py --only fixmatch 2>&1 | tail -40
[fixmatch] 31/31 fixmatch/sce_bwd_onehot OK (max_abs=2.72e-11) fixmatch/sce_bwd_soft OK (max_abs=3.07e-11) fixmatch/mse_consistency_bwd OK (max_abs=2.91e-12) fixmatch/kl_divergence_bwd OK (max_abs=1.02e-11) fixmatch/entropy_loss_bwd OK (max_abs=1.93e-11) fixmatch/class_balance_bwd OK (max_abs=2.09e-11) fixmatch/bce_consistency_bwd OK (max_abs=9.43e-12) fixmatch/pi_consistency_bwd OK (max_abs=8.00e-11) fixmatch/focal_bwd_onehot OK (max_abs=8.93e-12) fixmatch/focal_bwd_soft OK (max_abs=1.15e-11) fixmatch/js_divergence_bwd OK (max_abs=1.39e-12) fixmatch/negative_consistency_bwd OK (max_abs=7.06e-12) fixmatch/flexmatch_threshold OK fixmatch/rampup_schedules OK fixmatch/complement_distribution OK fixmatch/cons_bwd_tau0.3 OK (max_abs=2.06e-11) fixmatch/cons_bwd_tau0.5 OK (max_abs=0.00e+00) fixmatch/cons_mask_all_zero OK fixmatch/pseudo_label_gate OK fixmatch/pseudo_label_boundary OK fixmatch/confidence_mask OK fixmatch/sharpen_T0.5 OK fixmatch/sharpen_T2.0 OK fixmatch/sharpen_identity OK fixmatch/mixup_convex OK (lam=0.904) fixmatch/interleave_roundtrip OK fixmatch/da_recursion OK …[truncated 443 chars]
Run full check suite
python3 run_checks.py --quiet 2>&1 | tail -6
---------------------------------------------------------------------- autograd_grads 77/77 | nn_forward 20/20 | optim 16/16 | fixmatch 31/31 | integration 2/2 | data 4/4 TOTAL 150/150 public checks passing All public checks pass. Now make sure model.npz clears the held-out test threshold, and remember the grader re-checks with DIFFERENT seeds/shapes.
Inspect data structure
python3 -c "
import numpy as np
d=np.load('data/labeled.npz'); X,y=d['X'],d['y']
u=np.load('data/unlabeled.npz')['X']
# quick supervised-only baseline idea: check class separability via nearest centroid on raw 80-dim
from collections import Counter
print('labeled', X.shape, 'unlabeled', u.shape)
# PCA to see signal subspace
allX=np.vstack([X,u])
allX=allX-allX.mean(0)
cov=allX.T@allX/len(allX)
w,_=np.linalg.eigh(cov)
print('top eigs', w[-12:][::-1])
"labeled (200, 80) unlabeled (6000, 80) top eigs [1.3897966 1.3660743 1.3534957 1.3339317 1.3317931 1.3191792 1.3178654 1.3113981 1.3056706 1.2844936 1.2825733 1.269397 ]
/app/submission/train.py
in_dim = Xl.shape[1]
C = int(yl.max()) + 1
# TODO: build the model + optimizer + EMA teacher (+ distribution aligner), then run the
# FixMatch training loop (supervised CE + masked consistency with confidence-gated, optionally
# distribution-aligned pseudo-labels), and save_checkpoint(out_path, ema.state_dict(), cfg).
raise NotImplementedError("train") in_dim = Xl.shape[1]
C = int(yl.max()) + 1
Nl = Xl.shape[0]
Nu = Xu.shape[0]
rng = np.random.default_rng(seed)
aug = np.random.default_rng(seed + 12345)
model = MLPClassifier(in_dim, C, d_hidden=D_HIDDEN, n_layer=N_LAYER, seed=seed)
opt = O.AdamW(model.params(), lr=3e-3, betas=(0.9, 0.999), weight_decay=5e-4)
ema = S.EMATeacher(model, decay=0.999)
aligner = S.DistributionAligner(C, momentum=0.999)
tau = 0.95
lambda_u_max = 1.0
base_lr = 3e-3
warmup = 200
rampup_len = max(1, steps // 2)
lab_bs = 64
unlab_bs = 384
def sample(n, bs):
return rng.integers(0, n, size=bs)
for step in range(steps):
lr = lr_at(step, steps, base_lr, warmup)
opt.lr = lr
lam = lambda_u_max * S.linear_rampup(step, rampup_len)
# supervised: weak-augmented labeled batch
li = sample(Nl, lab_bs)
Xl_w = S.weak_augment(Xl[li], aug, sigma=0.1)
sup_logits = model(Xl_w)
sup_loss = ag.cross_entropy(sup_logits, yl[li])
opt.zero_grad()
if lam > 0.0:
ui = sample(Nu, unlab_bs)
Xu_b = Xu[ui]
# weak view -> detached pseudo-labels (distribution aligned)
wp = S._stable_softmax_np(np.asarray(model(S.weak_augment(Xu_b, aug, sigma=0.1)).data,
dtype=np.float64))
wp = aligner.align(wp)
strong_logits = model(S.strong_augment(Xu_b, aug, sigma=
…[truncated 811 chars]The file /app/submission/train.py has been updated successfully. (file state is current in your context , no need to Read it back)
Proxy evaluation supervised vs fixmatch
cd /app/submission && cat > /tmp/proxy.py <<'PY'
import sys, numpy as np
sys.path.insert(0,'/app/submission')
import autograd as ag, optim as O, ssl as S
from nn import MLPClassifier
d=np.load('/app/data/labeled.npz'); X=np.asarray(d['X'],float); y=np.asarray(d['y']).reshape(-1)
Xu=np.asarray(np.load('/app/data/unlabeled.npz')['X'],float)
C=int(y.max())+1; in_dim=X.shape[1]
rng=np.random.default_rng(0)
# stratified split: 14 train, 6 test per class
tr_idx=[]; te_idx=[]
for c in range(C):
ic=np.where(y==c)[0]; rng.shuffle(ic)
tr_idx+=ic[:14].tolist(); te_idx+=ic[14:].tolist()
tr_idx=np.array(tr_idx); te_idx=np.array(te_idx)
Xtr,ytr=X[tr_idx],y[tr_idx]; Xte,yte=X[te_idx],y[te_idx]
def evalacc(model,Xt,yt):
lg=model(Xt).data; return float((lg.argmax(1)==yt).mean())
def run(steps, use_ssl=True, seed=1):
rng=np.random.default_rng(seed); aug=np.random.default_rng(seed+99)
m=MLPClassifier(in_dim,C,d_hidden=256,n_layer=3,seed=seed)
opt=O.AdamW(m.params(),lr=3e-3,weight_decay=5e-4)
ema=S.EMATeacher(m,decay=0.999); al=S.DistributionAligner(C,momentum=0.999)
Nl=len(Xtr); Nu=len(Xu); tau=0.95
for step in range(steps):
lr=0.5*3e-3*(1+np.cos(np.pi*min(1,step/steps))) if step>=200 else 3e-3*(step+1)/200
opt.lr=lr; lam=S.linear_rampup(step,steps//2) if use_ssl else 0.0
li=rng.integers(0,Nl,64)
sl=m(S.weak_augment(Xtr[li],aug,0.1)); sup=ag.cross_entropy(sl,ytr[li])
opt.zero_grad()
if lam>0:
ui=r
…[truncated 752 chars]No file changes, the agent left the workspace untouched.
=== SyncVals verifier (local re-grade of the agent's own checkpoint) === COMPONENT CONFORMANCE SUITE: 197/197 cases passed (all milestones pass: True) FINAL METRIC: held-out worst-class top-1 = 0.8478 >= threshold 0.72 REWARD = 1 (1 iff every component milestone passes AND the sealed held-out metric clears its threshold) NOTE: the in-sandbox training in the 481MB e2b box was OOM-killed (0-byte checkpoint), so the raw in-sandbox reward was an infrastructure artifact. This reward is from a faithful LOCAL re-train of the agent's OWN train.py (24GB, no OOM) with its own default hyperparameters, then re-graded by the real verifier.
Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_6bd56c6f9b6a4962. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.
Trial trial_6bd56c6f9b6a4962 · verifier authoritative; classifier explanatory.