tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.ls -la && echo "---SUBMISSION---" && ls -la submission/ && echo "---DATA---" && ls -la data/ && python3 -c "import numpy as np; d=np.load('data/train.npz'); print({k:(v.shape,v.dtype) for k,v in d.items()}); print('y range', d['y'].min(), d['y'].max(), 'X range', d['X'].min(), d['X'].max())"total 101
drwxr-xr-x 4 user user 4096 Jun 30 22:27 .
drwxr-xr-x 21 root root 4096 Jun 30 22:27 ..
-rw-r--r-- 1 user user 14238 Jun 30 22:27 PROMPT.txt
-rw-r--r-- 1 user user 5 Jun 30 22:27 agent.pid
-rw-r--r-- 1 user user 0 Jun 30 22:27 agent.stderr
-rw-r--r-- 1 user user 4688 Jun 30 22:27 agent_trajectory.jsonl
drwxr-xr-x 2 user user 60 Jun 30 22:27 data
-rw-r--r-- 1 user user 0 Jun 30 22:27 run.log
-rwxr-xr-x 1 user user 57740 Jun 30 11:01 run_checks.py
drwxr-xr-x 2 user user 128 Jun 30 22:27 submission
-rwxr-xr-x 1 user user 998 Jun 30 11:01 test.sh
-rwxr-xr-x 1 user user 629 Jun 30 11:01 timer.sh
---SUBMISSION---
total 80
drwxr-xr-x 2 user user 128 Jun 30 22:27 .
drwxr-xr-x 4 user user 4096 Jun 30 22:27 ..
-rw-r--r-- 1 user user 5490 Jun 30 11:01 advtrain.py
-rw-r--r-- 1 user user 8776 Jun 30 11:01 attack.py
-rw-r--r-- 1 user user 18341 Jun 30 11:01 autograd.py
-rw-r--r-- 1 user user 2167 Jun 30 11:01 data.py
-rw-r--r-- 1 user user 15988 Jun 30 11:01 nn.py
-rw-r--r-- 1 user user 8853 Jun 30 11:01 optim.py
-rw-r--r-- 1 user user 3742 Jun 30 11:01 train.py
---DATA---
total 2904
drwxr-xr-x 2 user user 60 Jun 30 22:27 .
drwxr-xr-x 4 user user 4096 Jun 30 22:27 ..
-rw-r--r-- 1 user user 2967336 Jun 30 11:01 train.npz
{'X': ((3600, 1, 16, 16), dtype('float32')), 'y': ((3600,), dtype('int64'))}
y range 0 5 X range 0.0 1.0/app/submission/autograd.py
/app/submission/nn.py
1 """Reverse-mode automatic differentiation over numpy arrays (IMPLEMENT THIS). 2 3 A ``Tensor`` wraps a single ``numpy.ndarray`` (``float64``) and records the operation that 4 produced it so that ``.backward()`` can walk the graph in reverse-topological order and 5 accumulate ``.grad`` (the gradient of the scalar on which ``.backward()`` was called w.r.t. 6 this tensor) into every leaf. 7 8 This task is ADVERSARIAL ROBUSTNESS: the model is a small convolutional classifier and the 9 attacks (PGD/FGSM/MI-FGSM/PGD-L2/CW) ascend on the gradient of an attack loss w.r.t. the INPUT 10 image, so the autograd engine must (a) flow gradients back to PARAMETERS (for training) AND 11 (b) flow gradients back to the INPUT tensor (for the attack). BOTH directions are exercised by the 12 verifier's finite-difference gradient checks, so the convolution's input-gradient (the ``_col2im`` 13 adjoint) is load-bearing. 14 15 Contract (the verifier's conformance suite differentiates your analytic backward against finite 16 differences and checks your forward against its own reference, so match this exactly): 17 18 * ``Tensor(data, requires_grad=False)`` -- ``data`` is array-like, stored as ``float64`` in 19 ``self.data``; ``self.grad`` starts as ``None`` and becomes a same-shape ``float64`` array after 20 backward. ``requires_grad`` leaves accumulate grad; non-requiring leaves do not. 21 * Every op returns a NEW ``Tensor`` whose ``requires_grad`` is the OR of its differentiable inputs. 22 …[truncated 18629 chars]
1 """Neural-network layers + a small convolutional classifier, built on the autograd engine 2 (IMPLEMENT THE FORWARDS). 3 4 Every module exposes ``params()`` -> list[Tensor] (the leaf tensors with ``requires_grad=True``) 5 and ``named_params()`` -> dict[str, Tensor] with STABLE names that match the checkpoint format the 6 verifier loads. ``SmallCNN.state_dict()`` / ``load_state_dict()`` round-trip a plain 7 ``dict[str, np.ndarray]`` (saved as an ``.npz``); the verifier loads YOUR checkpoint into its OWN 8 reference SmallCNN via these names, so the names + shapes below are part of the contract. 9 10 Naming contract for SmallCNN (do not change -- the verifier indexes by these exact keys): 11 conv1.weight (C1, C_in, 3, 3) conv1.bias (C1,) 12 conv2.weight (C2, C1, 3, 3) conv2.bias (C2,) 13 fc1.weight (C2*(H/4)*(W/4), F) fc1.bias (F,) 14 fc2.weight (F, n_classes) fc2.bias (n_classes,) 15 16 Architecture (pinned): for an (N, C_in, H, W) input image batch, 17 conv1 (C_in->C1, 3x3, pad 1) -> relu 18 maxpool 2x2 (H,W -> H/2,W/2) 19 conv2 (C1->C2, 3x3, pad 1) -> relu 20 maxpool 2x2 (H/2,W/2 -> H/4,W/4) 21 flatten -> fc1 (-> F) -> relu -> fc2 (-> n_classes) logits. 22 ``Linear`` computes ``x @ weight + bias`` (weight is (in, out)). No BatchNorm inside SmallCNN (so 23 the attack runs a single deterministic eval-mode forward with no running-stat dependence). 24 25 The other …[truncated 15896 chars]
/app/submission/optim.py
1 """Optimizers + LR schedulers over a flat list of autograd ``Tensor`` parameters (IMPLEMENT THESE). 2 3 Every optimizer operates IN PLACE on ``param.data`` using ``param.grad`` (the gradient accumulated 4 by ``Tensor.backward``). ``zero_grad()`` resets every grad to None. The EXACT update formula each 5 optimizer must match (the verifier checks one and several steps against its own reference) is in 6 its docstring. 7 8 The adversarial-training loop uses AdamW (decoupled decay, with ``no_decay_ndim`` excluding 1-D 9 params/biases) + global grad-norm clipping; the rest of the optimizer zoo + the four LR schedulers 10 are part of the from-scratch framework the conformance suite audits. 11 """ 12 from __future__ import annotations 13 14 import math 15 from typing import List 16 17 import numpy as np 18 19 from autograd import Tensor 20 21 22 class Optimizer: 23 def __init__(self, params: List[Tensor]): 24 self.params = list(params) 25 26 def zero_grad(self): 27 for p in self.params: 28 p.grad = None 29 30 31 class SGD(Optimizer): 32 """SGD + momentum + optional Nesterov + (coupled/L2) weight decay: 33 g = grad + wd*param; v = mu*v + g; step = (g + mu*v) if nesterov else v; param -= lr*step.""" 34 35 def __init__(self, params, lr=0.1, momentum=0.0, weight_decay=0.0, nesterov=False): 36 super().__init__(params) 37 self.lr = lr 38 self.momentum = momentum 39 self.weight_decay = weight …[truncated 8125 chars]
/app/submission/data.py
1 """Data loading + minibatching for the shape-classification set (IMPLEMENT THESE).
2
3 The visible train split ships as ``/app/data/train.npz`` with arrays ``X`` (N, C, H, W) float
4 pixels in [0,1] and ``y`` (N,) int labels. The SEALED held-out split is NOT in the image -- the
5 verifier regenerates it from a secret feature seed and runs its own attack on it.
6
7 Contract:
8 load_npz(path) -> (X float64 (N,C,H,W), y int64 (N,))
9 train_val_split(X, y, val_frac, rng) -> ((Xtr,ytr),(Xval,yval)) random disjoint split (val slice
10 is a held-BACK slice of the VISIBLE train data you may use for best-checkpoint selection --
11 it is NOT the verifier's sealed held-out set)
12 iter_batches(X, y, batch_size, rng) -> yields (xb, yb) random-permutation minibatches
13 get_batch(X, y, batch_size, rng) -> a single random minibatch
14 to_onehot(y, n_classes) -> (N, n_classes) one-hot float matrix
15 clip01(X) -> X clamped into the [0,1] pixel domain
16 """
17 from __future__ import annotations
18
19 from typing import Iterator, Tuple
20
21 import numpy as np
22
23
24 def load_npz(path: str) -> Tuple[np.ndarray, np.ndarray]:
25 raise NotImplementedError("implement load_npz")
26
27
28 def train_val_split(X: np.ndarray, y: np.ndarray, val_frac: float,
29 rng: np.random.Generator):
30 raise NotImplementedError("implement train_val_split")
31
32
33 def iter_batches(X: np.ndarray, y: np.ndarray, batch_size: int,
34
…[truncated 814 chars]/app/submission/attack.py
/app/submission/advtrain.py
1 """White-box evasion attacks built on the from-scratch autograd engine (IMPLEMENT THESE). 2 3 An adversary perturbs each input within a norm budget (L-inf eps in the [0,1] pixel domain by 4 default) and tries to make the classifier WRONG. Every attack ascends on the gradient of an attack 5 loss with respect to the INPUT image, which is exactly the gradient the autograd engine produces 6 when the input is wrapped in a ``requires_grad=True`` Tensor and the loss is backpropagated -- so a 7 correct ``conv2d`` input-gradient (the ``_col2im`` adjoint) is load-bearing. 8 9 THE PRIMITIVE 10 input_grad(model, x, y) d/dx of mean cross-entropy of model(x) vs y (ndarray). 11 loss_input_grad(model, x, y, loss_fn) d/dx of an ARBITRARY scalar attack loss loss_fn(logits,y). 12 13 L-INF ATTACKS 14 fgsm(model, x, y, eps) clip(x + eps*sign(input_grad)). 15 pgd_attack(model, x, y, eps, steps, alpha) iterated FGSM with random start + project to the 16 L-inf eps-ball around x AND the [0,1] box each step. 17 mi_fgsm(model, x, y, eps, steps, alpha, mu) momentum-iterative FGSM: accumulate a decaying 18 momentum of the L1-NORMALIZED gradient, step on its sign, 19 project each step. 20 targeted_pgd(model, x, y_target, eps, steps, alpha) DESCEND CE toward y_target (step on the 21 …[truncated 7820 chars]
1 """Adversarial-training objectives built on the from-scratch autograd engine (IMPLEMENT THESE).
2
3 Several standard recipes turn a fragile classifier into a robust one. Each crafts adversarial
4 inputs per minibatch with the model's own attack, then takes a gradient step on a robustness-aware
5 loss; the recipes differ in the loss. All run a single forward+backward; the caller does grad-clip
6 + the optimizer step. ``x_clean`` / ``x_adv`` are ndarrays (N, C, H, W); ``y`` is an int (N,).
7
8 pgd_at_loss(model, x_adv, y) Madry PGD-AT: CE on the ADVERSARIAL batch.
9 trades_loss(model, x_clean, x_adv, y, beta) TRADES: CE(clean) + beta*KL(stopgrad(clean)||adv).
10 The clean branch INSIDE the KL is a STOP-GRADIENT
11 target (detached): the KL backprops only through
12 the adversarial branch.
13 mart_loss(model, x_clean, x_adv, y, beta) MART: boosted-CE on the adversarial batch
14 ( CE(adv,y) - mean log(1 - max_{j!=y} p_adv_j) ) +
15 beta * mean_i[ (1 - p_clean_i[y_i]) *
16 KL_i(softmax(adv)||stopgrad softmax(clean)) ].
17 alp_loss(model, x_clean, x_adv, y, lam) Adversarial Logit Pairing:
18 0.5*(CE(clean)+CE(ad
…[truncated 4278 chars]/app/submission/train.py
/app/test.sh
1 """Adversarial-training loop for the small CNN, using the from-scratch numpy framework 2 (IMPLEMENT THIS). 3 4 Standard (clean) training reaches ~99% CLEAN accuracy but ~chance ROBUST accuracy -- it is broken 5 by the verifier's STRONG multi-restart L-inf PGD attack. WEAK robust training is not enough either: 6 single-step (FGSM) adversarial training suffers "catastrophic overfitting" and is shattered by the 7 multi-step / multi-restart eval attack, and adversarial training at a SMALLER perturbation budget 8 than the eval budget leaves the model fragile exactly where it is attacked. The effective recipe is 9 full multi-step PGD ADVERSARIAL TRAINING (Madry et al. 2018) with the perturbation budget MATCHED 10 to the (larger) eval budget: 11 12 * on every minibatch, craft an inner k-step L-inf PGD adversarial batch against the CURRENT model 13 (attack.pgd_attack), at the eps the eval uses, then take the AdamW step on advtrain.pgd_at_loss 14 (cross-entropy on the adversarial batch); 15 * train long enough for robustness to converge (a few epochs is not enough at this eps); 16 * stability details that materially raise the achieved robust accuracy and make it reproducible: 17 (a) exclude biases (1-D params) from weight decay (AdamW no_decay_ndim=1); (b) clip the global 18 gradient norm (the adversarial loss surface is sharp); (c) keep the BEST-by-robust-accuracy 19 checkpoint, evaluated each epoch with the model's own multi-step PGD on a held-BACK slic …[truncated 2473 chars]
1 #!/usr/bin/env bash
2 # Smoke test (NOT the grader): confirms the environment is wired -- numpy imports, the training
3 # split loads, and the submission stubs are present. It does NOT check correctness (the stubs raise
4 # NotImplementedError until you implement them).
5 set -uo pipefail
6
7 python3 - <<'PY'
8 import numpy as np, os
9 p = "/app/data/train.npz"
10 assert os.path.exists(p), "missing train.npz"
11 d = np.load(p)
12 print(f"train.npz: X{d['X'].shape} {d['X'].dtype} y{d['y'].shape} classes={len(set(d['y'].tolist()))}")
13 for f in ("autograd.py", "nn.py", "optim.py", "data.py", "attack.py", "advtrain.py", "train.py"):
14 assert os.path.exists(f"/app/submission/{f}"), f"missing /app/submission/{f}"
15 assert os.path.exists("/app/run_checks.py"), "missing /app/run_checks.py (public component runner)"
16 print("submission stubs present:", sorted(os.listdir("/app/submission")))
17 print("numpy", np.__version__)
18 PY
19 echo "app_test smoke OK (run 'python3 /app/run_checks.py' for the per-check progress report)"
20/app/run_checks.py
<system-reminder>[Truncated: PARTIAL view , showing lines 1-868 of 1188 total (29062 tokens, cap 25000). Call Read with offset=869 limit=868 for the next page, or Grep to find a specific section. Do NOT answer from this page alone if the answer may be further in the file.]</system-reminder> 1 #!/usr/bin/env python3 2 """PUBLIC component check runner for the from-scratch numpy framework + the adversarial machinery. 3 4 Run this AS OFTEN AS YOU LIKE while you implement to get a per-check progress signal: 5 6 python3 /app/run_checks.py # check the framework in /app/submission 7 python3 /app/run_checks.py --submission /some/dir 8 python3 /app/run_checks.py --only attack # focus a single milestone 9 10 It imports YOUR modules from /app/submission (autograd.py / nn.py / optim.py / data.py / attack.py 11 / advtrain.py), exactly as the grader does, and runs a PUBLIC portion of the component conformance 12 suite, printing a clear per-check report. Each FAIL line names the exact op / layer / optimizer / 13 attack / adversarial-training property that is wrong. 14 15 HOW THIS RELATES TO YOUR GRADE 16 ------------------------------ 17 This runner is a DEBUGGING AID, not the grader. It checks YOUR framework against self-consistent 18 references it computes on its OWN: 19 * autograd_grads : central FINITE-DIFFERENCE of YOUR forward vs YOUR analytic backward (every op, 20 incl. the convolution input-gradient, the pools, an …[truncated 44212 chars]
/app/run_checks.py
869 def dlr_loss_grad():
870 rng = np.random.default_rng(sb + 60)
871 x0 = rng.standard_normal((5, 4)); y = rng.integers(0, 4, size=5)
872 lt = ag.Tensor(x0.copy(), requires_grad=True)
873 atk.dlr_loss(lt, y).backward()
874 gnum = _num_grad_input(lambda xv: np.array(float(atk.dlr_loss(ag.Tensor(xv), y).data)), x0.copy())
875 return _max_abs(lt.grad, gnum) < TOL_GRAD, f"max_abs={_max_abs(lt.grad, gnum):.2e}"
876
877 def dlr_input_grad_fd():
878 m = _model(61)
879 x0 = np.random.default_rng(sb + 62).standard_normal((3, 1, 8, 8)); y = np.array([0, 1, 2])
880 lf = lambda lg, t: atk.dlr_loss(lg, t)
881 gan = np.asarray(atk.loss_input_grad(m, x0, y, lf), np.float64)
882 gnum = _num_grad_input(lambda xv: np.array(float(lf(m(ag.Tensor(xv)), y).data)), x0.copy())
883 return _max_abs(gan, gnum) < TOL_GRAD, f"max_abs={_max_abs(gan, gnum):.2e}"
884
885 def bim_stronger():
886 m = _model(67)
887 x = np.clip(np.random.default_rng(sb + 68).random((8, 1, 8, 8)), 0, 1); y = np.array([0, 1, 2, 3, 0, 1, 2, 3]); eps = 0.12
888 xf = atk.fgsm(m, x, y, eps); xb = atk.bim(m, x, y, eps, 15, 0.025)
889 lf = float(ag.cross_entropy(m(xf), y).data); lb = float(ag.cross_entropy(m(xb), y).data)
890 return lb >= lf - 1e-6, f"fgsm={lf:.3f} bim={lb:.3f}"
891
892 def restarts_worst_case():
893 m = _model(69)
894 x = np.clip(np.random.default_
…[truncated 15653 chars]"""Reverse-mode automatic differentiation over numpy arrays."""
from __future__ import annotations
import math
from typing import Callable, Iterable, List, Optional
import numpy as np
_erf = np.vectorize(math.erf)
def _as_f64(x) -> np.ndarray:
return np.asarray(x, dtype=np.float64)
def _unbroadcast(grad: np.ndarray, shape: tuple) -> np.ndarray:
grad = np.asarray(grad, dtype=np.float64)
while grad.ndim > len(shape):
grad = grad.sum(axis=0)
for i, s in enumerate(shape):
if s == 1 and grad.shape[i] != 1:
grad = grad.sum(axis=i, keepdims=True)
return grad
class Tensor:
__slots__ = ("data", "grad", "requires_grad", "_backward", "_parents")
def __init__(self, data, requires_grad: bool = False, _parents: Iterable["Tensor"] = ()):
self.data: np.ndarray = _as_f64(data)
self.grad: Optional[np.ndarray] = None
self.requires_grad: bool = bool(requires_grad)
self._backward: Callable[[], None] = lambda: None
self._parents: tuple = tuple(_parents)
@property
def shape(self):
return self.data.shape
@property
def ndim(self):
return self.data.ndim
def zero_grad(self):
self.grad = None
def _accum(self, g: np.ndarray):
if self.grad is None:
self.grad = np.zeros_like(self.data)
self.grad += g
def backward(self, grad: Optional[np.ndarray] = None):
topo: List["Tensor"] = []
visited = set()
…[truncated 30274 chars]The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 -c "import sys; sys.path.insert(0,'submission'); import autograd" 2>&1 | head -20
(Bash completed with no output)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.conv2d(x, weight, bias, pad)."""
raise NotImplementedError("implement Conv2d.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.conv2d(x, weight, bias, pad)."""
return ag.conv2d(x, self.weight, self.bias, pad=self.pad)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.maxpool2d(x, k)."""
raise NotImplementedError("implement MaxPool2d.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.maxpool2d(x, k)."""
return ag.maxpool2d(x, self.k)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.avgpool2d(x, k)."""
raise NotImplementedError("implement AvgPool2d.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.avgpool2d(x, k)."""
return ag.avgpool2d(x, self.k)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.matmul(x, weight) + bias."""
raise NotImplementedError("implement Linear.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.matmul(x, weight) + bias."""
return ag.add(ag.matmul(x, self.weight), self.bias)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.layernorm(x, weight, bias, eps)."""
raise NotImplementedError("implement LayerNorm.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.layernorm(x, weight, bias, eps)."""
return ag.layernorm(x, self.weight, self.bias, self.eps)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.batchnorm(x, weight, bias, eps)."""
raise NotImplementedError("implement BatchNorm1d.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.batchnorm(x, weight, bias, eps)."""
return ag.batchnorm(x, self.weight, self.bias, self.eps)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x):
"""linear -> relu (between layers) -> ... -> linear (no final activation)."""
raise NotImplementedError("implement MLP.forward") def forward(self, x):
"""linear -> relu (between layers) -> ... -> linear (no final activation)."""
x = x if isinstance(x, Tensor) else Tensor(x)
n = len(self.layers)
for i, lin in enumerate(self.layers):
x = lin(x)
if i < n - 1:
x = ag.relu(x)
return xThe file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x):
"""for each conv: pool(relu(conv(x))); then flatten -> relu(fc1) -> fc2."""
raise NotImplementedError("implement ConvNet.forward") def forward(self, x):
"""for each conv: pool(relu(conv(x))); then flatten -> relu(fc1) -> fc2."""
x = x if isinstance(x, Tensor) else Tensor(x)
for conv in self.convs:
x = self.pool(ag.relu(conv(x)))
N = x.shape[0]
x = ag.reshape(x, (N, -1))
x = ag.relu(self.fc1(x))
return self.fc2(x)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.groupnorm(x, weight, bias, num_groups, eps)."""
raise NotImplementedError("implement GroupNorm.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.groupnorm(x, weight, bias, num_groups, eps)."""
return ag.groupnorm(x, self.weight, self.bias, self.num_groups, self.eps)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor) -> Tensor:
"""ag.rmsnorm(x, weight, eps)."""
raise NotImplementedError("implement RMSNorm.forward") def forward(self, x: Tensor) -> Tensor:
"""ag.rmsnorm(x, weight, eps)."""
return ag.rmsnorm(x, self.weight, self.eps)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor, h: Tensor) -> Tensor:
"""h' = tanh(x @ weight_ih^T + bias_ih + h @ weight_hh^T + bias_hh)."""
raise NotImplementedError("implement RNNCell.forward") def forward(self, x: Tensor, h: Tensor) -> Tensor:
"""h' = tanh(x @ weight_ih^T + bias_ih + h @ weight_hh^T + bias_hh)."""
x = x if isinstance(x, Tensor) else Tensor(x)
h = h if isinstance(h, Tensor) else Tensor(h)
pre = ag.add(ag.add(ag.matmul(x, ag.transpose(self.weight_ih)), self.bias_ih),
ag.add(ag.matmul(h, ag.transpose(self.weight_hh)), self.bias_hh))
return ag.tanh(pre)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor, h: Tensor) -> Tensor:
"""the GRU update above (slice the 3 gates out of the (N,3H) pre-activations)."""
raise NotImplementedError("implement GRUCell.forward") def forward(self, x: Tensor, h: Tensor) -> Tensor:
"""the GRU update above (slice the 3 gates out of the (N,3H) pre-activations)."""
x = x if isinstance(x, Tensor) else Tensor(x)
h = h if isinstance(h, Tensor) else Tensor(h)
H = self.hidden_size
gi = ag.add(ag.matmul(x, ag.transpose(self.weight_ih)), self.bias_ih)
gh = ag.add(ag.matmul(h, ag.transpose(self.weight_hh)), self.bias_hh)
i_r = gi[:, 0:H]; i_z = gi[:, H:2 * H]; i_n = gi[:, 2 * H:3 * H]
h_r = gh[:, 0:H]; h_z = gh[:, H:2 * H]; h_n = gh[:, 2 * H:3 * H]
r = ag.sigmoid(ag.add(i_r, h_r))
z = ag.sigmoid(ag.add(i_z, h_z))
n = ag.tanh(ag.add(i_n, ag.mul(r, h_n)))
one = Tensor(np.ones((1, H)))
return ag.add(ag.mul(ag.sub(one, z), n), ag.mul(z, h))The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x: Tensor, h: Tensor, c: Tensor):
"""the LSTM update above; return (h', c')."""
raise NotImplementedError("implement LSTMCell.forward") def forward(self, x: Tensor, h: Tensor, c: Tensor):
"""the LSTM update above; return (h', c')."""
x = x if isinstance(x, Tensor) else Tensor(x)
h = h if isinstance(h, Tensor) else Tensor(h)
c = c if isinstance(c, Tensor) else Tensor(c)
H = self.hidden_size
g = ag.add(ag.add(ag.matmul(x, ag.transpose(self.weight_ih)), self.bias_ih),
ag.add(ag.matmul(h, ag.transpose(self.weight_hh)), self.bias_hh))
i = ag.sigmoid(g[:, 0:H])
f = ag.sigmoid(g[:, H:2 * H])
gg = ag.tanh(g[:, 2 * H:3 * H])
o = ag.sigmoid(g[:, 3 * H:4 * H])
c2 = ag.add(ag.mul(f, c), ag.mul(i, gg))
h2 = ag.mul(o, ag.tanh(c2))
return h2, c2The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x):
"""h = pool(relu(conv1(x))); res = conv2b(relu(conv2a(h))); h = relu(h + res); flatten -> fc."""
raise NotImplementedError("implement SmallResCNN.forward") def forward(self, x):
"""h = pool(relu(conv1(x))); res = conv2b(relu(conv2a(h))); h = relu(h + res); flatten -> fc."""
x = x if isinstance(x, Tensor) else Tensor(x)
h = self.pool(ag.relu(self.conv1(x)))
res = self.conv2b(ag.relu(self.conv2a(h)))
h = ag.relu(ag.add(h, res))
N = h.shape[0]
h = ag.reshape(h, (N, -1))
return self.fc(h)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/nn.py
def forward(self, x) -> Tensor:
"""conv1 -> relu -> pool -> conv2 -> relu -> pool -> flatten -> fc1 -> relu -> fc2."""
raise NotImplementedError("implement SmallCNN.forward") def forward(self, x) -> Tensor:
"""conv1 -> relu -> pool -> conv2 -> relu -> pool -> flatten -> fc1 -> relu -> fc2."""
x = x if isinstance(x, Tensor) else Tensor(x)
x = self.pool(ag.relu(self.conv1(x)))
x = self.pool(ag.relu(self.conv2(x)))
N = x.shape[0]
x = ag.reshape(x, (N, -1))
x = ag.relu(self.fc1(x))
return self.fc2(x)The file /app/submission/nn.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 run_checks.py --only autograd_grads 2>&1 | tail -30
grad/cross_entropy OK grad/kl_div_q OK grad/kl_div_p OK grad/conv2d_input OK (max_abs=1.09e-09) grad/conv2d_weight OK (max_abs=3.38e-09) grad/conv2d_bias OK grad/conv2d_pad0_input OK (max_abs=7.79e-10) grad/maxpool2d OK grad/avgpool2d OK grad/nn.Conv2d_x OK grad/nn.Linear_x OK grad/nn.LayerNorm_x OK grad/nn.BatchNorm1d_x OK grad/cnn_grad_fc2.bias OK (max_abs=1.21e-11) grad/cnn_grad_conv1.weight OK (max_abs=2.55e-11) grad/cnn_grad_conv2.weight OK (max_abs=2.61e-11) grad/cnn_grad_fc1.weight OK (max_abs=3.00e-11) grad/cnn_input_grad OK (max_abs=2.27e-11) grad/nn.GroupNorm_x OK grad/nn.RMSNorm_x OK grad/rnncell OK (max_abs=1.03e-10) grad/grucell OK (max_abs=1.84e-10) grad/lstmcell OK (max_abs=4.13e-11) grad/rescnn_param OK (max_abs=1.81e-11) grad/rescnn_input OK (max_abs=1.58e-11) --------------------------------------- …[truncated 201 chars]
python3 run_checks.py --only autograd_grads 2>&1 | grep FAIL
grad/mean_all FAIL (raised AxisError: axis 1 is out of bounds for array of dimension 0) grad/mean_axis FAIL (raised AxisError: axis 4 is out of bounds for array of dimension 0) Fix the FAIL lines above, then re-run. Each names a backward/forward/optimizer/attack/adversarial-training bug.
python3 -c "
import sys; sys.path.insert(0,'submission'); import numpy as np, autograd as ag
T=ag.Tensor
x=np.random.randn(4,5)
t=T(x.copy(),requires_grad=True)
ag.sum(ag.mean(t)).backward()
print('mean_all grad ok', t.grad.shape)
" 2>&1 | tail -20Traceback (most recent call last):
File "<string>", line 6, in <module>
File "/app/submission/autograd.py", line 351, in mean
N = a.data.size // max(1, od.size)
^^^^^^^^^^^^^^^
File "/app/submission/autograd.py", line 362, in max
out = _mk(np.max(a.data, axis=axis, keepdims=keepdims), (a,), None, a.requires_grad)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/numpy/_core/fromnumeric.py", line 3164, in max
return _wrapreduction(a, np.maximum, 'max', axis, None, out,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/numpy/_core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.exceptions.AxisError: axis 1 is out of bounds for array of dimension 0/app/submission/autograd.py
N = a.data.size // max(1, od.size)
N = a.data.size // (od.size if od.size > 0 else 1)
The file /app/submission/autograd.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 run_checks.py --only autograd_grads 2>&1 | tail -3; echo ---; python3 run_checks.py --only nn_forward 2>&1 | tail -3
autograd_grads 82/82 TOTAL 82/82 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD. --- nn_forward 22/22 TOTAL 22/22 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD.
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement SGD.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
v = self.momentum * self._v[i] + g
self._v[i] = v
step = (g + self.momentum * v) if self.nesterov else v
p.data -= self.lr * stepThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement Adagrad.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._s[i] += g * g
p.data -= self.lr * g / (np.sqrt(self._s[i]) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement RMSprop.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._s[i] = self.alpha * self._s[i] + (1 - self.alpha) * (g * g)
p.data -= self.lr * g / (np.sqrt(self._s[i]) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement Adam.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._v[i] = self.b2 * self._v[i] + (1 - self.b2) * (g * g)
mhat = self._m[i] / (1 - self.b1 ** self.t)
vhat = self._v[i] / (1 - self.b2 ** self.t)
p.data -= self.lr * mhat / (np.sqrt(vhat) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement AdamW.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._v[i] = self.b2 * self._v[i] + (1 - self.b2) * (g * g)
mhat = self._m[i] / (1 - self.b1 ** self.t)
vhat = self._v[i] / (1 - self.b2 ** self.t)
wd_eff = 0.0 if p.ndim <= self.no_decay_ndim else self.weight_decay
p.data -= self.lr * (mhat / (np.sqrt(vhat) + self.eps) + wd_eff * p.data)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement Adamax.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._u[i] = np.maximum(self.b2 * self._u[i], np.abs(g))
p.data -= (self.lr / (1 - self.b1 ** self.t)) * self._m[i] / (self._u[i] + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement NAdam.step") def step(self):
self.t += 1
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._m[i] = self.b1 * self._m[i] + (1 - self.b1) * g
self._v[i] = self.b2 * self._v[i] + (1 - self.b2) * (g * g)
mhat = self._m[i] / (1 - self.b1 ** self.t)
vhat = self._v[i] / (1 - self.b2 ** self.t)
mnes = self.b1 * mhat + (1 - self.b1) * g / (1 - self.b1 ** self.t)
p.data -= self.lr * mnes / (np.sqrt(vhat) + self.eps)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement Lion.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad
c = np.sign(self.b1 * self._m[i] + (1 - self.b1) * g)
p.data -= self.lr * (c + self.weight_decay * p.data)
self._m[i] = self.b2 * self._m[i] + (1 - self.b2) * gThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
def step(self):
raise NotImplementedError("implement Adadelta.step") def step(self):
for i, p in enumerate(self.params):
if p.grad is None:
continue
g = p.grad + self.weight_decay * p.data
self._s[i] = self.rho * self._s[i] + (1 - self.rho) * (g * g)
delta = np.sqrt(self._d[i] + self.eps) / np.sqrt(self._s[i] + self.eps) * g
p.data -= self.lr * delta
self._d[i] = self.rho * self._d[i] + (1 - self.rho) * (delta * delta)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
If total > max_norm, scale
every grad by max_norm/(total+1e-6)."""
raise NotImplementedError("implement clip_grad_norm") If total > max_norm, scale
every grad by max_norm/(total+1e-6)."""
total = 0.0
for p in params:
if p.grad is not None:
total += float((p.grad ** 2).sum())
total = math.sqrt(total)
if total > max_norm:
scale = max_norm / (total + 1e-6)
for p in params:
if p.grad is not None:
p.grad = p.grad * scale
return total<tool_use_error>String to replace not found in file.
String: If total > max_norm, scale
every grad by max_norm/(total+1e-6)."""
raise NotImplementedError("implement clip_grad_norm")</tool_use_error>/app/submission/optim.py
raise NotImplementedError("implement clip_grad_norm") total = 0.0
for p in params:
if p.grad is not None:
total += float((p.grad ** 2).sum())
total = math.sqrt(total)
if total > max_norm:
scale = max_norm / (total + 1e-6)
for p in params:
if p.grad is not None:
p.grad = p.grad * scale
return totalThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement clip_grad_value") for p in params:
if p.grad is not None:
p.grad = np.clip(p.grad, -clip_value, clip_value)The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement cosine_lr") if warmup > 0 and step < warmup:
return base_lr * (step + 1) / warmup
prog = min(1.0, max(0.0, (step - warmup) / max(1, total - warmup)))
return min_lr + 0.5 * (base_lr - min_lr) * (1 + math.cos(math.pi * prog))The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement warmup_lr")return base_lr * (step + 1) / warmup if step < warmup else base_lr
The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement step_lr")return base_lr * gamma ** (step // step_size)
The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement exponential_lr")return base_lr * gamma ** step
The file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement polynomial_lr") prog = min(1.0, max(0.0, step / total))
return (base_lr - min_lr) * ((1.0 - prog) ** power) + min_lrThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/optim.py
raise NotImplementedError("implement linear_lr") prog = min(1.0, max(0.0, step / total))
return base_lr + (min_lr - base_lr) * progThe file /app/submission/optim.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/data.py
def load_npz(path: str) -> Tuple[np.ndarray, np.ndarray]:
raise NotImplementedError("implement load_npz")
def train_val_split(X: np.ndarray, y: np.ndarray, val_frac: float,
rng: np.random.Generator):
raise NotImplementedError("implement train_val_split")
def iter_batches(X: np.ndarray, y: np.ndarray, batch_size: int,
rng: np.random.Generator) -> Iterator[Tuple[np.ndarray, np.ndarray]]:
raise NotImplementedError("implement iter_batches")
def get_batch(X: np.ndarray, y: np.ndarray, batch_size: int,
rng: np.random.Generator) -> Tuple[np.ndarray, np.ndarray]:
"""A single random minibatch (without replacement if possible). Shape-preserving."""
raise NotImplementedError("implement get_batch")
def to_onehot(y: np.ndarray, n_classes: int) -> np.ndarray:
"""(N, n_classes) one-hot float matrix from integer labels y."""
raise NotImplementedError("implement to_onehot")
def clip01(X: np.ndarray) -> np.ndarray:
"""Clamp X into the [0,1] pixel domain (the valid attack range)."""
raise NotImplementedError("implement clip01")def load_npz(path: str) -> Tuple[np.ndarray, np.ndarray]:
d = np.load(path)
X = np.asarray(d["X"], dtype=np.float64)
y = np.asarray(d["y"], dtype=np.int64)
return X, y
def train_val_split(X: np.ndarray, y: np.ndarray, val_frac: float,
rng: np.random.Generator):
N = X.shape[0]
perm = rng.permutation(N)
nval = int(round(val_frac * N))
val_idx = perm[:nval]
tr_idx = perm[nval:]
return (X[tr_idx], y[tr_idx]), (X[val_idx], y[val_idx])
def iter_batches(X: np.ndarray, y: np.ndarray, batch_size: int,
rng: np.random.Generator) -> Iterator[Tuple[np.ndarray, np.ndarray]]:
N = X.shape[0]
perm = rng.permutation(N)
for s in range(0, N, batch_size):
idx = perm[s:s + batch_size]
yield X[idx], y[idx]
def get_batch(X: np.ndarray, y: np.ndarray, batch_size: int,
rng: np.random.Generator) -> Tuple[np.ndarray, np.ndarray]:
"""A single random minibatch (without replacement if possible). Shape-preserving."""
N = X.shape[0]
if batch_size <= N:
idx = rng.choice(N, size=batch_size, replace=False)
else:
idx = rng.choice(N, size=batch_size, replace=True)
return X[idx], y[idx]
def to_onehot(y: np.ndarray, n_classes: int) -> np.ndarray:
"""(N, n_classes) one-hot float matrix from integer labels y."""
y = np.asarray(y)
oh = np.zeros((y.shape[0], n_classes), dtype=np.float64)
oh[np.arange(y.shape[0]), y] = 1.0
return oh
def c
…[truncated 139 chars]The file /app/submission/data.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 run_checks.py --only optim 2>&1 | tail -3; echo ---; python3 run_checks.py --only data 2>&1 | tail -3
optim 20/20 TOTAL 20/20 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD. --- data 3/3 TOTAL 3/3 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD.
/app/submission/attack.py
in a requires_grad=True Tensor, forward, compute loss_fn, backprop, return xt.grad."""
raise NotImplementedError("implement loss_input_grad") in a requires_grad=True Tensor, forward, compute loss_fn, backprop, return xt.grad."""
xt = Tensor(np.asarray(x, dtype=np.float64), requires_grad=True)
logits = model(xt)
loss = loss_fn(logits, y)
loss.backward()
return np.asarray(xt.grad, dtype=np.float64)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""d/dx of mean CROSS-ENTROPY of model(x) vs y (the standard attack gradient)."""
raise NotImplementedError("implement input_grad") """d/dx of mean CROSS-ENTROPY of model(x) vs y (the standard attack gradient)."""
return loss_input_grad(model, x, y, lambda lg, t: ag.cross_entropy(lg, t))The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
Build it with autograd ops so the input-gradient flows; returns a scalar Tensor."""
raise NotImplementedError("implement cw_margin_loss") Build it with autograd ops so the input-gradient flows; returns a scalar Tensor."""
logits = logits if isinstance(logits, Tensor) else Tensor(logits)
N = logits.shape[0]
y = np.asarray(y)
idx = (np.arange(N), y)
z_y = ag.getitem(logits, idx)
big = np.zeros(logits.shape)
big[idx] = 1e9
masked = ag.sub(logits, Tensor(big))
z_other = ag.max(masked, axis=1)
margin = ag.sub(z_y, z_other)
neg_margin = ag.mul(margin, -1.0)
val = ag.maximum(neg_margin, Tensor(np.full(N, -kappa)))
return ag.mean(val)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
it differentiates exactly and the input-gradient flows; returns a scalar Tensor."""
raise NotImplementedError("implement dlr_loss") it differentiates exactly and the input-gradient flows; returns a scalar Tensor."""
logits = logits if isinstance(logits, Tensor) else Tensor(logits)
N = logits.shape[0]
y = np.asarray(y)
idx = (np.arange(N), y)
z_y = ag.getitem(logits, idx)
big = np.zeros(logits.shape)
big[idx] = 1e9
masked = ag.sub(logits, Tensor(big))
z_other = ag.max(masked, axis=1)
num = ag.sub(z_y, z_other)
zmax = ag.max(logits, axis=1)
zmean = ag.mean(logits, axis=1)
denom = ag.add(ag.sub(zmax, zmean), eps)
dlr = ag.mul(ag.div(num, denom), -1.0)
return ag.mean(dlr)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""Project x_adv into the L-inf eps-ball around x, then into [lo,hi]."""
raise NotImplementedError("implement project_linf") """Project x_adv into the L-inf eps-ball around x, then into [lo,hi]."""
out = np.clip(x_adv, x - eps, x + eps)
return np.clip(out, lo, hi)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""Project x_adv so the PER-EXAMPLE L-2 norm of (x_adv - x) is <= eps, then clip to [lo,hi]."""
raise NotImplementedError("implement project_l2") """Project x_adv so the PER-EXAMPLE L-2 norm of (x_adv - x) is <= eps, then clip to [lo,hi]."""
N = x.shape[0]
delta = (x_adv - x).reshape(N, -1)
norm = np.linalg.norm(delta, axis=1, keepdims=True)
factor = np.minimum(1.0, eps / (norm + 1e-12))
delta = (delta * factor).reshape(x.shape)
return np.clip(x + delta, lo, hi)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""One-step FGSM: clip(x + eps*sign(input_grad))."""
raise NotImplementedError("implement fgsm") """One-step FGSM: clip(x + eps*sign(input_grad))."""
lo, hi = clip
g = input_grad(model, x, y)
return np.clip(x + eps * np.sign(g), lo, hi)
def _l1_normalize(g):
N = g.shape[0]
l1 = np.abs(g).reshape(N, -1).sum(axis=1).reshape((N,) + (1,) * (g.ndim - 1))
return g / (l1 + 1e-12)
def _per_example_ce(model, x, y):
logits = model(Tensor(np.asarray(x, dtype=np.float64)))
return np.asarray(ag.nll_per_sample(logits, y).data, dtype=np.float64)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""Iterated FGSM with random start + L-inf projection (the standard PGD attack)."""
raise NotImplementedError("implement pgd_attack") """Iterated FGSM with random start + L-inf projection (the standard PGD attack)."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
if x0 is not None:
xadv = np.asarray(x0, dtype=np.float64).copy()
else:
xadv = x + rng.uniform(-eps, eps, x.shape)
xadv = project_linf(xadv, x, eps, lo, hi)
for _ in range(steps):
g = input_grad(model, xadv, y)
xadv = xadv + alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
project to the L-inf ball + box each step."""
raise NotImplementedError("implement mi_fgsm") project to the L-inf ball + box each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
xadv = x.copy()
g_mom = np.zeros_like(x)
for _ in range(steps):
g = input_grad(model, xadv, y)
g_mom = mu * g_mom + _l1_normalize(g)
xadv = xadv + alpha * np.sign(g_mom)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""Targeted PGD: DESCEND CE toward y_target (step on the NEGATIVE gradient sign); project each step."""
raise NotImplementedError("implement targeted_pgd") """Targeted PGD: DESCEND CE toward y_target (step on the NEGATIVE gradient sign); project each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
xadv = project_linf(x + rng.uniform(-eps, eps, x.shape), x, eps, lo, hi)
for _ in range(steps):
g = input_grad(model, xadv, y_target)
xadv = xadv - alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""PGD ascending the untargeted CW margin loss (cw_margin_loss); project to the ball + box each step."""
raise NotImplementedError("implement cw_pgd") """PGD ascending the untargeted CW margin loss (cw_margin_loss); project to the ball + box each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
xadv = project_linf(x + rng.uniform(-eps, eps, x.shape), x, eps, lo, hi)
lf = lambda lg, t: cw_margin_loss(lg, t, kappa)
for _ in range(steps):
g = loss_input_grad(model, xadv, y, lf)
xadv = xadv + alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
into the L-2 eps-ball + pixel box each step."""
raise NotImplementedError("implement pgd_l2") into the L-2 eps-ball + pixel box each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
xadv = x.copy()
N = x.shape[0]
for _ in range(steps):
g = input_grad(model, xadv, y)
gf = g.reshape(N, -1)
gn = (gf / (np.linalg.norm(gf, axis=1, keepdims=True) + 1e-12)).reshape(x.shape)
xadv = xadv + alpha * gn
xadv = project_l2(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
x_adv += alpha*sign(input_grad); project to the L-inf eps-ball + box."""
raise NotImplementedError("implement bim") x_adv += alpha*sign(input_grad); project to the L-inf eps-ball + box."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
xadv = x.copy()
for _ in range(steps):
g = input_grad(model, xadv, y)
xadv = xadv + alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadv<tool_use_error>String to replace not found in file.
String: x_adv += alpha*sign(input_grad); project to the L-inf eps-ball + box."""
raise NotImplementedError("implement bim")</tool_use_error>/app/submission/attack.py
raise NotImplementedError("implement bim") lo, hi = clip
x = np.asarray(x, dtype=np.float64)
xadv = x.copy()
for _ in range(steps):
g = input_grad(model, xadv, y)
xadv = xadv + alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
size alpha; project to the eps-ball + box. Default alpha = eps/2."""
raise NotImplementedError("implement rfgsm") size alpha; project to the eps-ball + box. Default alpha = eps/2."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if alpha is None:
alpha = eps / 2.0
if rng is None:
rng = np.random.default_rng()
xadv = x + (eps - alpha) * np.sign(rng.uniform(-1.0, 1.0, x.shape))
xadv = np.clip(xadv, lo, hi)
g = input_grad(model, xadv, y)
xadv = xadv + alpha * np.sign(g)
return project_linf(xadv, x, eps, lo, hi)The file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
g_mom = mu*g_mom + grad/||grad||_1, step on sign(g_mom); project each step."""
raise NotImplementedError("implement ni_fgsm") g_mom = mu*g_mom + grad/||grad||_1, step on sign(g_mom); project each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
xadv = x.copy()
g_mom = np.zeros_like(x)
for _ in range(steps):
x_nes = xadv + alpha * mu * g_mom
g = input_grad(model, x_nes, y)
g_mom = mu * g_mom + _l1_normalize(g)
xadv = xadv + alpha * np.sign(g_mom)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
adversarial input with the HIGHEST per-example cross-entropy (worst case). Return that batch."""
raise NotImplementedError("implement pgd_restarts") adversarial input with the HIGHEST per-example cross-entropy (worst case). Return that batch."""
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
best_x = None
best_loss = None
for _ in range(restarts):
xr = pgd_attack(model, x, y, eps, steps, alpha, clip=clip, rng=rng)
losses = _per_example_ce(model, xr, y)
if best_x is None:
best_x = xr.copy()
best_loss = losses
else:
mask = losses > best_loss
best_x[mask] = xr[mask]
best_loss = np.where(mask, losses, best_loss)
return best_xThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/attack.py
"""PGD ascending the DLR loss (dlr_loss) instead of CE; project to the L-inf ball + box each step."""
raise NotImplementedError("implement dlr_pgd") """PGD ascending the DLR loss (dlr_loss) instead of CE; project to the L-inf ball + box each step."""
lo, hi = clip
x = np.asarray(x, dtype=np.float64)
if rng is None:
rng = np.random.default_rng()
xadv = project_linf(x + rng.uniform(-eps, eps, x.shape), x, eps, lo, hi)
lf = lambda lg, t: dlr_loss(lg, t)
for _ in range(steps):
g = loss_input_grad(model, xadv, y, lf)
xadv = xadv + alpha * np.sign(g)
xadv = project_linf(xadv, x, eps, lo, hi)
return xadvThe file /app/submission/attack.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 run_checks.py --only attack 2>&1 | tail -25
[attack] 18/18 attack/input_grad_fd OK (max_abs=2.73e-11) attack/cw_loss_input_grad_fd OK (max_abs=4.13e-12) attack/dlr_loss_grad OK (max_abs=1.04e-09) attack/dlr_loss_input_grad_fd OK (max_abs=3.51e-11) attack/fgsm_step OK attack/pgd_projection OK attack/mi_fgsm_projection OK attack/cw_pgd_projection OK attack/pgd_l2_projection OK attack/bim_projection OK attack/rfgsm_projection OK attack/ni_fgsm_projection OK attack/pgd_restarts_projection OK attack/dlr_pgd_projection OK attack/pgd_stronger_than_fgsm OK (fgsm=1.450 pgd=1.469) attack/bim_stronger_than_fgsm OK (fgsm=1.411 bim=1.423) attack/pgd_restarts_worst_case OK (single=1.521 restarts=1.525) attack/targeted_pgd_lowers_target_ce OK (1.388->1.333) ------------------------------------------------------------------------ attack 18/18 TOTAL 18/18 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD.
/app/submission/advtrain.py
"""Madry PGD-AT loss: cross-entropy on the adversarial batch (trains parameters)."""
raise NotImplementedError("implement pgd_at_loss") """Madry PGD-AT loss: cross-entropy on the adversarial batch (trains parameters)."""
logits = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
return ag.cross_entropy(logits, y)The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""TRADES loss = CE(clean, y) + beta * KL(stopgrad(softmax(clean)) || softmax(adv))."""
raise NotImplementedError("implement trades_loss") """TRADES loss = CE(clean, y) + beta * KL(stopgrad(softmax(clean)) || softmax(adv))."""
cl = model(Tensor(np.asarray(x_clean, dtype=np.float64)))
ad = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce = ag.cross_entropy(cl, y)
tgt = Tensor(cl.data.copy()) # stop-gradient clean logits
kl = ag.kl_div(tgt, ad)
loss = ag.add(ce, ag.mul(kl, float(beta)))
return loss, {"ce": float(ce.data), "kl": float(kl.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
DETACHED); see the module docstring for the exact form."""
raise NotImplementedError("implement mart_loss") DETACHED); see the module docstring for the exact form."""
y = np.asarray(y)
ad = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
cl_data = model(Tensor(np.asarray(x_clean, dtype=np.float64))).data
N, C = ad.shape
idx = (np.arange(N), y)
# boosted CE on adversarial batch
ce = ag.cross_entropy(ad, y)
logp_adv = ag.log_softmax(ad)
p_adv = ag.exp(logp_adv)
big = np.zeros((N, C)); big[idx] = 1e9
maxother = ag.max(ag.sub(p_adv, Tensor(big)), axis=1)
margin_term = ag.mul(ag.log(ag.sub(Tensor(np.ones(N)), maxother)), -1.0)
boosted = ag.add(ce, ag.mean(margin_term))
# misclassification-aware weighted KL with detached clean target
logp_clean = _logsoftmax_np(cl_data)
p_clean = np.exp(logp_clean)
diff = ag.sub(logp_adv, Tensor(logp_clean))
kl_terms = ag.sum(ag.mul(p_adv, diff), axis=1)
w = 1.0 - p_clean[idx]
kl_w = ag.mean(ag.mul(kl_terms, Tensor(w)))
loss = ag.add(boosted, ag.mul(kl_w, float(beta)))
return loss, {"boosted": float(boosted.data), "kl": float(kl_w.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
import autograd as ag from autograd import Tensor
import autograd as ag
from autograd import Tensor
def _logsoftmax_np(z):
m = z.max(axis=-1, keepdims=True)
return z - m - np.log(np.exp(z - m).sum(axis=-1, keepdims=True))The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""Adversarial Logit Pairing: 0.5*(CE(clean)+CE(adv)) + lam*mean||z_clean - z_adv||^2 (no detach)."""
raise NotImplementedError("implement alp_loss") """Adversarial Logit Pairing: 0.5*(CE(clean)+CE(adv)) + lam*mean||z_clean - z_adv||^2 (no detach)."""
zc = model(Tensor(np.asarray(x_clean, dtype=np.float64)))
za = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce = ag.mul(ag.add(ag.cross_entropy(zc, y), ag.cross_entropy(za, y)), 0.5)
diff = ag.sub(zc, za)
pair = ag.mean(ag.mul(diff, diff))
loss = ag.add(ce, ag.mul(pair, float(lam)))
return loss, {"ce": float(ce.data), "pair": float(pair.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""CE against a label-smoothed target ((1-eps_ls)*onehot + eps_ls/C*uniform), via log_softmax."""
raise NotImplementedError("implement label_smoothed_ce") """CE against a label-smoothed target ((1-eps_ls)*onehot + eps_ls/C*uniform), via log_softmax."""
logits = logits if isinstance(logits, Tensor) else Tensor(logits)
y = np.asarray(y)
N, C = logits.shape
if n_classes is None:
n_classes = C
smooth = np.full((N, C), eps_ls / n_classes)
smooth[np.arange(N), y] += (1.0 - eps_ls)
ls = ag.log_softmax(logits)
return ag.mul(ag.mean(ag.sum(ag.mul(ls, Tensor(smooth)), axis=1)), -1.0)The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""Mixed clean+adversarial CE: lam*CE(adv,y) + (1-lam)*CE(clean,y)."""
raise NotImplementedError("implement robust_ce_loss") """Mixed clean+adversarial CE: lam*CE(adv,y) + (1-lam)*CE(clean,y)."""
zc = model(Tensor(np.asarray(x_clean, dtype=np.float64)))
za = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce_a = ag.cross_entropy(za, y)
ce_c = ag.cross_entropy(zc, y)
loss = ag.add(ag.mul(ce_a, float(lam)), ag.mul(ce_c, float(1.0 - lam)))
return loss, {"ce_adv": float(ce_a.data), "ce_clean": float(ce_c.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
through the adversarial branch; the clean branch still trains via its CE term)."""
raise NotImplementedError("implement clp_loss") through the adversarial branch; the clean branch still trains via its CE term)."""
zc = model(Tensor(np.asarray(x_clean, dtype=np.float64)))
za = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce = ag.mul(ag.add(ag.cross_entropy(zc, y), ag.cross_entropy(za, y)), 0.5)
zc_det = Tensor(zc.data.copy())
diff = ag.sub(zc_det, za)
pair = ag.mean(ag.mul(diff, diff))
loss = ag.add(ce, ag.mul(pair, float(lam)))
return loss, {"ce": float(ce.data), "pair": float(pair.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
as the second ("q") -- the opposite argument order from TRADES."""
raise NotImplementedError("implement kl_at_loss") as the second ("q") -- the opposite argument order from TRADES."""
cl = model(Tensor(np.asarray(x_clean, dtype=np.float64)))
ad = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce = ag.cross_entropy(ad, y)
tgt = Tensor(cl.data.copy())
kl = ag.kl_div(ad, tgt)
loss = ag.add(ce, ag.mul(kl, float(beta)))
return loss, {"ce": float(ce.data), "kl": float(kl.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""AT with logit squeezing: L = CE(adv,y) + lam*mean(z_adv**2) (penalize large logits)."""
raise NotImplementedError("implement logit_squeeze_at_loss") """AT with logit squeezing: L = CE(adv,y) + lam*mean(z_adv**2) (penalize large logits)."""
ad = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
ce = ag.cross_entropy(ad, y)
sq = ag.mean(ag.mul(ad, ad))
loss = ag.add(ce, ag.mul(sq, float(lam)))
return loss, {"ce": float(ce.data), "sq": float(sq.data), "loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
/app/submission/advtrain.py
"""Label-smoothed AT: L = label_smoothed_ce(model(adv), y, eps_ls)."""
raise NotImplementedError("implement ls_at_loss") """Label-smoothed AT: L = label_smoothed_ce(model(adv), y, eps_ls)."""
ad = model(Tensor(np.asarray(x_adv, dtype=np.float64)))
loss = label_smoothed_ce(ad, y, eps_ls)
return loss, {"loss": float(loss.data)}The file /app/submission/advtrain.py has been updated successfully. (file state is current in your context , no need to Read it back)
python3 run_checks.py --only advtrain 2>&1 | tail -20; echo ---INTEG---; python3 run_checks.py --only integration 2>&1 | tail -6
[advtrain] 14/14 advtrain/at_param_grad OK (max_abs=2.56e-11) advtrain/trades_param_grad OK (max_abs=5.66e-04) advtrain/mart_param_grad OK (max_abs=1.14e-04) advtrain/alp_param_grad OK (max_abs=2.60e-11) advtrain/robust_ce_param_grad OK (max_abs=5.12e-11) advtrain/clp_param_grad OK (max_abs=2.84e-05) advtrain/kl_at_param_grad OK (max_abs=1.62e-03) advtrain/logit_squeeze_param_grad OK (max_abs=4.80e-11) advtrain/ls_at_param_grad OK (max_abs=2.76e-11) advtrain/label_smoothed_ce_grad OK advtrain/trades_stopgrad OK (max_abs=0.00e+00) advtrain/alp_pairing OK (max_abs=0.00e+00) advtrain/clp_stopgrad OK (max_abs=0.00e+00) advtrain/kl_at_stopgrad OK (max_abs=0.00e+00) ------------------------------------------------------------------------ advtrain 14/14 TOTAL 14/14 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD. ---INTEG--- integration/trades_decreases OK (1.3933->1.3367) ------------------------------------------------------------------------ integration 4/4 TOTAL 4/4 public checks passing All public checks pass. Now make sure model.npz clears the …[truncated 106 chars]
python3 run_checks.py 2>&1 | tail -5
------------------------------------------------------------------------ autograd_grads 82/82 | nn_forward 22/22 | optim 20/20 | attack 18/18 | advtrain 14/14 | integration 4/4 | data 3/3 TOTAL 163/163 public checks passing All public checks pass. Now make sure model.npz clears the held-out ROBUST-accuracy threshold; the grader re-checks with DIFFERENT seeds/shapes + its OWN strong PGD.
/app/submission/train.py
"""Run multi-step PGD adversarial training and save the best-by-robust-val checkpoint to
``out_path``. See the module docstring for the recipe."""
raise NotImplementedError("implement the adversarial-training loop") """Run multi-step PGD adversarial training and save the best-by-robust-val checkpoint to
``out_path``. See the module docstring for the recipe."""
import time
rng = np.random.default_rng(seed)
X, y = D.load_npz(train_path)
n_classes = int(y.max()) + 1
c_in = X.shape[1]
img = X.shape[2]
(Xtr, ytr), (Xval, yval) = D.train_val_split(X, y, val_frac, rng)
print(f"data: train={Xtr.shape} val={Xval.shape} n_classes={n_classes} img={img}", flush=True)
model = SmallCNN(n_classes=n_classes, c_in=c_in, img=img, c1=C1, c2=C2, ff=FF, seed=seed)
params = model.params()
opt = O.AdamW(params, lr=lr, betas=(0.9, 0.999), weight_decay=weight_decay, no_decay_ndim=1)
nb = int(np.ceil(len(Xtr) / batch_size))
total_steps = epochs * nb
def robust_val_acc(steps, restarts=2, n_eval=1024):
m = min(n_eval, len(Xval))
idx = np.arange(m)
xv = Xval[idx]; yv = yval[idx]
# clean correctness
clean_pred = np.argmax(model(xv).data, axis=1)
correct = clean_pred == yv
rng2 = np.random.default_rng(12345)
robust = correct.copy()
for r in range(restarts):
xadv = A.pgd_attack(model, xv, yv, eps, steps, alpha, rng=rng2)
pred = np.argmax(model(xadv).data, axis=1)
robust &= (pred == yv)
return float(robust.mean()), float(correct.mean())
best_robust = -1.0
best_sd = model.state_dict()
gstep = 0
for ep in range(epochs):
…[truncated 1444 chars]The file /app/submission/train.py has been updated successfully. (file state is current in your context , no need to Read it back)
timeout 300 python3 submission/train.py --epochs 1 --eps 0.12 --at-steps 7 2>&1 | tail -10
data: train=(3060, 1, 16, 16) val=(540, 1, 16, 16) n_classes=6 img=16
timeout 120 python3 -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A, advtrain as AT, optim as O
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
t=time.time();
for _ in range(5): _=m(xb)
print('fwd x5', time.time()-t)
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,7,0.03,rng=np.random.default_rng(0))
print('pgd7 1batch', time.time()-t)
t=time.time()
loss=AT.pgd_at_loss(m,xadv,yb); loss.backward()
print('train fb', time.time()-t)
" 2>&1 | tail(Bash completed with no output)
timeout 60 python3 -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb=X[:128]
t=time.time(); _=m(xb); print('1 fwd', time.time()-t)
t=time.time(); o=m(xb);
import autograd as ag
l=ag.cross_entropy(o,y[:128]); l.backward(); print('1 fb', time.time()-t)
" 2>&1 | tail1 fwd 0.05009865760803223 1 fb 0.14673280715942383
timeout 90 python3 -u -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A, advtrain as AT
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,7,0.03,rng=np.random.default_rng(0))
print('pgd7 1batch', time.time()-t, flush=True)
" 2>&1 | tail(Bash completed with no output)
timeout 60 python3 -u -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
t=time.time()
g=A.input_grad(m,xb,yb)
print('input_grad', time.time()-t, g.shape, flush=True)
" 2>&1 | tailinput_grad 0.14638304710388184 (128, 1, 16, 16)
timeout 60 python3 -u -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
for st in [1,2,3]:
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,st,0.03,rng=np.random.default_rng(0))
print('pgd',st, time.time()-t, flush=True)
" 2>&1 | tailpgd 1 0.15434622764587402 pgd 2 0.2867879867553711 pgd 3 0.4444427490234375
timeout 60 python3 -u -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,7,0.03,rng=np.random.default_rng(0))
print('pgd7', time.time()-t, flush=True)
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,10,0.03,rng=np.random.default_rng(0))
print('pgd10', time.time()-t, flush=True)
" 2>&1 | tail(Bash completed with no output)
OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 timeout 60 python3 -u -c "
import sys,time; sys.path.insert(0,'submission')
import numpy as np, data as D, attack as A
from nn import SmallCNN
X,y=D.load_npz('data/train.npz')
m=SmallCNN(n_classes=6,c_in=1,img=16,c1=14,c2=28,ff=96,seed=0)
xb,yb=X[:128],y[:128]
for st in [7,10,20]:
t=time.time()
xadv=A.pgd_attack(m,xb,yb,0.12,st,0.03,rng=np.random.default_rng(0))
print('pgd',st, round(time.time()-t,3), flush=True)
" 2>&1 | tailNo file changes, the agent left the workspace untouched.
=== SyncVals verifier (local re-grade of the agent's own checkpoint) === COMPONENT CONFORMANCE SUITE: 201/201 cases passed (all milestones pass: True) FINAL METRIC: PGD robust accuracy = 0.5663 < threshold 0.6 REWARD = 0 (1 iff every component milestone passes AND the sealed held-out metric clears its threshold) NOTE: the in-sandbox training in the 481MB e2b box was OOM-killed (0-byte checkpoint), so the raw in-sandbox reward was an infrastructure artifact. This reward is from a faithful LOCAL re-train of the agent's OWN train.py (24GB, no OOM) with its own default hyperparameters, then re-graded by the real verifier.
Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_c4dfb84694d34c9f. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.
Trial trial_c4dfb84694d34c9f · verifier authoritative; classifier explanatory.