SyncVals · Trajectory

simjeb-bracket-fea-mass-prediction-real

claude-code claude-opus-4-8 ✓ resolved GOOD_SUCCESS ↑ View task

Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.

Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.

Classification , post-hoc; cannot change the reward

GOOD_SUCCESSLegitimate solve, implements the asked-for behavior; tests verify real functionality.

SubtypeLegitimate Solution

EvidenceTest stdout shows: 'reward = 1.0 (PASS)' and 'raw_score: 0.5'. The agent trajectory shows systematic investigation of data properties, careful hyperparameter tuning via out-of-fold CV, empirical discovery of y-reflection augmentation's benefit, and a principled training pipeline. The final checkpoint successfully meets all three target thresholds: rel_L2 ≤ 0.204, MAE ≤ 0.107, max_abs_err ≤ 0.8 (from task_config.json reward_policy).

Root causeThe agent implemented a rigorous, data-driven approach to training a PointNet regressor: out-of-fold cross-validation for robust metric estimation, ceiling analysis to verify the architecture was appropriate, geometry-aware augmentation (y-reflection discovered via Chamfer distance), and systematic hyperparameter search. This thorough methodology enabled the model to reach the maximum reward by meeting all target metrics.

RecommendationN/A - task is fine

Trajectory

Tool-by-tool agent trajectory

316 tool calls · 3 tool types · 316 steps

You are given point-cloud samples of 3D jet-engine bracket designs along with each bracket's FEA-derived mass (kg). Each point cloud is an unordered set of 2048 surface points in R^3; coordinates are pre-centered and rescaled so the axis-aligned bounding box of each bracket fits inside the unit cube. The mapping from geometry to mass is not given to you - you have to learn it from the data. Train a model that takes a `[2048, 3]` point cloud and predicts the scalar mass value. Your model is then applied to a held-out set of bracket geometries and the predictions are compared against the ground-truth FEA-simulated mass values. ## Files - `/app/data/{train,val,public_test}/samples.pt` - each file is a dict with keys `features` (shape `[N, 2048, 3]`) and `labels` (shape `[N, 1]`), both float16 on disk. Load with `torch.load(path, weights_only=True)`. - `/app/task_config.json` - architecture sizes and training defaults. - `/app/feature_schema.json` - input / label spec. - `/app/candidate_model.py` - the fixed PointNet-style architecture. Keep it as is; your saved checkpoint has to load back into this same architecture. - `/app/dataset.py` - a minimal loader for `samples.pt`. - `/app/train.py` - a small starter training loop you can extend or replace. ## What you must produce Write your trained checkpoint to `/app/outputs/model.pt` as a dict with the following required keys: - `state_dict` - your model's `state_dict()` mapping. - `feat_mean`, `feat_std` - length-3 tensors used to standardize the input coordinates before the forward pass. The same normalization is applied at inference time, so these are required. - `label_mean`, `label_std` - length-1 tensors used to standardize the target during training and de-standardize the model output at inference. The model's output is mapped back to physical units using these, so they are required too. The model must consume a `[B, 2048, 3]` tensor and produce a `[B, 1]` output. ## What you are optimizing Your predictions are measured against the true FEA-simulated mass labels with three error numbers, all lower is better: the relative L2 over the full set (the primary one), the mean absolute error, and the worst-case per-sample absolute error. A strong model needs to do well on all three, so do not trade one off against the others.

Let me start by exploring the environment to understand what I'm working with.

I'll start by exploring the environment to understand the task, data, and provided code.

bash

cd /app && ls -la && echo "---CONFIG---" && cat task_config.json && echo "---SCHEMA---" && cat feature_schema.json

output

total 14
drwxrws--- 3 h2tagent h2tagent  138 Jun 20 18:13 .
drwxr-xr-x 1 root     root     4096 Jun 20 20:55 ..
-rw-rw---- 1 h2tagent h2tagent 1639 Jun 20 18:10 candidate_model.py
drwxrws--- 5 h2tagent h2tagent   58 Jun 20 18:10 data
-rw-rw---- 1 h2tagent h2tagent 1183 Jun 20 18:10 dataset.py
-rw-rw---- 1 h2tagent h2tagent  499 Jun 20 18:10 feature_schema.json
-rw-rw---- 1 h2tagent h2tagent  581 Jun 20 18:10 task_config.json
-rw-rw---- 1 h2tagent h2tagent 3117 Jun 20 18:10 train.py
---CONFIG---
{
  "task_id": "simjeb-bracket-fea-mass-prediction-real",
  "dataset_name": "simjeb-bracket-fea-mass-pilot",
  "n_points": 2048,
  "point_dim": 3,
  "label_dim": 1,
  "label_per_point": false,
  "seed": 20260518,
  "model": {
    "feature_width": 64,
    "global_width": 256,
    "head_width": 128
  },
  "training": {
    "epochs": 1,
    "batch_size": 16,
    "lr": 0.001,
    "weight_decay": 0.0001,
    "loss": "MSE"
  },
  "evaluation": {
    "primary_metric": "mass_rel_l2_score",
    "secondary_metrics": [
      "mass_mae_score",
      "mass_max_abs_err_score"
    ]
  }
}
---SCHEMA---
{
  "version": "simjeb-bracket-fea-mass-pilot-v1",
  "input": {
    "type": "point_cloud",
    "n_points": 2048,
    "dim": 3,
    "coord_range": [
      -0.5,
      0.5
    ],
    "description": "Unordered surface point sample of a jet-engine bracket; each bracket mesh is centered at its bounding-box centroid and rescaled so the axis-aligned bounding box fits inside the unit cube."
  },
  "label": {
    "type": "scalar",
    "dim": 1,
    "description": "FEA-derived bracket mass in kg."
  }
}

Let me look at the model architecture, dataset loader, and training loop.

Let me examine the model architecture, dataset loader, and the starter training script.

bash

cd /app && echo "===candidate_model.py===" && cat candidate_model.py && echo "===dataset.py===" && cat dataset.py && echo "===train.py===" && cat train.py

output

===candidate_model.py===
from __future__ import annotations

from typing import Mapping

import torch
import torch.nn as nn


class SharedPointMLP(nn.Module):
    def __init__(self, in_dim: int, widths: tuple[int, ...]) -> None:
        super().__init__()
        layers: list[nn.Module] = []
        prev = in_dim
        for w in widths:
            layers.append(nn.Conv1d(prev, w, kernel_size=1))
            layers.append(nn.BatchNorm1d(w))
            layers.append(nn.GELU())
            prev = w
        self.net = nn.Sequential(*layers)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.net(x)


class PointNetRegressor(nn.Module):
    def __init__(self, feature_width: int, global_width: int, head_width: int) -> None:
        super().__init__()
        self.feature = SharedPointMLP(3, (feature_width, feature_width, feature_width))
        self.lift = SharedPointMLP(feature_width, (feature_width * 2, global_width))
        self.head = nn.Sequential(
            nn.Linear(global_width, head_width),
            nn.GELU(),
            nn.Linear(head_width, head_width // 2),
            nn.GELU(),
            nn.Linear(head_width // 2, 1),
        )

    def forward(self, points: torch.Tensor) -> torch.Tensor:
        x = points.transpose(1, 2).contiguous()
        x = self.feature(x)
        x = self.lift(x)
        x = x.max(dim=2).values
        return self.head(x)


def build_model(config: Mapping) -> PointNetRegressor:
    m = config["model"]
    return PointNetRegressor(
        feature_width=int(m["feature_width"]),
        global_width=int(m["global_width"]),
        head_width=int(m["head_width"]),
    )
===dataset.py===
from __future__ import annotations

from pathlib import Path
from typing import Iterable

import torch
from torch.utils.data import Dataset


class SampleStore(Dataset):
    def __init__(self, split_dir: str | Path) -> None:
        self.dir = Path(split_dir)
        payload = torch.load(self.dir / "samples.pt", map_location="cpu", weights_only=True)
        self.features: torch.Tensor = payload["features"].float()
        self.labels: torch.Tensor = payload["labels"].float()
        if self.features.shape[0] != self.labels.shape[0]:
            raise ValueError("features and labels must have the same number of rows")

    def __len__(self) -> int:
        return self.features.shape[0]

    def __getitem__(self, idx: int) -> tuple[torch.Tensor, torch.Tensor]:
        return self.features[idx], self.labels[idx]


def iter_batches(
    x: torch.Tensor,
    y: torch.Tensor,
    batch_size: int,
    shuffle: bool = True,
) -> Iterable[tuple[torch.Tensor, torch.Tensor]]:
    n = x.shape[0]
    idx = torch.randperm(n) if shuffle else torch.arange(n)
    for start in range(0, n, batch_size):
        chunk = idx[start : start + batch_size]
        yield x[chunk], y[chunk]
===train.py===
from __future__ import annotations

import json
import os
import sys
from pathlib import Path

import torch
import torch.nn.functional as F

APP_DIR = Path(os.environ.get("APP_DIR", "/app"))
sys.path.insert(0, str(APP_DIR))

from candidate_model import build_model  # noqa: E402
from dataset import SampleStore, iter_batches  # noqa: E402


def _device() -> torch.device:
    if torch.cuda.is_available():
        return torch.device("cuda")
    if torch.backends.mps.is_available():
        return torch.device("mps")
    return torch.device("cpu")


def _load_config() -> dict:
    with (APP_DIR / "task_config.json").open() as fh:
        return json.load(fh)


def main() -> None:
    cfg = _load_config()
    device = _device()
    torch.manual_seed(int(cfg["seed"]))

    train = SampleStore(APP_DIR / "data" / "train")
    val = SampleStore(APP_DIR / "data" / "val")

    feat_mean = train.features.mean(dim=(0, 1), keepdim=True)
    feat_std = train.features.std(dim=(0, 1), keepdim=True).clamp_min(1e-6)
    label_mean = train.labels.mean(dim=(0,), keepdim=True)
    label_std = train.labels.std(dim=(0,), keepdi
…[truncated]

Now let me understand the data. Let me look at the shapes and statistics.

Now let me inspect the data splits to understand sizes and label distributions.

bash

cd /app && ls -la data && echo "---" && for d in train val public_test; do echo "=== $d ==="; ls -la data/$d; done

output

total 3
drwxrws--- 5 h2tagent h2tagent  58 Jun 20 18:10 .
drwxrws--- 3 h2tagent h2tagent 138 Jun 20 18:13 ..
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 public_test
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 train
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 val
---
=== train ===
total 1210
drwxrws--- 2 h2tagent h2tagent      55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent      58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent    6483 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 1230821 Jun 20 18:10 samples.pt
=== val ===
total 389
drwxrws--- 2 h2tagent h2tagent     55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent     58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent   2011 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 395109 Jun 20 18:10 samples.pt
=== public_test ===
total 197
drwxrws--- 2 h2tagent h2tagent     55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent     58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent   1134 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 198437 Jun 20 18:10 samples.pt

bash

cd /app && python -c "import torch, sys; print(torch.__version__); print('cuda', torch.cuda.is_available())" && echo "---manifest head train---" && head -n 3 data/train/manifest.jsonl && echo "---manifest head val---" && head -n 2 data/val/manifest.jsonl && echo "---manifest head test---" && head -n 2 data/public_test/manifest.jsonl

output

2.5.1+cu124
cuda True
---manifest head train---
{"id": "train_0000", "source_bracket_id": 214, "n_points": 2048}
{"id": "train_0001", "source_bracket_id": 538, "n_points": 2048}
{"id": "train_0002", "source_bracket_id": 546, "n_points": 2048}
---manifest head val---
{"id": "val_0000", "source_bracket_id": 418, "n_points": 2048}
{"id": "val_0001", "source_bracket_id": 441, "n_points": 2048}
---manifest head test---
{"id": "public_test_0000", "source_bracket_id": 461, "n_points": 2048}
{"id": "public_test_0001", "source_bracket_id": 440, "n_points": 2048}

bash

cd /app && python -c "
import torch
for split in ['train','val','public_test']:
    d = torch.load(f'data/{split}/samples.pt', weights_only=True)
    f = d['features']; l = d['labels']
    print(f'=== {split} ===')
    print('features', f.shape, f.dtype, 'labels', l.shape, l.dtype)
    ff = f.float(); ll = l.float()
    print('feat min/max/mean/std', ff.min().item(), ff.max().item(), ff.mean().item(), ff.std().item())
    print('label min/max/mean/std', ll.min().item(), ll.max().item(), ll.mean().item(), ll.std().item())
    print('label sample', ll.flatten()[:10].tolist())
"

output

=== train ===
features torch.Size([100, 2048, 3]) torch.float16 labels torch.Size([100, 1]) torch.float16
feat min/max/mean/std -0.5 0.5 -0.025671008974313736 0.18626290559768677
label min/max/mean/std 0.207763671875 2.21875 0.7475781440734863 0.3894648551940918
label sample [1.6396484375, 1.3828125, 0.64111328125, 0.603515625, 0.369140625, 0.7783203125, 0.63623046875, 0.6767578125, 1.14453125, 0.85302734375]
=== val ===
features torch.Size([32, 2048, 3]) torch.float16 labels torch.Size([32, 1]) torch.float16
feat min/max/mean/std -0.5 0.5 -0.02480550855398178 0.184591144323349
label min/max/mean/std 0.293701171875 2.072265625 0.7621994018554688 0.41264674067497253
label sample [0.403076171875, 0.473876953125, 0.7001953125, 0.293701171875, 0.80224609375, 1.4267578125, 0.5927734375, 0.71484375, 1.1796875, 0.859375]
=== public_test ===
features torch.Size([16, 2048, 3]) torch.float16 labels torch.Size([16, 1]) torch.float16
feat min/max/mean/std -0.5 0.5 -0.025328462943434715 0.18716458976268768
label min/max/mean/std 0.1983642578125 1.44921875 0.7296066284179688 0.327779084444046
label sample [0.416748046875, 0.54931640625, 0.51953125, 1.1220703125, 0.7333984375, 0.2861328125, 0.947265625, 0.6328125, 0.56640625, 1.0830078125]

Key observations: - Very small dataset: only 100 training samples, 32 validation, 16 public test. - Each sample is 2048 points in R^3, coordinates in [-0.5, 0.5]. - Labels (mass) range roughly 0.2 to 2.2 kg, mean ~0.75, std ~0.39. - The input is pre-centered and bounding-box normalized to fit in the unit cube. This is a tiny dataset, which makes overfitting a serious concern. The architecture is fixed (PointNet-style), but I can control normalization, training procedure, augmentation, loss function, epochs, etc. Important insight: mass of a bracket. Since the bracket is rescaled to fit the unit cube, the raw geometry scale information is partly lost. But mass depends on volume × density. The point cloud is only surface points. But mass correlates with the shape. Wait - important. The bounding box is rescaled so that the AABB fits inside the unit cube. So the absolute scale is normalized away. That means the mass cannot be directly inferred from overall size - it must be inferred from the *shape* (relative proportions, how much material, thickness, etc.). Actually, the rescaling means we lose the overall size. Hmm, but different brackets have different masses even after normalization because the shape differs. Actually wait , let me reconsider. The bracket mesh is centered at bounding-box centroid and rescaled so AABB fits inside unit cube. The rescale factor itself is not given. So two brackets with identical shape but different absolute size would have the same point cloud but different masses. This introduces irreducible noise UNLESS the rescale factor is somehow encoded. But the task says we learn geometry→mass. Presumably the variation in mass is dominated by shape differences, and there's a learnable mapping. Actually, more precisely: the mass from FEA = volume × material density. If all brackets use the same material and the same physical mounting dimensions (SimJEB brackets all fit the same bolt pattern / bounding envelope), then the absolute scale is roughly fixed across brackets, and the normalization to unit cube is approximately consistent. In SimJEB, all brackets share the same bounding envelope constraints (they must connect the same interface points). So the normalization factor is roughly constant, and shape → mass is well-defined. Good. So this is a regression problem on a small dataset. The key challenges: 1. Small data (100 train) → regularization, augmentation critical. 2. Fixed architecture with BatchNorm → need careful handling at inference (eval mode). Batch norm with small batches can be tricky. 3. Metrics: relative L2 (primary), MAE, max abs error. Need to do well on all. Let me think about the metrics. Relative L2 over the full set: typically ||pred - true||_2 / ||true||_2 = sqrt(sum (pred-true)^2) / sqrt(sum true^2). MAE = mean |pred - true|. Max abs err = max |pred - true|. These are all in physical units (kg). So I should train to minimize squared error in physical units roughly, but also watch the worst-case. Given data is tiny, a PointNet can easily memorize 100 samples. I need strong augmentation and regularization to generalize to val/test. Augmentation ideas for point clouds (that preserve mass): - Random rotation? Careful , rotation changes the bounding box and the problem states coordinates are pre-centered and AABB fits in the unit cube. If I rotate, the AABB changes. But mass is rotation-invariant (physical property). However, the input distribution is axis-aligned. Rotating arbitrarily would take the points outside the training distribution (bounding box no longer axis aligned within unit cube). Actually, since brackets have a canonical orientation (mounting), the test set is in the same canonical orientation. So large arbitrary rotations would hurt. Small rotations / jitter might help. Actually, I should be careful: augmentations must keep the input in-distribution relative to test. Let me reconsider. The brackets are in a canonical pose. Augmentations that preserve mass AND …[truncated]

This is a small-data regression problem: only 100 training point clouds (plus 32 val, 16 public_test), each `[2048,3]`, predicting scalar mass. The architecture is fixed (PointNet-style with BatchNorm + max-pool). Key risks are overfitting and noisy tiny-set metrics. Let me first check a few data properties (like whether brackets have reflection symmetry, useful for augmentation) before building the training harness.

bash

cd /app && python -c "
import torch
d = torch.load('data/train/samples.pt', weights_only=True)
f = d['features'].float()  # [100,2048,3]
# per-axis stats
print('per-axis min', f.amin(dim=(0,1)).tolist())
print('per-axis max', f.amax(dim=(0,1)).tolist())
print('per-axis mean', f.mean(dim=(0,1)).tolist())
print('per-axis std', f.std(dim=(0,1)).tolist())
# per-sample bounding box extents (should each be ~1 along the largest axis if AABB fits unit cube)
ext = f.amax(dim=1) - f.amin(dim=1)  # [100,3]
print('extent mean per axis', ext.mean(0).tolist())
print('extent max per axis', ext.amax(0).tolist())
print('max extent over axes per sample: mean/min/max', ext.amax(1).mean().item(), ext.amax(1).min().item(), ext.amax(1).max().item())
"

output

per-axis min [-0.320556640625, -0.5, -0.2279052734375]
per-axis max [0.322998046875, 0.5, 0.22705078125]
per-axis mean [-0.013504491187632084, 0.007462748326361179, -0.0709712877869606]
per-axis std [0.14211857318878174, 0.2701786458492279, 0.0871192067861557]
extent mean per axis [0.5887036323547363, 0.9991015791893005, 0.34989869594573975]
extent max per axis [0.6435546875, 1.0, 0.4549560546875]
max extent over axes per sample: mean/min/max 0.9991015791893005 0.9912109375 1.0

So the y-axis is always the largest extent (~1.0 for all samples). The bounding box fits the unit cube and the y-axis spans the full unit (the longest dimension is always y, spanning ~1.0). So the normalization scale is tied to the y-extent being 1. This means the scale factor = 1/(y physical extent) roughly, consistent across brackets if they share the same y dimension. Interesting , so the actual physical scale is encoded by the fact that y extent is normalized to 1. If all brackets have the same physical y-length (mounting span), then scale is consistent and mass is well-defined from shape. Good. The x and z extents vary (x: up to 0.64, z: up to 0.45). So shape varies in x,z while y is fixed full. Now let me check reflection symmetry. Since y is the long axis (mounting direction), brackets might be symmetric across x=0 (left-right) or z. Let me compute Chamfer distance between cloud and reflections.

The y-axis always spans the full unit cube (extent ≈ 1.0 for every sample), while x and z vary. That means scale is consistently encoded, so shape→mass is well-defined. Now let me check for reflection symmetry (a potential mass-preserving augmentation) using Chamfer distance between each cloud and its mirror along each axis.

bash

cd /app && python -c "
import torch
d = torch.load('data/train/samples.pt', weights_only=True)
f = d['features'].float().cuda()  # [100,2048,3]

def chamfer(a, b):
    # a,b: [P,3]
    d = torch.cdist(a, b)  # [P,P]
    return (d.min(1).values.mean() + d.min(0).values.mean()).item()

import itertools
for axis in range(3):
    tot=0.0
    for i in range(20):
        c = f[i]
        r = c.clone(); r[:,axis] = -r[:,axis]
        tot += chamfer(c, r)
    print(f'axis {axis} mean chamfer(cloud, mirror) over 20 samples: {tot/20:.5f}')
# baseline: chamfer between distinct samples
tot=0.0
for i in range(20):
    tot += chamfer(f[i], f[(i+1)%20])
print('baseline chamfer(cloud_i, cloud_j):', tot/20)
# self scale: mean nearest-neighbor distance within a cloud
c=f[0]; dd=torch.cdist(c,c); dd.fill_diagonal_(1e9)
print('typical NN dist within cloud:', dd.min(1).values.mean().item())
"

output

axis 0 mean chamfer(cloud, mirror) over 20 samples: 0.08301
axis 1 mean chamfer(cloud, mirror) over 20 samples: 0.03532
axis 2 mean chamfer(cloud, mirror) over 20 samples: 0.20856
baseline chamfer(cloud_i, cloud_j): 0.062350213155150415
typical NN dist within cloud: 0.01402937714010477

Interpretation: - Axis 1 (y) mirror chamfer = 0.035, which is small-ish but larger than NN dist (0.014). Baseline between distinct clouds = 0.062. So y-mirror is closer than distinct clouds but not perfectly symmetric. - Axis 0 (x) mirror = 0.083 (larger than baseline 0.062) , not symmetric. - Axis 2 (z) mirror = 0.209 , definitely not symmetric. So the cleanest approximate symmetry is along y (axis 1), chamfer 0.035 vs NN 0.014. It's approximately symmetric but not exactly. Reflection along y could be a mild augmentation but introduces ~0.035 chamfer noise. Given the mass is preserved under reflection exactly (physical property), y-reflection augmentation could help if the brackets are roughly y-symmetric. But the imperfect symmetry means reflected clouds are slightly out of distribution. Hmm, actually if a bracket is NOT perfectly symmetric, its y-mirror is a DIFFERENT but physically-plausible bracket with the SAME mass (mirror image has identical mass). And mirror images are valid brackets in the same canonical frame (mounting is symmetric). So y-reflection gives a new valid (cloud, mass) pair. This is a legitimate, label-preserving augmentation that expands the dataset, regardless of whether the specific bracket is symmetric! The question is only whether the mirrored cloud is in the same distribution as test data. Since test brackets presumably come from the same design space (which is closed under y-mirror if the problem is symmetric), this should be fine and helpful. But wait , is the design space closed under x or z mirror too? Physically, mirroring produces a valid bracket with same mass. But if the canonical orientation has a specific meaning (e.g., z is loading direction, x is width), a mirrored bracket might not be in the test distribution. The chamfer analysis suggests y-mirror keeps clouds closest to original distribution (0.035), so y-reflection is safest. x-mirror gives 0.083 (further), z-mirror 0.209 (very different). Actually, the fact that y-mirror chamfer (0.035) < baseline (0.062) means y-mirrored clouds look like plausible members of the dataset. x-mirror (0.083) > baseline means x-mirrored clouds look atypical. So I'll use y-reflection as augmentation, and test whether it helps on val. Hmm, but actually let me reconsider: 0.035 is the average over samples; some might be quite symmetric. Let me just treat y-reflection as a candidate augmentation and empirically test it. Now, let me build the training harness and run a baseline. Let me put entire datasets on GPU and write a flexible train function. Let me write lib and run experiments. I'll make the training function return val/test metrics and optionally save checkpoint. Let me think about hyperparameters to start: - optimizer AdamW, lr 1e-3, weight_decay 1e-4 to 1e-2. - epochs: with 100 samples and bs 16 → ~7 batches/epoch. Need many epochs, like 200-500. - cosine schedule with warmup. - augmentation: jitter sigma, point dropout, y-reflection, small rotation. - grad clip 1.0. Let me write and run a first baseline with modest settings, measuring metrics on val and test. I'll write everything in a single reusable script that takes config via function args, and I'll call it for experiments. Let me create /app/exp.py. Actually, to iterate fast and keep state, let me create a library file and run small Python snippets. But re-importing and retraining each time from shell is fine. Let me write /app/lib_train.py. I need to be careful with BatchNorm and small final batch. With 100 samples, bs=16 → last batch has 4 samples. BN with 4 samples×2048 points = 8192 elements per channel, fine. Actually BN1d over [B, C, P] normalizes over B and P dims → plenty of elements. Good. Even bs=4 gives 8192 samples per channel stat. So BN is stable here. Good. For inference, eval mode uses running stats. Need momentum default 0.1, enough updates. Fine. Let me write the code.

y-axis reflection keeps clouds closest to the data distribution (Chamfer 0.035 vs 0.062 between distinct samples), and mirroring is exactly mass-preserving, so it's a promising augmentation. x/z mirrors look out-of-distribution. Let me build a flexible training harness with a metrics function and run a baseline.

bash

mkdir -p /app/outputs && cat > /app/lib_train.py << 'PYEOF'
from __future__ import annotations
import json, os, math, sys
from pathlib import Path
import torch
import torch.nn as nn
import torch.nn.functional as F

APP_DIR = Path("/app")
sys.path.insert(0, str(APP_DIR))
from candidate_model import PointNetRegressor

DEV = torch.device("cuda" if torch.cuda.is_available() else "cpu")

def load_split(name):
    d = torch.load(APP_DIR/"data"/name/"samples.pt", weights_only=True)
    return d["features"].float(), d["labels"].float()

def load_all():
    out = {}
    for s in ["train","val","public_test"]:
        out[s] = load_split(s)
    return out

@torch.no_grad()
def metrics(pred, true):
    pred = pred.flatten().float(); true = true.flatten().float()
    err = pred - true
    rel_l2 = (torch.sqrt((err**2).sum()) / torch.sqrt((true**2).sum())).item()
    mae = err.abs().mean().item()
    maxae = err.abs().max().item()
    return {"rel_l2": rel_l2, "mae": mae, "maxae": maxae}

def augment(x, cfg):
    # x: [B,2048,3] on device, raw coords
    B, P, _ = x.shape
    out = x
    if cfg.get("yreflect", False):
        m = (torch.rand(B,1,device=x.device) < 0.5).float()*(-2)+1  # +1 or -1
        out = out.clone(); out[:,:,1] = out[:,:,1]*m
    if cfg.get("rot_z", 0.0) > 0:
        ang = (torch.rand(B,device=x.device)*2-1)*cfg["rot_z"]*math.pi/180
        c, s = torch.cos(ang), torch.sin(ang)
        R = torch.zeros(B,3,3,device=x.device)
        R[:,0,0]=c; R[:,0,2]=s; R[:,2,0]=-s; R[:,2,2]=c; R[:,1,1]=1
        out = torch.bmm(out, R.transpose(1,2))
    if cfg.get("jitter", 0.0) > 0:
        out = out + torch.randn_like(out)*cfg["jitter"]
    if cfg.get("scale_aniso", 0.0) > 0:
        s = 1.0 + (torch.rand(B,1,3,device=x.device)*2-1)*cfg["scale_aniso"]
        out = out*s
    if cfg.get("dropout", 0.0) > 0:
        # random point dropout: replace dropped points with first point (duplicate)
        keep = (torch.rand(B,P,device=x.device) > cfg["dropout"])
        idx0 = keep.float().argmax(dim=1)  # first kept index per sample
        out = out.clone()
        # for dropped points, copy a random kept point via gather of first kept
        rep = out[torch.arange(B),idx0].unsqueeze(1).expand(-1,P,-1)
        out = torch.where(keep.unsqueeze(-1), out, rep)
    return out

def train_model(data, idx_train, idx_eval_dict, cfg, seed=0, verbose=False, return_model=False):
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    xf, yf = data
    xtr = xf[idx_train].to(DEV); ytr = yf[idx_train].to(DEV)
    feat_mean = xtr.mean(dim=(0,1))
    feat_std = xtr.std(dim=(0,1)).clamp_min(1e-6)
    label_mean = ytr.mean(dim=0)
    label_std = ytr.std(dim=0).clamp_min(1e-6)

    def normx(x): return (x-feat_mean)/feat_std
    def normy(y): return (y-label_mean)/label_std
    def denormy(y): return y*label_std+label_mean

    model = PointNetRegressor(cfg["feature_width"],cfg["global_width"],cfg["head_width"]).to(DEV)
    wd = cfg.get("weight_decay",1e-4)
    opt = torch.optim.AdamW(model.parameters(), lr=cfg["lr"], weight_decay=wd)
    epochs = cfg["epochs"]; bs = cfg["batch_size"]
    n = xtr.shape[0]; steps_per = math.ceil(n/bs); total_steps = epochs*steps_per
    warmup = int(cfg.get("warmup_frac",0.05)*total_steps)
    def lr_at(step):
        if step < warmup: return step/max(1,warmup)
        p = (step-warmup)/max(1,total_steps-warmup)
        return 0.5*(1+math.cos(math.pi*p))
    ytr_n = normy(ytr)
    loss_type = cfg.get("loss","mse"); huber_beta = cfg.get("huber_beta",1.0)
    step=0
    for ep in range(epochs):
        model.train()
        perm = torch.randperm(n, device=DEV)
        for st in range(0,n,bs):
            bidx = perm[st:st+bs]
            xb = xtr[bidx]; yb = ytr_n[bidx]
            xb = augment(xb, cfg)
            xb = normx(xb)
            for g in opt.param_groups: g["lr"]=cfg["lr"]*lr_at(step)
            opt.zero_grad(set_to_none=True)
            pred = model(xb)
            if loss_type=="mse": loss = F.m
…[truncated]

output

written

Now let me run a baseline experiment: train on train, eval on val and public_test. First with no augmentation, then with augmentation. Let me write a quick experiment runner.

Now let me run a baseline (no augmentation) to establish reference metrics on val and public_test.

bash

cd /app && python -c "
import torch, lib_train as L
data = L.load_all()
xtr,ytr = data['train']; xv,yv=data['val']; xte,yte=data['public_test']
dtr = (xtr,ytr)
ev = {'val':(xv,yv),'test':(xte,yte)}
base = dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-4,
            epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
idx = torch.arange(xtr.shape[0])
for seed in [0,1,2]:
    res = L.train_model(dtr, idx, ev, base, seed=seed)
    print('seed',seed,'val',{k:round(v,4) for k,v in res['val'].items()},'test',{k:round(v,4) for k,v in res['test'].items()})
" 2>&1 | grep -v Warning

output

seed 0 val {'rel_l2': 0.1791, 'mae': 0.1236, 'maxae': 0.3634} test {'rel_l2': 0.3494, 'mae': 0.2237, 'maxae': 0.5289}
seed 1 val {'rel_l2': 0.1966, 'mae': 0.1345, 'maxae': 0.3656} test {'rel_l2': 0.3552, 'mae': 0.2289, 'maxae': 0.623}
seed 2 val {'rel_l2': 0.2124, 'mae': 0.1351, 'maxae': 0.4498} test {'rel_l2': 0.292, 'mae': 0.1922, 'maxae': 0.4575}

Baseline (no augmentation): val rel_l2 ~0.18-0.21, test rel_l2 ~0.29-0.35. There's a gap between val and test , overfitting and tiny-set noise. MAE ~0.12-0.22. Max abs err 0.36-0.62. Let me now test augmentations. I'll add jitter, dropout, y-reflection, small rotation, and see effect on val/test. Let me run a sweep of augmentation configs. To reduce noise, average over 3 seeds. Let me write a helper to average metrics over seeds.

Baseline: val rel_L2 ≈ 0.18–0.21, test rel_L2 ≈ 0.29–0.35, with a clear train→test generalization gap. Let me now test augmentations, averaging over seeds to reduce noise from the tiny eval sets.

bash

cd /app && cat > /app/sweep.py << 'PYEOF'
import torch, lib_train as L, numpy as np, itertools, sys
data = L.load_all()
xtr,ytr = data['train']; xv,yv=data['val']; xte,yte=data['public_test']
dtr=(xtr,ytr); ev={'val':(xv,yv),'test':(xte,yte)}
idx=torch.arange(xtr.shape[0])
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-4,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
def run(name, over, seeds=(0,1,2,3)):
    cfg=dict(base); cfg.update(over)
    agg={'val':{},'test':{}}
    for s in seeds:
        r=L.train_model(dtr,idx,ev,cfg,seed=s)
        for sp in ['val','test']:
            for k,v in r[sp].items(): agg[sp].setdefault(k,[]).append(v)
    def fmt(sp): return {k:round(float(np.mean(v)),4) for k,v in agg[sp].items()}
    # combined rel_l2 over val+test as a stable-ish proxy
    print(f'{name:28s} val {fmt(\"val\")} test {fmt(\"test\")}')
    return agg
configs = {
 'none': {},
 'jitter01': {'jitter':0.01},
 'jitter02': {'jitter':0.02},
 'drop0.2': {'dropout':0.2},
 'yrefl': {'yreflect':True},
 'rotz10': {'rot_z':10.0},
 'aniso0.05': {'scale_aniso':0.05},
 'j01+drop+yrefl': {'jitter':0.01,'dropout':0.2,'yreflect':True},
}
for name in sys.argv[1:] if len(sys.argv)>1 else configs:
    run(name, configs[name])
PYEOF
python sweep.py 2>&1 | grep -v Warning

output

File "/app/sweep.py", line 17
    print(f'{name:28s} val {fmt(\"val\")} test {fmt(\"test\")}')
                                                               ^
SyntaxError: f-string expression part cannot include a backslash

bash

cd /app && sed -i 's/fmt(\\"val\\")/fmt("val")/; s/fmt(\\"test\\")/fmt("test")/' sweep.py && sed -i "s/print(f'{name:28s} val {fmt(\"val\")} test {fmt(\"test\")}')/v=fmt('val'); t=fmt('test'); print('%-28s val %s test %s'%(name,v,t))/" sweep.py && sed -n '15,18p' sweep.py

output

def fmt(sp): return {k:round(float(np.mean(v)),4) for k,v in agg[sp].items()}
    # combined rel_l2 over val+test as a stable-ish proxy
    v=fmt('val'); t=fmt('test'); print('%-28s val %s test %s'%(name,v,t))
    return agg

bash

cd /app && python sweep.py 2>&1 | grep -v Warning

output

none                         val {'rel_l2': 0.1979, 'mae': 0.1286, 'maxae': 0.4319} test {'rel_l2': 0.323, 'mae': 0.2107, 'maxae': 0.5478}
jitter01                     val {'rel_l2': 0.2174, 'mae': 0.1537, 'maxae': 0.4198} test {'rel_l2': 0.3264, 'mae': 0.2169, 'maxae': 0.519}
jitter02                     val {'rel_l2': 0.375, 'mae': 0.2481, 'maxae': 0.7507} test {'rel_l2': 0.4527, 'mae': 0.3084, 'maxae': 0.6514}
drop0.2                      val {'rel_l2': 0.1983, 'mae': 0.1332, 'maxae': 0.4324} test {'rel_l2': 0.3158, 'mae': 0.2048, 'maxae': 0.5798}
yrefl                        val {'rel_l2': 0.1979, 'mae': 0.1302, 'maxae': 0.4479} test {'rel_l2': 0.3168, 'mae': 0.1934, 'maxae': 0.5951}
rotz10                       val {'rel_l2': 0.226, 'mae': 0.1418, 'maxae': 0.5356} test {'rel_l2': 0.2893, 'mae': 0.1774, 'maxae': 0.5763}
aniso0.05                    val {'rel_l2': 0.2242, 'mae': 0.1452, 'maxae': 0.4688} test {'rel_l2': 0.3133, 'mae': 0.1984, 'maxae': 0.6133}
j01+drop+yrefl               val {'rel_l2': 0.2059, 'mae': 0.1438, 'maxae': 0.3818} test {'rel_l2': 0.3202, 'mae': 0.2167, 'maxae': 0.4525}

Augmentations give mixed/small effects here. jitter02 clearly hurts. jitter01 slightly hurts val. dropout0.2 slightly helps test. yrefl neutral-ish on val, slightly helps test mae. rotz10 helps test rel_l2 (0.289) but hurts val (0.226). The combined j01+drop+yrefl improves maxae on test (0.4525) and val (0.3818) , good for worst-case. The val and test sets are tiny and noisy, so differences are within noise. The generalization gap (val ~0.20 vs test ~0.32) is concerning , suggests the val set is "easier" or the model overfits in a way that test catches. Actually, the bigger issue: test rel_l2 ~0.32 is quite high. I need a fundamentally better approach. Let me think. The real problem: 100 training samples is very few for a PointNet. The model overfits. Strong regularization (weight decay, dropout in head, augmentation) and possibly reducing effective capacity (but architecture is fixed) could help. I can add dropout via... no, the architecture is fixed. But I can use weight decay more aggressively, and label/feature normalization. Let me reconsider the fundamental signal. What physically determines mass? Volume of material. For a surface point cloud normalized to unit bounding box (y-extent=1), the mass correlates with how "bulky" the shape is. Useful global features: the extents in x and z (mean extent), the spread/variance of points, etc. A PointNet max-pool can capture extents (max coordinate values → bounding box corners) and learned features. Let me check: how well does a simple linear/MLP regression on hand-crafted global features (like per-axis extent, std, etc.) predict mass? This tells me the achievable signal and whether PointNet is underperforming. But I can't change the architecture for the final model. Still, it's informative. Actually, let me check correlation of simple features with mass , if a simple feature like "mean distance from center" or "x-extent * z-extent" strongly predicts mass, then the PointNet should be able to learn it, and I should make sure training captures it. Let me analyze.

Augmentations give only small, noisy effects; `jitter02` clearly hurts. The bigger issue is the val→test generalization gap. Before over-tuning on tiny noisy eval sets, let me understand the underlying signal: how well do simple hand-crafted geometric features predict mass? This tells me what's achievable and whether the PointNet is underfitting the real signal.

bash

cd /app && python -c "
import torch, numpy as np
import lib_train as L
data=L.load_all()
def feats(x):
    # x [N,2048,3]
    ext = (x.amax(1)-x.amin(1))  # [N,3]
    std = x.std(1)  # [N,3]
    mean = x.mean(1)
    absmean = x.abs().mean(1)
    r = x.norm(dim=2)  # [N,2048]
    rad = torch.stack([r.mean(1), r.std(1), r.amax(1)],1)
    # occupancy proxy: voxel count
    feats = torch.cat([ext, std, absmean, rad, ext[:, :1]*ext[:, 2:3], std.prod(1,keepdim=True)],1)
    return feats
Xtr=feats(data['train'][0]).numpy(); ytr=data['train'][1].numpy().ravel()
Xv=feats(data['val'][0]).numpy(); yv=data['val'][1].numpy().ravel()
Xte=feats(data['public_test'][0]).numpy(); yte=data['public_test'][1].numpy().ravel()
# standardize
mu=Xtr.mean(0); sd=Xtr.std(0)+1e-9
Xtr=(Xtr-mu)/sd; Xv=(Xv-mu)/sd; Xte=(Xte-mu)/sd
from numpy.linalg import lstsq
A=np.concatenate([Xtr,np.ones((len(Xtr),1))],1)
# ridge
lam=1.0
W=np.linalg.solve(A.T@A+lam*np.eye(A.shape[1]), A.T@ytr)
def met(X,y):
    p=np.concatenate([X,np.ones((len(X),1))],1)@W
    err=p-y
    return dict(rel_l2=float(np.sqrt((err**2).sum()/(y**2).sum())), mae=float(np.abs(err).mean()), maxae=float(np.abs(err).max()))
print('ridge train',met(Xtr,ytr))
print('ridge val  ',met(Xv,yv))
print('ridge test ',met(Xte,yte))
# correlation of each feature with y
print('corr with mass:', [round(float(np.corrcoef(Xtr[:,i],ytr)[0,1]),3) for i in range(Xtr.shape[1])])
"

output

ridge train {'rel_l2': 0.24549253773831017, 'mae': 0.1664022943419268, 'maxae': 0.7637679320800963}
ridge val   {'rel_l2': 0.2171271437898442, 'mae': 0.1609993498371935, 'maxae': 0.36816204600808455}
ridge test  {'rel_l2': 0.3955622978938344, 'mae': 0.2608529097357678, 'maxae': 0.642132924667487}
corr with mass: [0.234, 0.143, 0.177, 0.635, 0.149, 0.25, 0.568, 0.113, -0.078, 0.394, -0.173, -0.169, 0.249, 0.667]

Interesting. Simple ridge on handcrafted features gets train rel_l2 0.245 (underfits), val 0.217, test 0.395. The strongest single features: index 3 (std-z? let me recount) corr 0.635, index 6 (absmean-z?) 0.568, index 13 (std.prod) 0.667, index 9 (rad max?) 0.394. Feature layout: ext[0,1,2] (0,1,2), std[0,1,2] (3,4,5), absmean[0,1,2] (6,7,8), rad[mean,std,max] (9,10,11), ext0*ext2 (12), std.prod (13). So corr: ext (0.234,0.143,0.177), std (0.635? wait index3 is std[0]=std-x 0.635)... wait index3 corr 0.635 is std-x. index 6 absmean-x 0.568. index13 std.prod 0.667. Hmm so std in x and the product of stds correlate strongly. Makes sense: bulkier brackets (more spread in x) weigh more. The PointNet got test rel_l2 ~0.29-0.32, which is actually BETTER than this simple ridge (0.395). So PointNet is already extracting more than these handcrafted features. Good , the architecture isn't the bottleneck necessarily. But both have high test error (~0.3-0.4). The test set of 16 seems genuinely hard / different. Let me look at whether there's a distribution shift, or whether a few test samples are outliers driving rel_l2. Actually, rel_l2 on test = 0.29-0.39. Let me look at per-sample errors on test to see if it's a few bad ones (which would inflate rel_l2 and maxae). Let me examine the best PointNet model's predictions on test. Actually, more importantly, let me reconsider: the combined labeled data is 148 samples. For the FINAL model, I'll train on all 148. But to estimate generalization, I should do cross-validation across all 148, not rely on the specific val/test split. The val set might be unusually easy and test unusually hard. Let me do k-fold CV over the pooled 148 to get a robust estimate and to tune. Let me set up 5-fold or 6-fold CV on pooled data. With 148 samples, 6 folds ≈ 25 per fold. I'll average metrics across folds and seeds. Actually, rel_l2 computed per-fold (25 samples) then averaged is a reasonable proxy. Better: pool all out-of-fold predictions then compute metrics once over all 148 (like a full-set rel_l2). That mirrors the grading (rel_l2 over full set). Let me do out-of-fold (OOF) predictions: for each fold, train on the other folds, predict the held-out fold; collect all predictions; compute metrics over the full 148. This gives a single robust estimate. Let me implement CV/OOF evaluation. This is the right tool to reduce noise and tune. Let me write cv.py.

The PointNet (test rel_L2 ≈ 0.29–0.32) already beats handcrafted-feature ridge (0.395), so the architecture isn't the main bottleneck , noise and the small train set are. The fixed val/test split is tiny and noisy (val looks easy, test hard). I'll switch to **out-of-fold cross-validation over the pooled 148 labeled samples** to get a robust, grading-like rel_L2 estimate for tuning.

bash

cd /app && cat > /app/cv.py << 'PYEOF'
import torch, numpy as np, lib_train as L, sys, math
from lib_train import DEV
import torch.nn as nn, torch.nn.functional as F
from candidate_model import PointNetRegressor

data=L.load_all()
X=torch.cat([data['train'][0],data['val'][0],data['public_test'][0]],0)
Y=torch.cat([data['train'][1],data['val'][1],data['public_test'][1]],0)
N=X.shape[0]
print('pooled N=',N)

def train_predict(xtr,ytr,xte,cfg,seed):
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    xtr=xtr.to(DEV); ytr=ytr.to(DEV); xte=xte.to(DEV)
    fm=xtr.mean((0,1)); fs=xtr.std((0,1)).clamp_min(1e-6)
    lm=ytr.mean(0); ls=ytr.std(0).clamp_min(1e-6)
    nx=lambda x:(x-fm)/fs; ny=lambda y:(y-lm)/ls; dy=lambda y:y*ls+lm
    model=PointNetRegressor(cfg['feature_width'],cfg['global_width'],cfg['head_width']).to(DEV)
    opt=torch.optim.AdamW(model.parameters(),lr=cfg['lr'],weight_decay=cfg['weight_decay'])
    n=xtr.shape[0]; bs=cfg['batch_size']; epochs=cfg['epochs']
    total=epochs*math.ceil(n/bs); warm=int(cfg.get('warmup_frac',0.05)*total)
    def lrf(s):
        if s<warm: return s/max(1,warm)
        p=(s-warm)/max(1,total-warm); return 0.5*(1+math.cos(math.pi*p))
    ytrn=ny(ytr); step=0
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=DEV)
        for st in range(0,n,bs):
            bidx=perm[st:st+bs]; xb=xtr[bidx]; yb=ytrn[bidx]
            xb=L.augment(xb,cfg); xb=nx(xb)
            for g in opt.param_groups: g['lr']=cfg['lr']*lrf(step)
            opt.zero_grad(set_to_none=True); pred=model(xb)
            lt=cfg.get('loss','mse')
            if lt=='mse': loss=F.mse_loss(pred,yb)
            elif lt=='huber': loss=F.smooth_l1_loss(pred,yb,beta=cfg.get('huber_beta',1.0))
            elif lt=='l1': loss=F.l1_loss(pred,yb)
            loss.backward(); nn.utils.clip_grad_norm_(model.parameters(),cfg.get('clip',1.0)); opt.step(); step+=1
    model.eval()
    with torch.no_grad():
        p=dy(model(nx(xte))).cpu()
    return p

def oof(cfg, seeds=(0,1), kfold=6, folds_seed=123):
    g=torch.Generator().manual_seed(folds_seed)
    perm=torch.randperm(N,generator=g)
    foldid=torch.zeros(N,dtype=torch.long); 
    for i,idx in enumerate(perm): foldid[idx]=i%kfold
    preds=torch.zeros(len(seeds),N,1)
    for si,seed in enumerate(seeds):
        for f in range(kfold):
            te=(foldid==f).nonzero().ravel(); tr=(foldid!=f).nonzero().ravel()
            preds[si,te]=train_predict(X[tr],Y[tr],X[te],cfg,seed)
    pmean=preds.mean(0)
    m=L.metrics(pmean,Y)
    # also per-seed variability
    return m, preds

if __name__=='__main__':
    base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-4,
              epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
    m,_=oof(base,seeds=(0,1))
    print('baseline OOF',{k:round(v,4) for k,v in m.items()})
PYEOF
python cv.py 2>&1 | grep -v Warning

output

pooled N= 148
baseline OOF {'rel_l2': 0.2334, 'mae': 0.1398, 'maxae': 0.6915}

OK, so pooled OOF rel_l2 baseline = 0.233, mae 0.140, maxae 0.69. This is a more robust estimate (over 148 samples, 2 seeds). The maxae 0.69 is large , a worst-case sample. Now let me sweep key hyperparameters using this OOF metric: weight decay, epochs, augmentation, loss type, lr. Let me build a sweeper that runs OOF for several configs. This is more expensive (6 folds × seeds), but data is tiny so each train is fast. Let me time one run first. Actually the baseline OOF with 2 seeds × 6 folds = 12 trainings took some seconds. Let me check timing and then sweep.

Pooled OOF gives a robust baseline: rel_L2 = 0.233, MAE = 0.140, max abs err = 0.69 (the worst-case is large). Now let me sweep the main regularization knobs against this OOF metric. First a timing check.

bash

cd /app && python -c "
import time, torch, cv
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-4,epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
t=time.time()
m,_=cv.oof(base,seeds=(0,),kfold=6)
print('one seed 6fold time %.1fs'%(time.time()-t), {k:round(v,3) for k,v in m.items()})
" 2>&1 | grep -v Warning

output

pooled N= 148
one seed 6fold time 43.8s {'rel_l2': 0.235, 'mae': 0.143, 'maxae': 0.706}

~44s per seed for 6-fold at 200 epochs. Let me run sweeps efficiently. I'll use 2 seeds for each config to balance noise and time (~90s each). Let me sweep weight decay and epochs and augmentation together in a batch, running in background and logging. Let me be strategic. Key knobs: 1. weight_decay: 1e-4, 1e-3, 1e-2, 3e-2 2. epochs: 150, 300, 400 3. augmentation combos: none, jitter0.005, dropout0.2, yrefl, rotz5, and combos 4. loss: mse vs huber 5. lr: 1e-3, 2e-3, 5e-4 Let me run a sweep over weight_decay and augmentation first since those most affect generalization. I'll write a script that runs a list and prints results, run in background to a log. To save time, let me use 2 seeds and maybe 4-fold (37/fold). Actually 6-fold gives more training data per fold (123 train). Let me keep 6-fold, 2 seeds. ~90s each. I'll run ~8 configs = ~12 min. Acceptable. Let me run in background. Actually, let me parallelize by running multiple configs in one Python process sequentially but log each. Let me just run a focused sweep. Let me prioritize: Phase 1: weight decay sweep (no aug): wd in {1e-4, 3e-4, 1e-3, 3e-3, 1e-2}. Phase 2: with best wd, augmentation sweep. Phase 3: epochs & lr. Phase 4: loss type. Let me run Phase 1 now.

About 44s per seed. I'll sweep with 2 seeds × 6-fold (~90s/config), logging to a file in the background. Phase 1: weight decay (strong regularization is likely key for 100-sample training).

bash

cd /app && cat > /app/run_sweep.py << 'PYEOF'
import torch, cv, sys, time, numpy as np
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-4,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
def ev(name,over,seeds=(0,1,2)):
    cfg=dict(base); cfg.update(over)
    t=time.time(); m,_=cv.oof(cfg,seeds=seeds)
    print('%-26s rel_l2=%.4f mae=%.4f maxae=%.4f (%.0fs)'%(name,m['rel_l2'],m['mae'],m['maxae'],time.time()-t),flush=True)
name=sys.argv[1]
grid={
 'wd1e-4':{'weight_decay':1e-4},
 'wd3e-4':{'weight_decay':3e-4},
 'wd1e-3':{'weight_decay':1e-3},
 'wd3e-3':{'weight_decay':3e-3},
 'wd1e-2':{'weight_decay':1e-2},
 'wd3e-2':{'weight_decay':3e-2},
}
if name=='wd':
    for k,v in grid.items(): ev(k,v)
PYEOF
nohup python run_sweep.py wd > /app/sweep_wd.log 2>&1 &
echo "started pid $!"

output

started pid 223

bash

sleep 200 && cat /app/sweep_wd.log | grep -v Warning

output

pooled N= 148
wd1e-4                     rel_l2=0.2288 mae=0.1364 maxae=0.6752 (122s)

3 seeds × 6 folds = 18 trainings = 122s per config. 6 configs ≈ 12 min. Let me wait more.

3 seeds takes ~122s per config. Let me wait for the full weight-decay sweep.

bash

sleep 280 && cat /app/sweep_wd.log | grep -v Warning

output

pooled N= 148
wd1e-4                     rel_l2=0.2288 mae=0.1364 maxae=0.6752 (122s)
wd3e-4                     rel_l2=0.2247 mae=0.1358 maxae=0.7072 (107s)
wd1e-3                     rel_l2=0.2244 mae=0.1365 maxae=0.6274 (101s)
wd3e-3                     rel_l2=0.2254 mae=0.1350 maxae=0.7262 (116s)

bash

sleep 180 && cat /app/sweep_wd.log | grep -v Warning

output

pooled N= 148
wd1e-4                     rel_l2=0.2288 mae=0.1364 maxae=0.6752 (122s)
wd3e-4                     rel_l2=0.2247 mae=0.1358 maxae=0.7072 (107s)
wd1e-3                     rel_l2=0.2244 mae=0.1365 maxae=0.6274 (101s)
wd3e-3                     rel_l2=0.2254 mae=0.1350 maxae=0.7262 (116s)
wd1e-2                     rel_l2=0.2260 mae=0.1367 maxae=0.6842 (119s)

Weight decay has little effect (rel_l2 0.224-0.229 across the range). wd=1e-3 gives best rel_l2 (0.2244) and best maxae (0.627). So weight decay alone isn't the lever. The model's generalization is plateauing around rel_l2 ≈ 0.225. This suggests the bottleneck is elsewhere. Let me think harder. The OOF rel_l2 ~0.225 means typical relative error ~22%. That's high. Given the mass std/mean ≈ 0.39/0.75 ≈ 0.52, a model predicting the mean would get rel_l2 = std-ish/rms. Let me compute the "predict mean" baseline rel_l2: rel_l2 = ||y-ybar||/||y|| = std*sqrt((N-1)/N) / rms. rms = sqrt(mean(y^2)) = sqrt(mean^2+std^2)=sqrt(0.75^2+0.39^2)=sqrt(0.5625+0.152)=sqrt(0.714)=0.845. ||y-ybar||/||y|| = 0.39/0.845 ≈ 0.46. So predicting the mean gives rel_l2 ~0.46. Our model at 0.225 explains about... R^2-ish. rel_l2^2 = 0.0506 vs 0.21 → explains ~76% of variance. Not bad but not great. How to push further? Ideas: 1. The PointNet max-pool might be losing info. But architecture is fixed. However, I control input normalization. What if I feed richer geometric signal by transforming coordinates? The input must be [B,2048,3], but I could apply a nonlinear per-point transform via feat_mean/feat_std? No, that's just affine per-axis. Actually wait , feat_mean and feat_std are applied as (x - feat_mean)/feat_std. That's it. Can't add channels. 2. Test-time augmentation (TTA): At inference, the harness just calls model(points). I can't add TTA since the harness controls inference. Unless... the harness normalizes then calls forward. I only provide normalization constants and weights. So no TTA. Hmm, wait. Let me re-read the task. "Your model is then applied to a held-out set... predictions compared against ground truth." The harness loads state_dict into PointNetRegressor, applies feat normalization, forward, denormalizes. So I truly only control weights + normalization. No architecture/inference changes. 3. Given fixed architecture, the main lever is better training to generalize. Options: - More epochs / better schedule. - Better augmentation (but sweep showed small effects). - Ensemble into a single model via weight averaging (SWA) , Stochastic Weight Averaging can improve generalization AND produces a single set of weights! SWA averages weights along the trajectory. That's compatible with a single checkpoint. Let me try SWA. - Also, averaging multiple independently-trained models' weights generally does NOT work due to permutation symmetry. But SWA (averaging along one trajectory with cyclic/high constant LR) works. Let me implement SWA: after initial training, switch to a moderate constant LR and average weights every epoch, then recompute BN statistics with a forward pass over training data. 4. Data: I'm already using all 148 for final (in CV, 123/fold). For final model train on all 148. Let me focus on the things most likely to help: - SWA / weight averaging along trajectory (single checkpoint, better generalization). - EMA of weights (similar benefit, single checkpoint). - Tuning epochs and LR schedule. Let me also reconsider whether the PointNet underfits or overfits. Let me check train vs OOF. If train rel_l2 is near 0 but OOF 0.22 → overfit. Let me quickly check training error at the end. Actually from the earlier sweep script, baseline train wasn't printed. Let me reason: with 100-123 samples and this capacity, training error likely near 0. So it's overfitting. Regularization via augmentation/weight-averaging should help, but the augmentation sweep was weak. Let me try combining mild augmentation + EMA + more epochs. Let me also reconsider augmentation magnitudes. jitter0.01 slightly hurt; maybe 0.005 helps. dropout0.2 was neutral/slightly positive. Let me try dropout with resampling differently. Also combine yrefl (doubles data, label-preserving). Let me implement EMA in the training and test it in CV. EMA of parameters is straightforward; I then need to set BN running stats appropriately. With EMA applied to all para …[truncated]

Weight decay barely moves the needle (rel_L2 ≈ 0.224–0.229); the model plateaus around 0.225 and is overfitting the tiny train set. Since I can only ship a single checkpoint (no inference-time ensembling/TTA), the most promising lever is **weight averaging along the trajectory (EMA/SWA)**, which improves generalization while still producing one set of weights. Let me add EMA support and test it in CV.

bash

cd /app && cat > /app/cv2.py << 'PYEOF'
import torch, numpy as np, lib_train as L, math, copy
from lib_train import DEV
import torch.nn as nn, torch.nn.functional as F
from candidate_model import PointNetRegressor

data=L.load_all()
X=torch.cat([data['train'][0],data['val'][0],data['public_test'][0]],0)
Y=torch.cat([data['train'][1],data['val'][1],data['public_test'][1]],0)
N=X.shape[0]

def train_predict(xtr,ytr,xte,cfg,seed):
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    xtr=xtr.to(DEV); ytr=ytr.to(DEV); xte=xte.to(DEV)
    fm=xtr.mean((0,1)); fs=xtr.std((0,1)).clamp_min(1e-6)
    lm=ytr.mean(0); ls=ytr.std(0).clamp_min(1e-6)
    nx=lambda x:(x-fm)/fs; ny=lambda y:(y-lm)/ls; dy=lambda y:y*ls+lm
    model=PointNetRegressor(cfg['feature_width'],cfg['global_width'],cfg['head_width']).to(DEV)
    opt=torch.optim.AdamW(model.parameters(),lr=cfg['lr'],weight_decay=cfg['weight_decay'])
    n=xtr.shape[0]; bs=cfg['batch_size']; epochs=cfg['epochs']
    total=epochs*math.ceil(n/bs); warm=int(cfg.get('warmup_frac',0.05)*total)
    swa_start=cfg.get('swa_start_frac',1.01)  # >1 disables
    swa_lr=cfg.get('swa_lr',cfg['lr']*0.5)
    def lrf(s):
        if s<warm: return s/max(1,warm)
        p=(s-warm)/max(1,total-warm); return 0.5*(1+math.cos(math.pi*p))
    ytrn=ny(ytr); step=0
    ema=None; ema_decay=cfg.get('ema',0.0)
    if ema_decay>0: ema={k:v.detach().clone().float() for k,v in model.state_dict().items()}
    swa_state=None; swa_count=0
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=DEV)
        in_swa = ep>=int(swa_start*epochs)
        for st in range(0,n,bs):
            bidx=perm[st:st+bs]; xb=xtr[bidx]; yb=ytrn[bidx]
            xb=L.augment(xb,cfg); xb=nx(xb)
            cur_lr = swa_lr if in_swa else cfg['lr']*lrf(step)
            for g in opt.param_groups: g['lr']=cur_lr
            opt.zero_grad(set_to_none=True); pred=model(xb)
            lt=cfg.get('loss','mse')
            if lt=='mse': loss=F.mse_loss(pred,yb)
            elif lt=='huber': loss=F.smooth_l1_loss(pred,yb,beta=cfg.get('huber_beta',1.0))
            elif lt=='l1': loss=F.l1_loss(pred,yb)
            loss.backward(); nn.utils.clip_grad_norm_(model.parameters(),cfg.get('clip',1.0)); opt.step(); step+=1
            if ema_decay>0:
                sd=model.state_dict()
                for k in ema:
                    if ema[k].dtype.is_floating_point: ema[k].mul_(ema_decay).add_(sd[k].float(),alpha=1-ema_decay)
                    else: ema[k]=sd[k].clone()
        if in_swa:
            sd=model.state_dict()
            if swa_state is None: swa_state={k:v.detach().clone().float() for k,v in sd.items()}
            else:
                for k in swa_state:
                    if swa_state[k].dtype.is_floating_point: swa_state[k].mul_(swa_count/(swa_count+1)).add_(sd[k].float(),alpha=1/(swa_count+1))
            swa_count+=1
    # choose final weights
    mode=cfg.get('final','last')
    if mode=='ema' and ema is not None:
        model.load_state_dict({k:(ema[k] if ema[k].dtype.is_floating_point else ema[k]) for k in ema})
    elif mode=='swa' and swa_state is not None:
        model.load_state_dict(swa_state)
        # recompute BN stats
        model.train()
        for m in model.modules():
            if isinstance(m,nn.BatchNorm1d): m.reset_running_stats(); m.momentum=None
        with torch.no_grad():
            for _ in range(cfg.get('bn_passes',3)):
                perm=torch.randperm(n,device=DEV)
                for st in range(0,n,bs):
                    bidx=perm[st:st+bs]; xb=nx(L.augment(xtr[bidx],cfg)); model(xb)
    model.eval()
    with torch.no_grad():
        p=dy(model(nx(xte))).cpu()
    return p

def oof(cfg, seeds=(0,1,2), kfold=6, folds_seed=123):
    g=torch.Generator().manual_seed(folds_seed)
    perm=torch.randperm(N,generator=g)
    foldid=torch.zeros(N,dtype=torch.long)
    for i,idx in enumerate(perm): foldid[idx]=i%kfold
    preds=torch.zeros(len(seeds),N,1)
    for 
…[truncated]

output

ok

bash

cd /app && cat > /app/run2.py << 'PYEOF'
import cv2 as cv, time, sys
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-3,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
def ev(name,over,seeds=(0,1,2)):
    cfg=dict(base); cfg.update(over)
    t=time.time(); m,_=cv.oof(cfg,seeds=seeds)
    print('%-22s rel_l2=%.4f mae=%.4f maxae=%.4f (%.0fs)'%(name,m['rel_l2'],m['mae'],m['maxae'],time.time()-t),flush=True)
ev('last(base)',{})
ev('ema0.99',{'ema':0.99,'final':'ema'})
ev('ema0.995',{'ema':0.995,'final':'ema'})
ev('ema0.999',{'ema':0.999,'final':'ema'})
ev('swa0.75',{'final':'swa','swa_start_frac':0.75,'swa_lr':5e-4})
ev('swa0.6',{'final':'swa','swa_start_frac':0.6,'swa_lr':5e-4})
PYEOF
nohup python run2.py > /app/run2.log 2>&1 &
echo "pid $!"

output

pid 272

bash

sleep 290 && cat /app/run2.log | grep -v Warning

output

last(base)             rel_l2=0.2244 mae=0.1365 maxae=0.6274 (125s)
ema0.99                rel_l2=0.2244 mae=0.1364 maxae=0.6313 (129s)

EMA 0.99 basically identical to last (makes sense, with cosine decay to ~0, the last epochs have tiny LR so weights barely move, EMA ≈ last). I need EMA with a higher decay relative to update count, or SWA with constant moderate LR. Let me wait for the rest.

bash

sleep 290 && cat /app/run2.log | grep -v Warning

output

last(base)             rel_l2=0.2244 mae=0.1365 maxae=0.6274 (125s)
ema0.99                rel_l2=0.2244 mae=0.1364 maxae=0.6313 (129s)
ema0.995               rel_l2=0.2249 mae=0.1370 maxae=0.6292 (120s)
ema0.999               rel_l2=0.4609 mae=0.2947 maxae=1.4857 (129s)

EMA 0.999 is too slow to catch up (only ~1500 steps total, decay 0.999 → effective window ~1000 steps, lags badly). EMA doesn't help because of cosine decay. Let me wait for SWA results.

bash

sleep 250 && cat /app/run2.log | grep -v Warning

output

last(base)             rel_l2=0.2244 mae=0.1365 maxae=0.6274 (125s)
ema0.99                rel_l2=0.2244 mae=0.1364 maxae=0.6313 (129s)
ema0.995               rel_l2=0.2249 mae=0.1370 maxae=0.6292 (120s)
ema0.999               rel_l2=0.4609 mae=0.2947 maxae=1.4857 (129s)
swa0.75                rel_l2=0.2239 mae=0.1361 maxae=0.6573 (121s)
swa0.6                 rel_l2=0.2238 mae=0.1362 maxae=0.6445 (120s)

SWA gives essentially the same (0.2238-0.2239 vs 0.2244). No meaningful improvement. The model is stuck at ~0.224 rel_l2. So weight averaging doesn't help here. The plateau at rel_l2 ≈ 0.224 seems to be a genuine limit of what this PointNet extracts from the data with MSE training. Let me step back and think about what could be limiting: 1. The mass depends on volume/material distribution, which surface points + max-pooling struggle to capture. Max pooling captures only the max of each learned feature across points , essentially describing the "silhouette/extent" in feature space, not density or volume well. 2. But I can't change the architecture. However, I CAN change what I feed in via normalization... only affine. Wait , actually, I realize there might be a more important lever: the number of points and how they're used. The architecture does max over dim=2 (points). That's fixed. Let me reconsider the problem. Is rel_l2 0.224 actually bad? Let me compare to what's achievable. Let me check: what does a k-NN in a good feature space give? Let me compute OOF with a strong model like gradient boosting on richer handcrafted features (histograms of coordinates, pairwise distance stats, etc.) to gauge the ceiling. If even strong classical models can't beat ~0.2, then 0.22 is near the noise floor (recall the scale ambiguity could inject irreducible noise). Actually, recall the potential irreducible noise: brackets rescaled so AABB fits unit cube, with y-extent always =1. If the physical y-length varies across brackets, then the normalization scale varies and mass isn't determined by normalized shape alone. Let me check if y-extent is EXACTLY 1 for all (which would mean consistent scale) , earlier: max extent over axes per sample mean 0.9991, min 0.9912, max 1.0. And extent along y: mean 0.9991, max 1.0. So y-extent ranges 0.991-1.0, essentially always ~1 (the 0.991 is due to point sampling not hitting the exact extreme). So scale is consistent: every bracket's physical y-length maps to 1.0. Good , minimal scale ambiguity. So mass IS determined by normalized shape (assuming constant density and consistent scale). So there's a real deterministic function to learn; 0.22 is not forced by scale noise. So why can't we do better? Likely: (a) only 148 samples total, (b) PointNet max-pool limitations, (c) the function is complex. Let me test the classical ceiling with richer features + a powerful regressor (e.g., k-NN or random forest / GBM) via OOF. Let me build a rich feature vector: coordinate histograms (marginal distributions), radial distribution, pairwise distance distribution, PCA eigenvalues, etc. Then GBM/RF OOF. Actually scikit-learn may be available. Let me check. This will tell me the ceiling and whether to invest more. Let me engineer strong features and try RandomForest/GradientBoosting and k-NN with OOF on pooled 148. Let me think about the most informative features for mass (volume of material): - Occupancy on a voxel grid: count of occupied voxels ≈ surface area proxy, and the convex hull volume. But volume of a thin shell vs solid differs. Still, a voxelization of surface points at resolution R: number of occupied voxels correlates with surface area. Mass correlates with volume. For brackets of similar "thickness," surface area correlates with volume. Hmm. - Better: estimate volume via voxel occupancy "filled" , but we only have surface points, filling requires solidification. Could approximate: for each (x,y) column, the z-range occupied → integrate. But noisy. - Marginal histograms of x, y, z (say 16 bins each) capture the distribution of surface points. - Pairwise distance histogram (shape distribution D2) , classic shape descriptor, 2048 points → sample pairs. - PCA eigenvalues of point cloud (3 values) , overall shape. - Extents, stds, etc. Let me compute these and run RF/GBM/kNN OOF to find the ceiling. Let me check sklearn availability.

SWA/EMA give no real gain , the PointNet plateaus at rel_L2 ≈ 0.224. I confirmed scale is consistent (y-extent ≈ 1.0 for all), so mass is a genuine deterministic function of the normalized shape; 0.224 isn't a scale-noise floor. Let me estimate the achievable **ceiling** with classical models on rich hand-crafted shape descriptors (OOF), to judge whether the bottleneck is the architecture or the data itself.

bash

cd /app && python -c "import sklearn, scipy; print('sklearn',sklearn.__version__,'scipy',scipy.__version__)" 2>&1 | tail -1

output

ModuleNotFoundError: No module named 'sklearn'

bash

cd /app && pip install scikit-learn 2>&1 | tail -2

output

ERROR: Could not find a version that satisfies the requirement scikit-learn (from versions: none)
ERROR: No matching distribution found for scikit-learn

No internet, no sklearn. I'll implement a simple k-NN and ridge/GBM-lite myself in numpy/torch. Let me build rich features and do OOF with k-NN regression and ridge (I already did ridge). Let me add k-NN and a simple kernel ridge (RBF) which can capture nonlinearity. Kernel ridge is easy to implement with numpy and is powerful for small data. Let me engineer rich features and test kernel ridge (RBF) + kNN OOF. This estimates the ceiling. Features per cloud: - marginal histograms of x,y,z (e.g., 24 bins each over [-0.5,0.5]) → 72 - radial distance histogram (r = norm) 24 bins - pairwise distance D2 histogram: sample 4096 random pairs, distances, 24 bins - PCA eigenvalues (3) - extents (3), stds (3), absmeans(3) - voxel occupancy count at a few resolutions Let me compute and run kernel ridge with RBF, tuning bandwidth and lambda via nested OOF (or just pick reasonable). Let me do it. Actually, kernel ridge with proper CV will show the ceiling. Let me implement.

No internet/sklearn, so I'll implement kernel ridge (RBF) and k-NN myself , both strong for small data. Let me build rich shape descriptors and run OOF to estimate the ceiling.

bash

cd /app && cat > /app/ceiling.py << 'PYEOF'
import torch, numpy as np, lib_train as L
data=L.load_all()
X=torch.cat([data['train'][0],data['val'][0],data['public_test'][0]],0)
Y=torch.cat([data['train'][1],data['val'][1],data['public_test'][1]],0).numpy().ravel()
N=X.shape[0]
dev='cuda'
Xg=X.to(dev)
def hist(vals, lo, hi, b):  # vals [N,P]
    edges=torch.linspace(lo,hi,b+1,device=vals.device)
    idx=torch.bucketize(vals, edges[1:-1].contiguous())
    oh=torch.zeros(vals.shape[0],b,device=vals.device)
    oh.scatter_add_(1, idx, torch.ones_like(vals))
    return oh/vals.shape[1]
def features(Xg):
    feats=[]
    for a in range(3): feats.append(hist(Xg[:,:,a],-0.5,0.5,24))
    r=Xg.norm(dim=2); feats.append(hist(r,0,0.75,24))
    # pairwise D2 on subsample
    P=Xg.shape[1]; 
    i=torch.randint(0,P,(4096,),device=Xg.device); j=torch.randint(0,P,(4096,),device=Xg.device)
    d=(Xg[:,i]-Xg[:,j]).norm(dim=2)  # [N,4096]
    feats.append(hist(d,0,1.2,24))
    ext=Xg.amax(1)-Xg.amin(1); std=Xg.std(1); am=Xg.abs().mean(1)
    feats += [ext,std,am]
    # PCA eigenvalues
    Xc=Xg-Xg.mean(1,keepdim=True)
    cov=torch.einsum('npi,npj->nij',Xc,Xc)/P
    ev=torch.linalg.eigvalsh(cov)  # [N,3]
    feats.append(ev)
    # voxel occupancy at res 16 and 32
    for res in [12,20,32]:
        q=((Xg+0.5).clamp(0,0.999)*res).long()
        code=(q[:,:,0]*res+q[:,:,1])*res+q[:,:,2]
        occ=torch.zeros(Xg.shape[0],res**3,device=Xg.device)
        occ.scatter_(1,code,1.0)
        feats.append(occ.sum(1,keepdim=True)/ (res**3))
    return torch.cat(feats,1).cpu().numpy()
torch.manual_seed(0)
F=features(Xg)
print('feat dim',F.shape)
def metrics(p,y):
    e=p-y; return dict(rel_l2=float(np.sqrt((e**2).sum()/(y**2).sum())),mae=float(np.abs(e).mean()),maxae=float(np.abs(e).max()))
# OOF kfold
def oof_krr(F,Y,gamma,lam,k=6,seed=123):
    rng=np.random.default_rng(seed); idx=rng.permutation(N); fold=np.zeros(N,int)
    for i,v in enumerate(idx): fold[v]=i%k
    pred=np.zeros(N)
    mu=F.mean(0); sd=F.std(0)+1e-9; Fs=(F-mu)/sd
    for f in range(k):
        te=fold==f; tr=~te
        Xtr=Fs[tr]; ytr=Y[tr]; Xte=Fs[te]
        D=((Xtr[:,None]-Xtr[None])**2).sum(2)
        Kt=np.exp(-gamma*D)
        a=np.linalg.solve(Kt+lam*np.eye(tr.sum()), ytr-ytr.mean())
        Dte=((Xte[:,None]-Xtr[None])**2).sum(2)
        Kte=np.exp(-gamma*Dte)
        pred[te]=Kte@a+ytr.mean()
    return metrics(pred,Y)
def oof_knn(F,Y,kk,k=6,seed=123):
    rng=np.random.default_rng(seed); idx=rng.permutation(N); fold=np.zeros(N,int)
    for i,v in enumerate(idx): fold[v]=i%k
    pred=np.zeros(N); mu=F.mean(0); sd=F.std(0)+1e-9; Fs=(F-mu)/sd
    for f in range(k):
        te=fold==f; tr=~te
        D=((Fs[te][:,None]-Fs[tr][None])**2).sum(2)
        nn=np.argsort(D,1)[:,:kk]
        pred[te]=Y[tr][nn].mean(1)
    return metrics(pred,Y)
best=None
for gamma in [0.001,0.003,0.01,0.03,0.1]:
    for lam in [0.01,0.03,0.1,0.3,1.0]:
        m=oof_krr(F,Y,gamma,lam)
        if best is None or m['rel_l2']<best[0]: best=(m['rel_l2'],gamma,lam,m)
print('best KRR',round(best[0],4),'gamma',best[1],'lam',best[2],{k:round(v,4) for k,v in best[3].items()})
for kk in [3,5,8,12]:
    print('kNN k=%d'%kk,{k:round(v,4) for k,v in oof_knn(F,Y,kk).items()})
PYEOF
python ceiling.py 2>&1 | grep -v Warning

output

idx=torch.bucketize(vals, edges[1:-1].contiguous())
feat dim (148, 135)
best KRR 0.2359 gamma 0.003 lam 0.01 {'rel_l2': 0.2359, 'mae': 0.1462, 'maxae': 0.8923}
kNN k=3 {'rel_l2': 0.2908, 'mae': 0.1852, 'maxae': 0.7609}
kNN k=5 {'rel_l2': 0.3067, 'mae': 0.1862, 'maxae': 0.8381}
kNN k=8 {'rel_l2': 0.3099, 'mae': 0.1914, 'maxae': 0.9578}
kNN k=12 {'rel_l2': 0.3117, 'mae': 0.1947, 'maxae': 1.0819}

Kernel ridge on rich features: best OOF rel_l2 0.236 , slightly WORSE than the PointNet (0.224). k-NN worse (0.29+). So the PointNet at 0.224 is actually already competitive with/better than classical approaches on these features. This strongly suggests rel_l2 ≈ 0.22 is near the achievable ceiling given 148 samples and the information in the point clouds. The limiting factor is data quantity and intrinsic difficulty, not the model family. So I should: 1. Make sure the final model is as good as possible (~0.22) and robust. 2. Consider combining PointNet predictions with the KRR via... no, must be single PointNet checkpoint. Can't combine at inference. Wait, but maybe I can improve by reconsidering the target. rel_l2 weights larger masses more (denominator is fixed ||y||, numerator sums squared errors). Actually all samples contribute equally to numerator via squared error. To reduce rel_l2, reduce large squared errors (the worst predictions). The maxae is large (~0.63). A few hard samples dominate. Let me look at which samples have large OOF errors , are they extreme-mass samples (very heavy)? If the model systematically underpredicts heavy brackets (regression to mean), that hurts. Let me examine the OOF residuals vs true mass. If there's a bias (shrinkage toward mean), I can correct it (e.g., scale predictions, or train with a loss that penalizes large values more, or adjust label normalization). A classic fix for regression-to-mean shrinkage: the model underpredicts high values and overpredicts low values. A linear recalibration p' = a*p + b fit on OOF could help, but I can't postprocess at inference beyond label_mean/label_std (which is linear!). Actually label_mean/label_std IS a linear map applied at inference: output mapped back via y*label_std+label_mean. If I adjust label_std/label_mean to counteract shrinkage, that's a global linear recalibration I CAN bake in! So: if OOF predictions show p ≈ shrink*y + offset (regression to mean), I can fix by choosing label_std larger to de-shrink. Let me examine the shrinkage. Let me analyze OOF predictions vs true for the PointNet baseline. Let me get the OOF preds and fit a line. Let me reconsider. The model outputs in normalized space z, then denorm: pred = z*label_std + label_mean. During training, target z_true = (y - label_mean)/label_std, model learns to predict z. If model regresses toward mean in z-space (predicts a*z_true), then pred = a*z_true*label_std+label_mean = a*(y-label_mean)+label_mean = a*y + (1-a)*label_mean. So slope a<1 shrinkage. To correct, I want pred_corrected = y. If I set label_std' = label_std/a (and keep label_mean), then at inference pred' = z*label_std/a + label_mean... but z is the model output which ≈ a*z_true. So pred' = a*z_true*label_std/a+label_mean = z_true*label_std+label_mean = y. But wait , the model is trained with specific label_std. If I change label_std at inference only, that rescales. Since during training the model sees targets normalized by label_std, and learns to output a*z_true. At inference harness does pred=out*label_std+label_mean. If I store a modified label_std_infer = label_std/a, predictions de-shrink. The catch: label_mean/std are used BOTH in training (by me) and inference (by harness). But I control what I save. I can train with the natural label_std, but SAVE a corrected label_std. That's a legitimate linear recalibration baked into the checkpoint. But careful: is the shrinkage consistent enough to exploit? Let me measure slope a from OOF. If a≈0.85, de-shrinking could reduce error on extremes. But it also amplifies noise/variance. Net effect on rel_l2? Let me test empirically: compute OOF preds, then find optimal linear recalibration (a,b) minimizing rel_l2, and see how much it helps. If meaningful, bake it in. Let me also reconsider: maybe training longer or with different capacity settings could reduce shrinkage. But let me first quantify. Let me get OOF predictions for the base PointNet and analyze …[truncated]

Kernel ridge on rich descriptors (OOF rel_L2 = 0.236) and k-NN (≥0.29) are both **worse** than the PointNet (0.224), so ~0.22 is near the data/information ceiling , the architecture isn't the bottleneck. A remaining lever: regression-to-the-mean shrinkage. Since the harness applies `out*label_std + label_mean` (a linear map I control), I can bake in a global recalibration. Let me measure the OOF slope/bias and whether de-shrinking helps.

bash

cd /app && python -c "
import torch, numpy as np, cv2 as cv
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-3,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
m,preds=cv.oof(base,seeds=(0,1,2))
p=preds.mean(0).flatten().numpy(); y=cv.Y.flatten().numpy()
print('OOF base',{k:round(v,4) for k,v in m.items()})
# fit p = a*y+b (shrinkage view)
A=np.polyfit(y,p,1); print('p vs y slope,intercept',np.round(A,4))
# optimal recalibration c*p+d minimizing sum (c p + d - y)^2
M=np.polyfit(p,y,1); c,d=M
pr=c*p+d; 
def met(pp):
    e=pp-y; return dict(rel_l2=float(np.sqrt((e**2).sum()/(y**2).sum())),mae=float(np.abs(e).mean()),maxae=float(np.abs(e).max()))
print('recal c,d',round(c,4),round(d,4),'->',{k:round(v,4) for k,v in met(pr).items()})
# scale-only (through origin) minimizing rel_l2: s=sum(p*y)/sum(p*p)
s=(p*y).sum()/(p*p).sum(); print('scale-only s',round(float(s),4),'->',{k:round(v,4) for k,v in met(s*p).items()})
# residual vs mass: correlation
print('corr(resid, y)',round(float(np.corrcoef(p-y,y)[0,1]),3))
# largest errors
order=np.argsort(-np.abs(p-y))[:8]
for i in order: print('  y=%.3f pred=%.3f err=%.3f'%(y[i],p[i],p[i]-y[i]))
" 2>&1 | grep -v Warning

output

OOF base {'rel_l2': 0.2244, 'mae': 0.1365, 'maxae': 0.6274}
p vs y slope,intercept [0.7912 0.1711]
recal c,d 0.9625 0.0139 -> {'rel_l2': 0.2232, 'mae': 0.1363, 'maxae': 0.6159}
scale-only s 0.9775 -> {'rel_l2': 0.2233, 'mae': 0.1364, 'maxae': 0.605}
corr(resid, y) -0.427
  y=0.521 pred=1.148 err=0.627
  y=1.151 pred=1.731 err=0.580
  y=2.219 pred=1.651 err=-0.568
  y=0.573 pred=1.101 err=0.529
  y=0.596 pred=1.061 err=0.465
  y=0.777 pred=1.217 err=0.440
  y=0.680 pred=1.105 err=0.425
  y=0.515 pred=0.937 err=0.422

The shrinkage (slope 0.79 of p vs y) exists but the residual-vs-y correlation suggests underprediction of high, overprediction of low. However, optimal linear recal only improves rel_l2 from 0.2244 to 0.2232 , marginal. Scale-only to 0.2233. Not much. Notably, the largest errors are NOT all at extremes , e.g., y=0.521 predicted 1.148 (big overprediction of a LIGHT bracket), y=1.151 pred 1.731. These are just hard/confusable samples, not simple shrinkage. So the error is mostly irreducible confusion, not correctable bias. So recalibration won't help much. The worst sample: y=0.521 but predicted 1.148 , a light bracket predicted heavy. And y=2.219 (heaviest) predicted 1.651. These dominate maxae. Given the ceiling analysis, ~0.22 rel_l2 seems to be near the limit. But let me reconsider whether there's something fundamentally better I can do with the POINT CLOUD directly rather than through this bottleneck. Wait. Let me reconsider the problem from scratch. Maybe I'm missing something about scale. The bracket is rescaled so AABB fits in unit cube, y-extent=1. But actually the KEY realization: if brackets all share the same physical bounding box in the y-direction (the mounting span is fixed by the engineering spec), then the scale factor is the SAME for all, and normalized volume ∝ physical volume ∝ mass. In that case, mass is a deterministic function of normalized shape and we should be able to get volume. But here's a thought: can I estimate VOLUME from the surface point cloud more directly, and is volume highly correlated with mass? If density is constant, mass = density × volume exactly. So if I can estimate volume well, I get mass well. The 2048 surface points define the shape. Volume estimation from surface points is hard without normals/mesh, but a voxel-fill or alpha-shape could approximate. But the ceiling analysis included voxel occupancy features and only got 0.236. So even volume-proxy features don't beat 0.224. Hmm. Let me reconsider: maybe the issue is that the 148 samples just don't pin down the function well, and the held-out test will be similar (~0.22). Let me accept ~0.22 as near-ceiling but try a few more things that could genuinely help generalization: 1. Larger batch / full-batch training dynamics, different LR. 2. Different loss: Huber (robust to the few hard samples → lower maxae, maybe lower rel_l2). 3. Longer training with more augmentation (mild). 4. Test combos: mild jitter + yrefl + dropout + wd=1e-3, longer epochs. Also, I should reconsider that the OOF estimate uses 123 training samples/fold, but the FINAL model trains on all 148 and is graded on a separate held-out set. The held-out set size is unknown. My final model should be trained on all 148. Let me also reconsider the evaluation: the grading "relative L2 over the full set" , my OOF rel_l2 over 148 is a good proxy. I'm at 0.224. Let me try to squeeze improvements via: - loss=huber with various beta - augmentation combo tuned - epochs longer (300-400) - maybe two-layer: does averaging predictions over multiple seeds help OOF? Yes (I already average 3 seeds in OOF). But final model is ONE model , I can't average seeds at inference! Wait, this is important. In my OOF I average predictions over 3 seeds, which reduces variance and improves rel_l2. But the final single checkpoint can't do that. Let me check: how much does seed-averaging help vs a single seed? If single-seed OOF is ~0.24 and 3-seed avg is 0.224, then my final single model will be ~0.24, not 0.224. Let me measure single-seed OOF (no averaging) to know the realistic single-model performance. This matters a lot. Let me measure per-seed OOF rel_l2 individually vs averaged.

The shrinkage is real but residuals are mostly hard-sample confusion, not correctable bias , optimal linear recal only improves rel_L2 from 0.2244 to 0.2232. One critical caveat: my OOF averages 3 seeds, but the final checkpoint is a **single** model. Let me check how much of that 0.224 depends on seed-averaging , this tells me the realistic single-model number I'll actually ship.

bash

cd /app && python -c "
import torch, numpy as np, cv2 as cv, lib_train as L
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-3,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
m,preds=cv.oof(base,seeds=(0,1,2,3,4))
# per seed
for s in range(preds.shape[0]):
    mm=L.metrics(preds[s],cv.Y); print('seed',s,{k:round(v,4) for k,v in mm.items()})
for ns in [1,2,3,5]:
    mm=L.metrics(preds[:ns].mean(0),cv.Y); print('avg first',ns,{k:round(v,4) for k,v in mm.items()})
" 2>&1 | grep -v Warning

output

seed 0 {'rel_l2': 0.2289, 'mae': 0.1401, 'maxae': 0.6005}
seed 1 {'rel_l2': 0.2409, 'mae': 0.1459, 'maxae': 0.7336}
seed 2 {'rel_l2': 0.2259, 'mae': 0.1374, 'maxae': 0.5748}
seed 3 {'rel_l2': 0.2404, 'mae': 0.1452, 'maxae': 0.6987}
seed 4 {'rel_l2': 0.228, 'mae': 0.1402, 'maxae': 0.8304}
avg first 1 {'rel_l2': 0.2289, 'mae': 0.1401, 'maxae': 0.6005}
avg first 2 {'rel_l2': 0.2292, 'mae': 0.139, 'maxae': 0.667}
avg first 3 {'rel_l2': 0.2244, 'mae': 0.1365, 'maxae': 0.6274}
avg first 5 {'rel_l2': 0.2227, 'mae': 0.1347, 'maxae': 0.6823}

Single-seed OOF rel_l2 ranges 0.226–0.241 (mean ~0.233). Seed-averaging helps a bit (5 seeds → 0.2227) but not dramatically. Since I ship one model, realistic single-model rel_l2 ≈ 0.226–0.241. Now, the key question: can I ship something better than a single model? I can only provide ONE state_dict for PointNetRegressor. BUT , I could train a single model that itself behaves like an ensemble via... no. Wait, there's a legitimate trick: I could train MANY models and do "output-space distillation" into one model, OR use weight averaging across seeds , but weight averaging across independent seeds fails due to permutation symmetry. Alternatively, can I exploit the fixed architecture cleverly? The head is an MLP. No. Let me reconsider: the realistic single model is ~0.23. Let me try to get the best single model by: 1. Picking the best training recipe (loss, augmentation, epochs) to maximize single-seed OOF AND reduce variance across seeds. 2. Then at final, I train on all 148 and pick... but I can't pick by OOF for the final (no held-out). I just use the recipe. Actually, I CAN do better: since val + public_test have labels, after training the final model on, say, train only (100) I could select among seeds using val+test. But the best is to train on all data. Trade-off: train on all 148 → no selection signal, but more data. Or train on 100, select best seed via val+test (48 samples) → selection but less data. Given the held-out grading set, more data + robust recipe is safer. But seed selection could help pick a good single model. Let me think: if I train ~10 seeds on all 148, I can't evaluate them (no held-out). If I train on train+val (132) and evaluate on public_test (16) to pick seed, that's weak (16 samples, noisy) and wastes test data from training. Better idea: Train final model on ALL 148 with a recipe that gives low-variance, low-OOF-error. To reduce seed variance in the single shipped model, I can use a longer/more-regularized training which tends to be more stable. Actually, here's a strong legitimate approach to get ensemble-like benefit in ONE model: Knowledge distillation. Train K teacher models (different seeds) on all 148. Generate averaged teacher predictions on the training inputs (and optionally on augmented versions). Then train a SINGLE student PointNet to match the averaged teacher predictions (soft targets) plus true labels. The student, trained to regress the ensemble's smooth function, can approach ensemble performance with one model. Since I can generate unlimited augmented inputs and get teacher predictions on them, the student gets a richer, smoother target function → better generalization. This is a real, well-established technique and fully compatible (single checkpoint). But does distillation actually help here given the ensemble only improves 0.233→0.223? The gain is small (~0.01). Distillation would at best approach 0.223. Marginal but could matter. Given time, maybe worth it, but let me first lock in the best base recipe. Actually, let me reconsider the whole thing. Let me reconsider whether I can reduce the error more fundamentally by improving the training objective to target rel_l2/maxae. Let me try: - Huber loss (robust) , might reduce the influence of hard samples during training and generalize better. - Log-space target: predict log(mass). Mass is positive, ranges 0.2–2.2 (ratio ~11x). Relative L2 is about relative errors; training in log space optimizes relative error more directly. Let me test log-target: label' = log(y). Normalize log. This could help rel_l2 and especially the light brackets (where absolute errors like 0.52→1.15 are huge relatively). Let me test log target. Actually log target changes the loss landscape to penalize relative errors. Since rel_l2 and mae are in linear units, log-target might or might not help. Let me just test it in OOF. But note: label_mean/label_std map is linear; if I train in log space, the inference mapping out*label_std+label_mean gives …[truncated]

Single-seed OOF is ~0.226–0.241 (mean ≈ 0.233); seed-averaging only reaches 0.223 but I can't ensemble in one checkpoint. Also note the label transform must stay affine (the harness only does `out*std+mean`), so log-target tricks are out. Let me sweep the loss function, augmentation combos, and schedule, ranking by **mean single-seed OOF** (the realistic shipped number) plus the ensemble for reference.

bash

cd /app && cat > /app/run3.py << 'PYEOF'
import cv2 as cv, lib_train as L, time, numpy as np, sys
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-3,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0)
SEEDS=(0,1,2,3)
def ev(name,over):
    cfg=dict(base); cfg.update(over)
    t=time.time(); _,preds=cv.oof(cfg,seeds=SEEDS)
    per=[L.metrics(preds[s],cv.Y) for s in range(len(SEEDS))]
    msingle=np.mean([p['rel_l2'] for p in per])
    ens=L.metrics(preds.mean(0),cv.Y)
    mae_s=np.mean([p['mae'] for p in per]); max_s=np.mean([p['maxae'] for p in per])
    print('%-24s single rel_l2=%.4f mae=%.4f maxae=%.4f | ens rel_l2=%.4f (%.0fs)'%(
        name,msingle,mae_s,max_s,ens['rel_l2'],time.time()-t),flush=True)
cfgs=[
 ('mse',{}),
 ('huber0.5',{'loss':'huber','huber_beta':0.5}),
 ('huber1.0',{'loss':'huber','huber_beta':1.0}),
 ('l1',{'loss':'l1'}),
 ('jit0.005',{'jitter':0.005}),
 ('yrefl',{'yreflect':True}),
 ('drop0.1',{'dropout':0.1}),
 ('combo',{'jitter':0.005,'yreflect':True,'dropout':0.1}),
 ('ep350',{'epochs':350}),
 ('bs32lr15',{'batch_size':32,'lr':1.5e-3}),
]
which=sys.argv[1] if len(sys.argv)>1 else 'all'
for name,over in cfgs:
    if which=='all' or which==name: ev(name,over)
PYEOF
nohup python run3.py > /app/run3.log 2>&1 &
echo "pid $!"

output

pid 378

bash

sleep 295 && cat /app/run3.log | grep -v Warning

output

mse                      single rel_l2=0.2340 mae=0.1422 maxae=0.6519 | ens rel_l2=0.2256 (139s)

This is slow: 139s per config × 10 = ~23 min. Let me wait. Meanwhile, let me think about the distillation idea more concretely as a potential bigger win, and also reconsider. Actually, let me reconsider the single vs ensemble gap. Single 0.234, ensemble(4) 0.2256. The ensemble advantage is ~0.008. Distillation could capture most of that → ~0.225 single. Worth doing at the end. But there might be a bigger win I'm missing. Let me reconsider the data. Let me think about whether there are near-duplicate brackets across folds causing optimistic or pessimistic OOF. Also, let me reconsider the held-out grading: it's a separate set; my OOF on pooled 148 is the best proxy. Let me wait for the sweep.

The sweep runs ~139s/config. Let me wait and collect results.

bash

sleep 295 && cat /app/run3.log | grep -v Warning

output

mse                      single rel_l2=0.2340 mae=0.1422 maxae=0.6519 | ens rel_l2=0.2256 (139s)
huber0.5                 single rel_l2=0.2383 mae=0.1462 maxae=0.7179 | ens rel_l2=0.2290 (157s)
huber1.0                 single rel_l2=0.2363 mae=0.1433 maxae=0.7061 | ens rel_l2=0.2265 (174s)

Huber is slightly worse than MSE. MSE remains best so far. Let me wait for the rest (l1, augmentations, epochs, batch).

bash

sleep 295 && cat /app/run3.log | grep -v Warning

output

mse                      single rel_l2=0.2340 mae=0.1422 maxae=0.6519 | ens rel_l2=0.2256 (139s)
huber0.5                 single rel_l2=0.2383 mae=0.1462 maxae=0.7179 | ens rel_l2=0.2290 (157s)
huber1.0                 single rel_l2=0.2363 mae=0.1433 maxae=0.7061 | ens rel_l2=0.2265 (174s)
l1                       single rel_l2=0.2479 mae=0.1518 maxae=0.7587 | ens rel_l2=0.2389 (179s)
jit0.005                 single rel_l2=0.2382 mae=0.1473 maxae=0.7012 | ens rel_l2=0.2298 (155s)

bash

sleep 295 && cat /app/run3.log | grep -v Warning

output

mse                      single rel_l2=0.2340 mae=0.1422 maxae=0.6519 | ens rel_l2=0.2256 (139s)
huber0.5                 single rel_l2=0.2383 mae=0.1462 maxae=0.7179 | ens rel_l2=0.2290 (157s)
huber1.0                 single rel_l2=0.2363 mae=0.1433 maxae=0.7061 | ens rel_l2=0.2265 (174s)
l1                       single rel_l2=0.2479 mae=0.1518 maxae=0.7587 | ens rel_l2=0.2389 (179s)
jit0.005                 single rel_l2=0.2382 mae=0.1473 maxae=0.7012 | ens rel_l2=0.2298 (155s)
yrefl                    single rel_l2=0.2221 mae=0.1356 maxae=0.7207 | ens rel_l2=0.2107 (154s)
drop0.1                  single rel_l2=0.2294 mae=0.1378 maxae=0.6914 | ens rel_l2=0.2214 (197s)

Now we're talking. yrefl (y-reflection augmentation) helps notably: single rel_l2 0.2221 (vs mse 0.2340), ens 0.2107 (vs 0.2256). That's a real improvement! mae also down to 0.1356. maxae slightly up (0.72). drop0.1 also helps single (0.2294) and ens (0.2214). So y-reflection is a strong augmentation as I hypothesized (brackets are ~symmetric across y, and mirror is mass-preserving and in-distribution). Let me see the combo result (jitter+yrefl+drop) which is still pending, plus ep350 and bs32. Let me wait.

y-reflection is a clear win: single rel_L2 0.2221 (vs 0.234 for plain MSE) and ensemble 0.2107, with lower MAE too , confirming the mirror symmetry hypothesis. `drop0.1` also helps. Let me see the combo and schedule variants.

bash

sleep 295 && cat /app/run3.log | grep -v Warning

output

mse                      single rel_l2=0.2340 mae=0.1422 maxae=0.6519 | ens rel_l2=0.2256 (139s)
huber0.5                 single rel_l2=0.2383 mae=0.1462 maxae=0.7179 | ens rel_l2=0.2290 (157s)
huber1.0                 single rel_l2=0.2363 mae=0.1433 maxae=0.7061 | ens rel_l2=0.2265 (174s)
l1                       single rel_l2=0.2479 mae=0.1518 maxae=0.7587 | ens rel_l2=0.2389 (179s)
jit0.005                 single rel_l2=0.2382 mae=0.1473 maxae=0.7012 | ens rel_l2=0.2298 (155s)
yrefl                    single rel_l2=0.2221 mae=0.1356 maxae=0.7207 | ens rel_l2=0.2107 (154s)
drop0.1                  single rel_l2=0.2294 mae=0.1378 maxae=0.6914 | ens rel_l2=0.2214 (197s)
combo                    single rel_l2=0.2256 mae=0.1430 maxae=0.6119 | ens rel_l2=0.2191 (193s)

combo (jitter0.005+yrefl+drop0.1): single 0.2256, ens 0.2191, but notably maxae is best so far: single maxae 0.6119 (vs yrefl 0.72). So combo trades a tiny bit of rel_l2 for much better worst-case. Interesting. yrefl alone: best single rel_l2 (0.2221) and ens (0.2107), but maxae 0.72 (worst-case higher). combo: rel_l2 slightly worse (0.2256) but maxae much better (0.61). Since we care about all three metrics (rel_l2 primary, but mae and maxae too), combo might be a better all-rounder. But jitter0.005 alone hurt (0.2382), so the jitter in combo isn't helping rel_l2. Let me try yrefl+drop0.1 (no jitter) , might get best of both. Let me wait for ep350 and bs32 first, then do a refined sweep around yrefl.

`combo` (jitter+yrefl+drop) has the best worst-case (maxae 0.61) but slightly higher rel_L2. Since jitter alone hurt, `yrefl+drop0.1` (no jitter) may combine low rel_L2 with good maxae. Let me finish the current runs, then refine around y-reflection.

bash

sleep 240 && cat /app/run3.log | grep -v Warning | tail -3

output

combo                    single rel_l2=0.2256 mae=0.1430 maxae=0.6119 | ens rel_l2=0.2191 (193s)
ep350                    single rel_l2=0.2263 mae=0.1365 maxae=0.7100 | ens rel_l2=0.2172 (236s)
bs32lr15                 single rel_l2=0.2450 mae=0.1509 maxae=0.6886 | ens rel_l2=0.2343 (82s)

ep350 (more epochs, no aug): single 0.2263, ens 0.2172 , better than 200ep mse (0.2340/0.2256). So longer training helps. bs32 hurts (0.2450). So the winners: yrefl augmentation + more epochs. Let me now refine: combine yrefl with longer epochs and dropout, and tune reflection/dropout. Let me run a refined sweep: - yrefl + ep350 - yrefl + drop0.1 - yrefl + drop0.1 + ep350 - yrefl + drop0.2 - yrefl + drop0.1 + ep350 + wd variations - yrefl + rotz small + ep350 - yrefl + drop0.1 + jitter smaller (0.003) Also test a mild x-reflection? earlier x-mirror looked out-of-distribution (chamfer 0.083 > baseline). Probably hurts. Skip. Let me also reconsider: maybe combine yrefl with a slightly larger reflection probability, or combine with "both" reflect (randomly reflect y always vs 50%). Current yreflect reflects each sample with prob 0.5. That's standard. Let me run refined sweep with 4 seeds, focusing on top candidates. I'll also push epochs to 400-500 to see if it keeps improving. Let me prioritize rel_l2 (primary) but keep an eye on maxae. Let me run: 1. yrefl_ep350 2. yrefl_ep500 3. yrefl_drop0.1_ep350 4. yrefl_drop0.1_ep500 5. yrefl_drop0.15_ep350 6. yrefl_drop0.1_jit003_ep350 7. yrefl_ep350_wd3e-3 8. yrefl_drop0.1_ep350_wd3e-4 I'll run in background.

Longer training (ep350) and y-reflection both help independently. Let me refine: combine y-reflection with dropout and longer schedules, and tune weight decay around these winners.

bash

cd /app && cat > /app/run4.py << 'PYEOF'
import cv2 as cv, lib_train as L, time, numpy as np, sys
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-3,
          epochs=200,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0,yreflect=True)
SEEDS=(0,1,2,3)
def ev(name,over):
    cfg=dict(base); cfg.update(over)
    t=time.time(); _,preds=cv.oof(cfg,seeds=SEEDS)
    per=[L.metrics(preds[s],cv.Y) for s in range(len(SEEDS))]
    msingle=np.mean([p['rel_l2'] for p in per]); mae_s=np.mean([p['mae'] for p in per]); max_s=np.mean([p['maxae'] for p in per])
    ens=L.metrics(preds.mean(0),cv.Y)
    print('%-26s single rel_l2=%.4f mae=%.4f maxae=%.4f | ens=%.4f (%.0fs)'%(name,msingle,mae_s,max_s,ens['rel_l2'],time.time()-t),flush=True)
cfgs=[
 ('yrefl_ep350',{'epochs':350}),
 ('yrefl_ep500',{'epochs':500}),
 ('yrefl_drop0.1_ep350',{'epochs':350,'dropout':0.1}),
 ('yrefl_drop0.1_ep500',{'epochs':500,'dropout':0.1}),
 ('yrefl_drop0.15_ep350',{'epochs':350,'dropout':0.15}),
 ('yrefl_drop0.1_jit003_ep350',{'epochs':350,'dropout':0.1,'jitter':0.003}),
 ('yrefl_ep350_wd3e-3',{'epochs':350,'weight_decay':3e-3}),
 ('yrefl_drop0.1_ep350_wd3e-4',{'epochs':350,'dropout':0.1,'weight_decay':3e-4}),
]
for name,over in cfgs: ev(name,over)
PYEOF
nohup python run4.py > /app/run4.log 2>&1 &
echo "pid $!"

output

pid 448

bash

sleep 297 && cat /app/run4.log | grep -v Warning

output

yrefl_ep350                single rel_l2=0.2155 mae=0.1283 maxae=0.7209 | ens=0.2049 (255s)

yrefl_ep350: single 0.2155, ens 0.2049. Better! mae 0.1283. maxae 0.72 (still high). These take ~255s each (8 configs → ~30 min). Let me wait but check periodically. Meanwhile let me think about reducing maxae. The worst-case (maxae ~0.72) comes from a few hard samples. Let me think , maxae over 148 OOF. One or two samples are badly predicted regardless. Augmentations that increase diversity (dropout, combo) reduced maxae earlier. Let me keep an eye. Let me wait for more results.

`yrefl_ep350`: single rel_L2 0.2155, ensemble 0.2049, MAE 0.128 , clear progress. The worst-case (maxae ≈ 0.72) is still high. Let me continue collecting the refinement sweep.

bash

sleep 297 && cat /app/run4.log | grep -v Warning

output

yrefl_ep350                single rel_l2=0.2155 mae=0.1283 maxae=0.7209 | ens=0.2049 (255s)

bash

sleep 250 && cat /app/run4.log | grep -v Warning

output

yrefl_ep350                single rel_l2=0.2155 mae=0.1283 maxae=0.7209 | ens=0.2049 (255s)
yrefl_ep500                single rel_l2=0.2104 mae=0.1254 maxae=0.7154 | ens=0.2030 (365s)

yrefl_ep500: single 0.2104, ens 0.2030. Even better with more epochs. mae 0.1254. Longer training + yrefl keeps improving. Diminishing returns but still going. Let me keep waiting for the dropout combos. The trend: more epochs help (with yrefl providing regularization so no overfit). Let me consider even longer (800-1000) later. Let me wait for remaining configs.

More epochs keep helping with y-reflection regularizing: `yrefl_ep500` gives single 0.2104, ensemble 0.2030. Let me wait for the dropout/jitter/wd combinations.

bash

sleep 297 && cat /app/run4.log | grep -v Warning

output

yrefl_ep350                single rel_l2=0.2155 mae=0.1283 maxae=0.7209 | ens=0.2049 (255s)
yrefl_ep500                single rel_l2=0.2104 mae=0.1254 maxae=0.7154 | ens=0.2030 (365s)
yrefl_drop0.1_ep350        single rel_l2=0.2133 mae=0.1287 maxae=0.6784 | ens=0.2039 (333s)

bash

sleep 297 && cat /app/run4.log | grep -v Warning

output

yrefl_ep350                single rel_l2=0.2155 mae=0.1283 maxae=0.7209 | ens=0.2049 (255s)
yrefl_ep500                single rel_l2=0.2104 mae=0.1254 maxae=0.7154 | ens=0.2030 (365s)
yrefl_drop0.1_ep350        single rel_l2=0.2133 mae=0.1287 maxae=0.6784 | ens=0.2039 (333s)
yrefl_drop0.1_ep500        single rel_l2=0.2128 mae=0.1267 maxae=0.7145 | ens=0.2049 (482s)

yrefl_drop0.1_ep500: single 0.2128, ens 0.2049 , similar to yrefl_ep500 but not better on rel_l2. drop0.1_ep350 gave maxae 0.678 (better worst-case). So the leader on rel_l2 is yrefl_ep500 (single 0.2104). drop helps maxae marginally. Let me wait for the remaining configs (drop0.15, jit003, wd variants). Let me also plan next: test even longer epochs (700, 1000) with yrefl, and maybe yrefl+drop0.05. Let me wait for current batch to finish.

bash

sleep 297 && cat /app/run4.log | grep -v Warning | tail -4

output

yrefl_ep500                single rel_l2=0.2104 mae=0.1254 maxae=0.7154 | ens=0.2030 (365s)
yrefl_drop0.1_ep350        single rel_l2=0.2133 mae=0.1287 maxae=0.6784 | ens=0.2039 (333s)
yrefl_drop0.1_ep500        single rel_l2=0.2128 mae=0.1267 maxae=0.7145 | ens=0.2049 (482s)
yrefl_drop0.15_ep350       single rel_l2=0.2142 mae=0.1278 maxae=0.7140 | ens=0.2057 (293s)

bash

sleep 297 && cat /app/run4.log | grep -v Warning | tail -3

output

yrefl_drop0.1_ep350        single rel_l2=0.2133 mae=0.1287 maxae=0.6784 | ens=0.2039 (333s)
yrefl_drop0.1_ep500        single rel_l2=0.2128 mae=0.1267 maxae=0.7145 | ens=0.2049 (482s)
yrefl_drop0.15_ep350       single rel_l2=0.2142 mae=0.1278 maxae=0.7140 | ens=0.2057 (293s)

bash

sleep 280 && cat /app/run4.log | grep -v Warning | tail -3

output

yrefl_drop0.15_ep350       single rel_l2=0.2142 mae=0.1278 maxae=0.7140 | ens=0.2057 (293s)
yrefl_drop0.1_jit003_ep350 single rel_l2=0.2142 mae=0.1297 maxae=0.7278 | ens=0.2072 (319s)
yrefl_ep350_wd3e-3         single rel_l2=0.2044 mae=0.1230 maxae=0.6907 | ens=0.1963 (253s)

yrefl_ep350_wd3e-3: single 0.2044, ens 0.1963! Higher weight decay (3e-3) with yrefl+ep350 gives the best yet: single 0.2044, mae 0.1230, maxae 0.6907, ens 0.1963. So stronger weight decay helps when combined with more epochs + yrefl. This makes sense: more epochs + augmentation need more regularization; wd=3e-3 is the sweet spot. Let me explore wd further (3e-3, 5e-3, 1e-2) combined with yrefl and longer epochs (500, 700). Let me wait for the last config (drop0.1_ep350_wd3e-4), then launch a new sweep around wd and epochs with yrefl.

`yrefl_ep350_wd3e-3` is the new best: single rel_L2 0.2044, ensemble 0.1963, MAE 0.123. Stronger weight decay pairs well with longer training + augmentation. Let me explore higher weight decay and longer epochs around this winner.

bash

sleep 120 && cat /app/run4.log | grep -v Warning | tail -1
cd /app && cat > /app/run5.py << 'PYEOF'
import cv2 as cv, lib_train as L, time, numpy as np
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=3e-3,
          epochs=350,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0,yreflect=True)
SEEDS=(0,1,2,3)
def ev(name,over):
    cfg=dict(base); cfg.update(over)
    t=time.time(); _,preds=cv.oof(cfg,seeds=SEEDS)
    per=[L.metrics(preds[s],cv.Y) for s in range(len(SEEDS))]
    msingle=np.mean([p['rel_l2'] for p in per]); mae_s=np.mean([p['mae'] for p in per]); max_s=np.mean([p['maxae'] for p in per])
    ens=L.metrics(preds.mean(0),cv.Y)
    print('%-26s single rel_l2=%.4f mae=%.4f maxae=%.4f | ens=%.4f (%.0fs)'%(name,msingle,mae_s,max_s,ens['rel_l2'],time.time()-t),flush=True)
cfgs=[
 ('wd5e-3_ep350',{'weight_decay':5e-3}),
 ('wd1e-2_ep350',{'weight_decay':1e-2}),
 ('wd3e-3_ep500',{'weight_decay':3e-3,'epochs':500}),
 ('wd5e-3_ep500',{'weight_decay':5e-3,'epochs':500}),
 ('wd3e-3_ep700',{'weight_decay':3e-3,'epochs':700}),
 ('wd5e-3_ep700',{'weight_decay':5e-3,'epochs':700}),
 ('wd5e-3_drop0.1_ep500',{'weight_decay':5e-3,'epochs':500,'dropout':0.1}),
 ('wd1e-2_ep700',{'weight_decay':1e-2,'epochs':700}),
]
for name,over in cfgs: ev(name,over)
PYEOF
nohup python run5.py > /app/run5.log 2>&1 &
echo "launched pid $!"

output

yrefl_ep350_wd3e-3         single rel_l2=0.2044 mae=0.1230 maxae=0.6907 | ens=0.1963 (253s)
launched pid 542

Interesting - the last config from run4 (drop0.1_ep350_wd3e-4) didn't print, maybe it's still the tail. Anyway, I've launched run5. The last config of run4 might still be running concurrently with run5 now, that could slow both down or cause GPU contention. Let me check GPU. Actually run4 finished its 8 configs? Let me check. The tail shows wd3e-3 as last printed; there was one more (drop0.1_ep350_wd3e-4). Let me check if run4 is still running. Actually it printed 7 lines; the 8th (yrefl_drop0.1_ep350_wd3e-4) may still be going. Running run5 simultaneously could cause contention. Let me check processes.

Let me check for process/GPU contention since run4's last config may still be running alongside run5.

bash

cd /app && nvidia-smi --query-gpu=memory.used,utilization.gpu --format=csv,noheader; echo "---procs---"; ps aux | grep -E "run[45].py" | grep -v grep | awk '{print $2, $11, $12}'; echo "---run4 tail---"; tail -2 run4.log | grep -v Warning

output

32280 MiB, 100 %
---procs---
448 python run4.py
542 python run5.py
---run4 tail---
yrefl_drop0.1_jit003_ep350 single rel_l2=0.2142 mae=0.1297 maxae=0.7278 | ens=0.2072 (319s)
yrefl_ep350_wd3e-3         single rel_l2=0.2044 mae=0.1230 maxae=0.6907 | ens=0.1963 (253s)

Both run4 (pid 448) and run5 (pid 542) are running. run4 is on its last config. GPU memory 32GB used (high!), util 100%. Two processes sharing GPU. Memory 32GB is a lot , might be because two processes each loaded data. Let me let run4 finish (one more config) and run5 continue. The contention will slow things but not break. Actually 32GB used is concerning for OOM. Let me check it doesn't crash. The models are tiny though; 32GB seems high , maybe leftover. Let me just monitor. Let me wait for run4 to finish then run5 has full GPU. Actually, let me wait and collect run5 results over the next several minutes. Let me wait.

Both are sharing the GPU (the models are tiny, so it'll just be a bit slower). Let me wait for run5 results.

bash

sleep 297 && cat /app/run5.log | grep -v Warning

output

wd5e-3_ep350               single rel_l2=0.2120 mae=0.1273 maxae=0.7120 | ens=0.2001 (276s)

wd5e-3_ep350: single 0.2120 , slightly worse than wd3e-3_ep350 (0.2044). So wd3e-3 better than 5e-3 at ep350. Let me wait for longer-epoch variants. Note there's seed noise here: wd3e-3_ep350 gave 0.2044, wd5e-3_ep350 gave 0.2120. The difference could be partly noise. These OOF estimates with 4 seeds still have noise. I should be careful not to over-fit to OOF noise. Let me gather more and look at the overall picture, preferring configs that are robustly good. Let me wait for the rest of run5.

bash

sleep 297 && cat /app/run5.log | grep -v Warning

output

wd5e-3_ep350               single rel_l2=0.2120 mae=0.1273 maxae=0.7120 | ens=0.2001 (276s)
wd1e-2_ep350               single rel_l2=0.2136 mae=0.1298 maxae=0.7016 | ens=0.2046 (247s)

bash

sleep 297 && cat /app/run5.log | grep -v Warning

output

wd5e-3_ep350               single rel_l2=0.2120 mae=0.1273 maxae=0.7120 | ens=0.2001 (276s)
wd1e-2_ep350               single rel_l2=0.2136 mae=0.1298 maxae=0.7016 | ens=0.2046 (247s)
wd3e-3_ep500               single rel_l2=0.2137 mae=0.1268 maxae=0.7020 | ens=0.2063 (354s)

Hmm, wd3e-3_ep500: single 0.2137 , worse than wd3e-3_ep350 (0.2044). So ep500 is not better than ep350 at wd3e-3. This contradicts the earlier trend (yrefl ep500 better than ep350 at wd1e-3). There's noise. It seems ep350 + wd3e-3 hit a sweet spot (0.2044) but it may be partly lucky seeds. I'm seeing OOF noise of ~±0.005-0.01. I need to be careful. Let me gather the remaining run5 results, then do a more careful comparison of the top 2-3 configs with MORE seeds to average out noise and pick robustly. Let me wait for the rest.

Results are bunching around 0.21 with OOF noise of ~±0.01, so I shouldn't over-trust a single config's number. Let me collect the rest, then re-evaluate the top candidates with more seeds to pick robustly.

bash

sleep 297 && cat /app/run5.log | grep -v Warning

output

wd5e-3_ep350               single rel_l2=0.2120 mae=0.1273 maxae=0.7120 | ens=0.2001 (276s)
wd1e-2_ep350               single rel_l2=0.2136 mae=0.1298 maxae=0.7016 | ens=0.2046 (247s)
wd3e-3_ep500               single rel_l2=0.2137 mae=0.1268 maxae=0.7020 | ens=0.2063 (354s)

bash

sleep 200 && cat /app/run5.log | grep -v Warning | tail -3

output

wd1e-2_ep350               single rel_l2=0.2136 mae=0.1298 maxae=0.7016 | ens=0.2046 (247s)
wd3e-3_ep500               single rel_l2=0.2137 mae=0.1268 maxae=0.7020 | ens=0.2063 (354s)
wd5e-3_ep500               single rel_l2=0.2108 mae=0.1255 maxae=0.7010 | ens=0.2029 (348s)

bash

sleep 297 && cat /app/run5.log | grep -v Warning | tail -3

output

wd3e-3_ep500               single rel_l2=0.2137 mae=0.1268 maxae=0.7020 | ens=0.2063 (354s)
wd5e-3_ep500               single rel_l2=0.2108 mae=0.1255 maxae=0.7010 | ens=0.2029 (348s)
wd3e-3_ep700               single rel_l2=0.2093 mae=0.1230 maxae=0.7272 | ens=0.2028 (450s)

wd3e-3_ep700: single 0.2093, ens 0.2028. Comparable to ep500. Diminishing returns, noise dominates. Let me get the last two and then settle. The picture: yrefl + wd in [3e-3, 5e-3] + epochs in [350, 700] all give single OOF rel_l2 ≈ 0.205-0.214, ens ≈ 0.196-0.206. The differences are within noise. Let me settle on a robust recipe and verify with many seeds. Candidate: yrefl + wd3e-3 + ep500 (good balance). But I should reduce maxae too. Let me consider adding drop0.1 which helped maxae. Let me wait for the final two configs, then run a careful head-to-head of ~3 finalist configs with 8 seeds each for a robust decision.

y-reflection + wd∈[3e-3,5e-3] + epochs∈[350,700] all cluster at single OOF rel_L2 ≈ 0.205–0.214 (differences within noise). Let me get the last two results, then run a careful head-to-head of the finalists with more seeds to decide robustly.

bash

sleep 297 && cat /app/run5.log | grep -v Warning | tail -2

output

wd5e-3_ep500               single rel_l2=0.2108 mae=0.1255 maxae=0.7010 | ens=0.2029 (348s)
wd3e-3_ep700               single rel_l2=0.2093 mae=0.1230 maxae=0.7272 | ens=0.2028 (450s)

bash

sleep 297 && cat /app/run5.log | grep -v Warning | tail -2

output

wd3e-3_ep700               single rel_l2=0.2093 mae=0.1230 maxae=0.7272 | ens=0.2028 (450s)
wd5e-3_ep700               single rel_l2=0.2130 mae=0.1264 maxae=0.6889 | ens=0.2058 (448s)

bash

sleep 200 && cat /app/run5.log | grep -v Warning | tail -2

output

wd5e-3_ep700               single rel_l2=0.2130 mae=0.1264 maxae=0.6889 | ens=0.2058 (448s)
wd5e-3_drop0.1_ep500       single rel_l2=0.2184 mae=0.1280 maxae=0.7270 | ens=0.2108 (365s)

wd5e-3_drop0.1_ep500: single 0.2184 , dropout not helping here. Let me wait for the last (wd1e-2_ep700) then finalize. The finalists are all clustered ~0.209-0.213 for single. The best single numbers: wd3e-3_ep350 (0.2044), wd3e-3_ep700 (0.2093), wd5e-3_ep500 (0.2108), yrefl_ep500 (0.2104). I suspect wd3e-3_ep350's 0.2044 was a bit lucky. Let me do a robust head-to-head with 8 seeds for the top ~3 configs to decide. I'll use the SAME folds (folds_seed=123) but 8 seeds. Actually, to reduce fold noise too, I could also average over a couple of fold partitions. But that doubles cost. Let me do 8 seeds on the standard folds. Candidates for head-to-head: A: yrefl, wd3e-3, ep350 B: yrefl, wd3e-3, ep500 C: yrefl, wd3e-3, ep700 D: yrefl, wd5e-3, ep500 Let me also include a drop0.1 variant for maxae consideration: E: yrefl, wd3e-3, ep500, drop0.1 Let me wait for the last run5 result and then launch the head-to-head with 8 seeds. Given ~350-450s each at 4 seeds, 8 seeds ≈ 700-900s each; 5 configs ≈ 70 min. That's long. Let me reduce: use 6 seeds and 4 configs. Still ~50 min. Alternatively, I can be smarter: evaluate on 2 different fold partitions with 4 seeds each = 8 train runs per fold-set... similar cost. Let me just pick a robust config based on accumulated evidence and do a final confirmation. The evidence strongly supports: yreflect=True, wd≈3e-3, epochs≈500, lr 1e-3, bs16, mse. This gives single ~0.21, ens ~0.20. Let me do ONE careful head-to-head with 8 seeds comparing: A: wd3e-3 ep500 (main candidate) B: wd3e-3 ep500 drop0.1 (for maxae) to decide whether to include dropout. And compare ep350 vs ep500 vs ep700 at wd3e-3. Given time budget, let me run a background job with 6 seeds for: {ep350, ep500, ep700} × wd3e-3, plus ep500+drop0.1. 4 configs × ~500s = ~33 min. Let me launch after run5 finishes. Let me wait for run5 to finish.

Let me wait for the final run5 result, then run a robust head-to-head of the finalists with more seeds.

bash

sleep 220 && cat /app/run5.log | grep -v Warning | tail -1; ps aux | grep -E "run5.py" | grep -v grep | wc -l

output

wd5e-3_drop0.1_ep500       single rel_l2=0.2184 mae=0.1280 maxae=0.7270 | ens=0.2108 (365s)
1

bash

sleep 180 && cat /app/run5.log | grep -v Warning | tail -1; echo "still running:"; ps aux | grep run5.py | grep -v grep | wc -l

output

wd5e-3_drop0.1_ep500       single rel_l2=0.2184 mae=0.1280 maxae=0.7270 | ens=0.2108 (365s)
still running:
1

wd1e-2_ep700 is still running (last config). Let me wait a bit more. Meanwhile, let me launch the head-to-head in the background now , but GPU contention. Let me wait for run5 to fully finish first to avoid contention skewing timing. Let me check again shortly.

bash

sleep 120 && cat /app/run5.log | grep -v Warning | tail -1; echo "still running:"; ps aux | grep run5.py | grep -v grep | wc -l

output

wd1e-2_ep700               single rel_l2=0.2080 mae=0.1241 maxae=0.6840 | ens=0.2003 (451s)
still running:
0

wd1e-2_ep700: single 0.2080, ens 0.2003, maxae 0.684. So wd1e-2 with ep700 is also competitive and has lower maxae. Summary of the strongest configs (all with yreflect, lr1e-3, bs16, mse): - wd3e-3 ep350: single 0.2044, mae 0.1230, maxae 0.691, ens 0.1963 - wd1e-2 ep700: single 0.2080, mae 0.1241, maxae 0.684, ens 0.2003 - wd3e-3 ep700: single 0.2093, mae 0.1230, maxae 0.727, ens 0.2028 - wd5e-3 ep500: single 0.2108, mae 0.1255, maxae 0.701, ens 0.2029 These are all close. The noise across seeds is ~±0.01. Now let me do a robust head-to-head with 6 seeds to pick the final recipe. I'll compare a few promising ones. I also want to reduce maxae. wd1e-2_ep700 has good maxae. Let me run head-to-head with 6 seeds: A: wd3e-3 ep500 B: wd1e-2 ep700 C: wd5e-3 ep500 D: wd3e-3 ep350 Actually, let me reduce to fewer, more distinct configs to save time, and use more seeds for robustness. Let me do 6 seeds for: - wd3e-3 ep500 - wd1e-2 ep500 - wd5e-3 ep500 all at ep500 to isolate wd, plus confirm ep. Actually I keep spending time. Let me be decisive. Decision approach: I'll run a final robust comparison with 8 seeds across 3 configs that represent the sweet spot, then pick the best by a combined view (rel_l2 primary, then mae, then maxae). Then train the final model on ALL 148 data with that recipe, and also build in the cheap recalibration if it helps. Let me also reconsider: for the FINAL model trained on all 148, should I use the same epochs? With more data (148 vs 123 per fold), the same epochs is fine. Also, crucially: for the final single model, seed matters (single-model variance). I could train several seeds on all 148 and... I can't select (no held-out). BUT I could hold out the public_test (16) as a selection set: train final candidates on train+val (132), pick best seed on public_test, then... no, that wastes 16 samples and selection on 16 is noisy. Alternative: Train on all 148 but reduce single-model variance by using a longer training (more stable) and possibly SWA (which gave stable, slightly better). Actually SWA earlier didn't help at wd1e-3/ep200. Let me reconsider SWA/EMA with the new recipe (yrefl, longer epochs) , tail averaging over the last part of cosine could reduce variance. But earlier SWA ≈ last. The cleanest variance reduction available in a single checkpoint: train on all data, and to reduce the "unlucky seed" risk, I can generate the final model and sanity-check its metrics on the full 148 (train error) , but that's train error, not generalization. Hmm, I think the robust move is: pick the recipe, train the final model on all 148 data. To hedge seed variance, I'll train a handful of seeds and pick the one whose predictions best agree with the ensemble-of-all-seeds on the training data (i.e., the "most central/representative" model), or simply pick median. Actually, a reasonable selection criterion without a holdout: the model closest to the average prediction of all seed models ON THE TRAINING inputs is likely the most "typical" (less overfit to a particular init). That's a heuristic. Alternatively, just fix a seed. Actually, the BEST approach for a single robust model might be distillation: train K=8 seed teachers on all 148, compute their averaged predictions on training inputs (+ augmented copies), then train one student PointNet to regress those averaged soft targets. The student approximates the ensemble (which is ~0.196-0.203 OOF vs single ~0.21). This is a legit way to get ensemble-quality in one checkpoint. The gain is ~0.01 rel_l2, which is meaningful given we're optimizing this metric. Let me verify distillation works in OOF before committing. Plan: - Distillation OOF: for each fold, train K teachers on train-part, average their predictions on a large set of augmented training inputs, train student on those. Evaluate student on held-out fold. Compare to single and ensemble. That's expensive (K+1 trainings per fold per seed). Let me do a lighter check: K=5 teachers, student, 6 folds, 2 s …[truncated]

`wd1e-2_ep700` also looks strong (single 0.208, best-ish maxae 0.684). All top configs cluster within OOF noise (±0.01). Let me settle the recipe with a robust 8-seed head-to-head varying weight decay at ep500.

bash

cd /app && cat > /app/run6.py << 'PYEOF'
import cv2 as cv, lib_train as L, time, numpy as np
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=3e-3,
          epochs=500,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0,yreflect=True)
SEEDS=tuple(range(8))
def ev(name,over):
    cfg=dict(base); cfg.update(over)
    t=time.time(); _,preds=cv.oof(cfg,seeds=SEEDS)
    per=[L.metrics(preds[s],cv.Y) for s in range(len(SEEDS))]
    rl=np.array([p['rel_l2'] for p in per])
    print('%-16s single rel_l2=%.4f±%.4f mae=%.4f maxae=%.4f | ens=%.4f (%.0fs)'%(
        name,rl.mean(),rl.std(),np.mean([p['mae'] for p in per]),np.mean([p['maxae'] for p in per]),
        L.metrics(preds.mean(0),cv.Y)['rel_l2'],time.time()-t),flush=True)
for name,over in [('wd3e-3',{'weight_decay':3e-3}),('wd6e-3',{'weight_decay':6e-3}),('wd1e-2',{'weight_decay':1e-2})]:
    ev(name,over)
PYEOF
nohup python run6.py > /app/run6.log 2>&1 &
echo "pid $!"

output

pid 699

While the head-to-head runs, let me prepare and validate the distillation idea , training seed-ensemble teachers and distilling into one student could capture the ~0.01 ensemble gain in a single checkpoint. First let me verify it in OOF.

bash

cd /app && cat > /app/distill.py << 'PYEOF'
import torch, numpy as np, lib_train as L, math
from lib_train import DEV
import torch.nn as nn, torch.nn.functional as F
from candidate_model import PointNetRegressor
data=L.load_all()
X=torch.cat([data['train'][0],data['val'][0],data['public_test'][0]],0)
Y=torch.cat([data['train'][1],data['val'][1],data['public_test'][1]],0)
N=X.shape[0]

def make_model(cfg): return PointNetRegressor(cfg['feature_width'],cfg['global_width'],cfg['head_width']).to(DEV)

def train_one(xtr,ytr_n,nx,cfg,seed,soft=None):
    # soft: optional [n,1] normalized soft targets to blend
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    model=make_model(cfg)
    opt=torch.optim.AdamW(model.parameters(),lr=cfg['lr'],weight_decay=cfg['weight_decay'])
    n=xtr.shape[0]; bs=cfg['batch_size']; epochs=cfg['epochs']
    total=epochs*math.ceil(n/bs); warm=int(0.05*total)
    def lrf(s):
        if s<warm: return s/max(1,warm)
        p=(s-warm)/max(1,total-warm); return 0.5*(1+math.cos(math.pi*p))
    step=0
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=DEV)
        for st in range(0,n,bs):
            bidx=perm[st:st+bs]; xb=L.augment(xtr[bidx],cfg); xb=nx(xb)
            tgt=ytr_n[bidx] if soft is None else soft[bidx]
            for g in opt.param_groups: g['lr']=cfg['lr']*lrf(step)
            opt.zero_grad(set_to_none=True); pred=model(xb); loss=F.mse_loss(pred,tgt)
            loss.backward(); nn.utils.clip_grad_norm_(model.parameters(),1.0); opt.step(); step+=1
    return model

def predict(model,xe,nx,dy):
    model.eval()
    with torch.no_grad(): return dy(model(nx(xe)))

def run_fold(Xtr,Ytr,Xte,cfg,K,student_seed=100,tta=8):
    Xtr=Xtr.to(DEV); Ytr=Ytr.to(DEV); Xte=Xte.to(DEV)
    fm=Xtr.mean((0,1)); fs=Xtr.std((0,1)).clamp_min(1e-6); lm=Ytr.mean(0); ls=Ytr.std(0).clamp_min(1e-6)
    nx=lambda x:(x-fm)/fs; ny=lambda y:(y-lm)/ls; dy=lambda y:y*ls+lm
    ytr_n=ny(Ytr)
    teachers=[train_one(Xtr,ytr_n,nx,cfg,seed=s) for s in range(K)]
    # teacher soft targets on clean train inputs (normalized space), averaged, with TTA (y-reflect)
    soft=torch.zeros_like(ytr_n)
    cnt=0
    for t in teachers:
        t.eval()
        with torch.no_grad():
            for r in range(tta):
                xa=L.augment(Xtr,{'yreflect':True}) if r>0 else Xtr
                soft+=ny(dy(t(nx(xa)))); cnt+=1
    soft/=cnt
    # ensemble test prediction (reference)
    with torch.no_grad():
        ens=torch.stack([predict(t,Xte,nx,dy) for t in teachers],0).mean(0).cpu()
    # student distilled on soft targets
    stu=train_one(Xtr,ytr_n,nx,cfg,seed=student_seed,soft=soft)
    stu_pred=predict(stu,Xte,nx,dy).cpu()
    # single teacher (seed 0) ref
    single=predict(teachers[0],Xte,nx,dy).cpu()
    return single,ens,stu_pred

def oof(cfg,K=5,kfold=6,seeds=(0,),folds_seed=123):
    g=torch.Generator().manual_seed(folds_seed); perm=torch.randperm(N,generator=g)
    fold=torch.zeros(N,dtype=torch.long)
    for i,idx in enumerate(perm): fold[idx]=i%kfold
    S=torch.zeros(N,1); E=torch.zeros(N,1); T=torch.zeros(N,1)
    for f in range(kfold):
        te=(fold==f).nonzero().ravel(); tr=(fold!=f).nonzero().ravel()
        s,e,t=run_fold(X[tr],Y[tr],X[te],cfg,K)
        S[te]=s; E[te]=e; T[te]=t
    for nm,P in [('single',S),('ensemble',E),('student',T)]:
        m=L.metrics(P,Y); print('  %-9s rel_l2=%.4f mae=%.4f maxae=%.4f'%(nm,m['rel_l2'],m['mae'],m['maxae']),flush=True)

if __name__=='__main__':
    import time
    cfg=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=3e-3,
             epochs=350,batch_size=16,loss='mse',yreflect=True)
    t=time.time(); print('distill OOF K=5 ep350'); oof(cfg,K=5); print('(%.0fs)'%(time.time()-t))
PYEOF
echo "waiting for run6 to free GPU a bit..."; sleep 1; echo ok

output

waiting for run6 to free GPU a bit...
ok

Let me wait for run6 (head-to-head wd) results first, which will also tell me the recipe. Then run the distillation test. Let me check run6 progress. But run6 uses 8 seeds × 6 folds × ep500 ≈ 8*6*17s ≈ 800s per config × 3 = 40 min. That's long. Let me check. Actually, let me run the distillation test now in parallel (GPU can handle it, models tiny). But contention slows both. Given my token budget, let me be efficient. Let me launch the distillation test in background now too, so both finish around the same time. Actually, concern: running run6 (3 configs × 8 seeds) AND distill simultaneously = heavy. But the GPU is underutilized per-process for tiny models. Let me launch distill and monitor both. Let me launch distill.

Let me launch the distillation OOF test in the background to run alongside the weight-decay head-to-head (models are tiny, so they can share the GPU).

bash

cd /app && nohup python distill.py > /app/distill.log 2>&1 & echo "distill pid $!"; sleep 280; echo "=== run6 ==="; cat run6.log | grep -v Warning; echo "=== distill ==="; cat distill.log | grep -v Warning

output

distill pid 721
=== run6 ===
=== distill ===

Both still running, no output yet. run6 at 8 seeds × 6 folds ep500 takes ~800s for first config. distill K=5 ep350 6 folds = 6*(5 teachers + 1 student)=36 trainings ep350 ~ 36*12s=430s plus TTA overhead. Let me wait more.

Both are still computing (they share the GPU, so each is slower). Let me wait.

bash

sleep 290; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning; echo "=== distill ==="; cat /app/distill.log | grep -v Warning

output

=== run6 ===
=== distill ===
distill OOF K=5 ep350
  single    rel_l2=0.2044 mae=0.1262 maxae=0.6514
  ensemble  rel_l2=0.1957 mae=0.1187 maxae=0.6769
  student   rel_l2=0.2243 mae=0.1333 maxae=0.6551
(348s)

Distillation result: - single (1 teacher): rel_l2 0.2044 - ensemble (5 teachers): rel_l2 0.1957 (the target) - student (distilled): rel_l2 0.2243 , WORSE than single! So distillation HURT. The student trained on soft targets did worse than a single model. This can happen because the soft targets on training inputs are just the teachers' fit to training data (which they nearly memorize), so soft targets ≈ true labels but slightly smoothed; the student doesn't gain the ensemble's generalization. The ensemble benefit comes from averaging independent errors at TEST time, which can't be distilled from train-input predictions alone. So distillation is not helpful. Drop it. I'll ship a single well-trained model. This means my realistic target is single-model rel_l2 ≈ 0.205-0.21 on OOF, and I should pick the best robust recipe and a good seed. Now, the key lever remaining for a SINGLE model: reduce seed variance by picking a good seed via some valid selection, OR accept ~0.205. Wait , there's a subtle but valid approach to get ensemble benefit in ONE PointNet: since the architecture ends in a max-pool + MLP head, I cannot merge. But what about training a single model with more capacity usage... no. Let me reconsider the "single model selection" problem. For the final model trained on all 148 data, I cannot evaluate generalization. But I CAN use cross-validation to measure which SEED tends to be good on average , but seed quality doesn't transfer across different training sets. However, I realize the ensemble OOF (0.1957) is meaningfully better than single (0.2044). Since I can only ship one model, can I ship the ensemble "folded" into the allowed structure? No. Hmm wait, let me reconsider whether I can exploit multiple forward passes. The harness calls model(points) once. I can't. OK so single model it is. Let me focus on: 1. Best recipe (from run6). 2. Reduce single-model variance: the best practical approach is to select the seed that's likely good. Since I train the final on all 148, and I have labels for all 148, I can't get an unbiased generalization estimate. BUT I can hold out a small internal validation from the 148 to select the seed, then... that reduces training data. Alternative: Accept single model, but make training as stable as possible (the std across seeds was ~0.006-0.01). Pick the recipe with lowest variance AND lowest mean. Actually, here's a cleaner idea to reduce variance in a single shipped model: average the predictions of multiple seeds is not allowed, BUT I could select, among several final models trained on all 148, the one whose predictions on the 148 training points are closest to the MEAN prediction of all the models (the medoid). The medoid model is the most "representative" and avoids outlier/unlucky seeds. This is a valid heuristic selection using only training inputs (no labels needed for selection, so no overfit to labels). Let me consider it. The medoid should have near-average (hence lower-variance) behavior → closer to ensemble-ish but it's still a single model, so it'll be around the mean single performance (~0.205), not the ensemble (0.196). Marginal benefit, mainly avoids bad seeds. Let me first get run6 results to finalize wd, then decide on epochs and seed strategy. Let me wait for run6. Also, let me reconsider the maxae. The ensemble maxae 0.677, single 0.651. The worst-case sample is inherently hard. Not much to do. Let me wait for run6. It's taking long (8 seeds). Let me check progress.

Distillation actually **hurt** (student 0.224 vs single 0.204): the ensemble's benefit comes from averaging independent errors at test time, which can't be recovered from teachers' train-input predictions. So I'll ship a single well-trained model. Let me wait for the weight-decay head-to-head to finalize the recipe.

bash

sleep 240; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning; echo "running:"; ps aux|grep -E 'run6.py'|grep -v grep|wc -l

output

=== run6 ===
wd3e-3           single rel_l2=0.2109±0.0077 mae=0.1252 maxae=0.7289 | ens=0.2016 (657s)
running:
1

wd3e-3 ep500 over 8 seeds: single rel_l2 = 0.2109 ± 0.0077, mae 0.1252, maxae 0.729, ens 0.2016. So the robust single estimate is ~0.211 (not 0.204; that earlier 0.2044 was a lucky 4-seed draw). Good to have the honest number with 8 seeds. Now waiting for wd6e-3 and wd1e-2. Note the ensemble is 0.2016 , consistently ~0.01 better but unattainable in one model. Let me wait for the other two wd values. Distill is done (pid still showing 1 for run6 only). Let me wait. Given the variance, the three wd values will likely be within noise. Let me also think about what else could reduce the SINGLE model error beyond recipe tuning, since we seem stuck ~0.21. Idea: Test-time reflection is not available. But here's a thought , I can bake y-reflection "averaging" into the WEIGHTS? No. Another idea: The input normalization (feat_mean/feat_std) , what if I center each cloud to make the model more robust? But the harness applies fixed feat_mean/feat_std (global), not per-cloud. I provide global constants. Per-cloud centering isn't possible through the provided hook... unless feat_mean could be set to 0 and rely on... no, per-cloud centering requires computing per input, which the harness won't do. Wait , actually the data is already pre-centered at bounding-box centroid. So it's fine. Let me reconsider feature normalization: currently per-axis std. The three axes have very different stds (x 0.142, y 0.270, z 0.087). Per-axis standardization scales them to unit std, which distorts the aspect ratio. Maybe ISOTROPIC scaling (divide by a single scalar) preserves shape and could help the conv layers see true geometry. Let me test isotropic vs per-axis normalization in OOF. This is a cheap change and might matter. Actually the first conv layer can learn per-axis scaling, so it shouldn't matter in theory, but with limited data, the inductive bias of isotropic (preserving true proportions) might help or hurt. Worth a quick test. Actually, let me also reconsider: the y-reflection augmentation must be applied in the RAW coordinate space (before normalization), which I do (augment then normx). Good. And at inference, harness normalizes with feat_mean/feat_std. If I use per-axis mean, the y-reflection during training flips y around 0, but the raw y-mean is 0.0075 (near 0). Reflecting y→ -y then subtracting feat_mean_y=0.0075... slight asymmetry. Negligible. Actually to be clean, for y-reflection symmetry, feat_mean_y should be ~0. It's 0.0075, fine. Let me test isotropic normalization quickly once run6 finishes. Let me wait. Let me also think about whether I've exhausted architecture-free gains. Current best robust single ≈ 0.211. Let me see if there's a better LR or schedule. I used cosine to 0. Maybe a higher initial LR with longer warmup, or lr=1.5e-3. bs32 hurt. lr sweep not done thoroughly with the new recipe. Let me include an lr test. Let me wait for run6 and then run a small combined test: isotropic norm, lr variants.

`wd3e-3 ep500` over 8 seeds: single rel_L2 = 0.2109 ± 0.0077 , so the honest robust number is ~0.211 (the earlier 0.204 was a lucky draw), with ensemble at 0.202. Let me wait for the other weight decays.

bash

sleep 290; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning; echo "running:"; ps aux|grep -E 'run6.py'|grep -v grep|wc -l

output

=== run6 ===
wd3e-3           single rel_l2=0.2109±0.0077 mae=0.1252 maxae=0.7289 | ens=0.2016 (657s)
running:
1

bash

sleep 290; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning

output

=== run6 ===
wd3e-3           single rel_l2=0.2109±0.0077 mae=0.1252 maxae=0.7289 | ens=0.2016 (657s)
wd6e-3           single rel_l2=0.2112±0.0053 mae=0.1251 maxae=0.7105 | ens=0.2010 (634s)

wd6e-3: single 0.2112 ± 0.0053 (lower variance!), mae 0.1251, maxae 0.7105, ens 0.2010. Very similar mean to wd3e-3 but lower std (0.0053 vs 0.0077) and slightly better maxae. So wd6e-3 is a bit more stable. Let me wait for wd1e-2. The means are essentially identical (~0.211). Lower variance is good for a single shipped model. Let me lean toward wd6e-3 or 1e-2. Let me wait for the last one.

`wd6e-3` matches the mean (0.2112) with lower variance (±0.0053) and better maxae , appealing for a single shipped model. Let me get wd1e-2.

bash

sleep 200; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning; ps aux|grep run6.py|grep -v grep|wc -l

output

=== run6 ===
wd3e-3           single rel_l2=0.2109±0.0077 mae=0.1252 maxae=0.7289 | ens=0.2016 (657s)
wd6e-3           single rel_l2=0.2112±0.0053 mae=0.1251 maxae=0.7105 | ens=0.2010 (634s)
1

bash

sleep 160; echo "=== run6 ==="; cat /app/run6.log | grep -v Warning; ps aux|grep run6.py|grep -v grep|wc -l

output

=== run6 ===
wd3e-3           single rel_l2=0.2109±0.0077 mae=0.1252 maxae=0.7289 | ens=0.2016 (657s)
wd6e-3           single rel_l2=0.2112±0.0053 mae=0.1251 maxae=0.7105 | ens=0.2010 (634s)
1

bash

sleep 120; cat /app/run6.log | grep -v Warning | tail -1; ps aux|grep run6.py|grep -v grep|wc -l

output

wd6e-3           single rel_l2=0.2112±0.0053 mae=0.1251 maxae=0.7105 | ens=0.2010 (634s)
1

bash

sleep 120; cat /app/run6.log | grep -v Warning | tail -1; echo "running:"; ps aux|grep run6.py|grep -v grep|wc -l

output

wd1e-2           single rel_l2=0.2088±0.0058 mae=0.1241 maxae=0.6612 | ens=0.1995 (630s)
running:
0

wd1e-2 ep500: single rel_l2 = 0.2088 ± 0.0058, mae 0.1241, maxae 0.6612 (best maxae!), ens 0.1995 (best ensemble). So wd1e-2 is the best across ALL three metrics: lowest single rel_l2 (0.2088), lowest mae (0.1241), lowest maxae (0.6612), lowest variance-ish (0.0058). Great. So stronger weight decay (1e-2) with yrefl + ep500 is the winner. Let me confirm whether even higher wd (1.5e-2, 2e-2) or ep700 helps further, and test isotropic normalization and lr. Let me run a final refinement with 8 seeds: - wd1e-2 ep500 (baseline confirm) - wd1.5e-2 ep500 - wd2e-2 ep500 - wd1e-2 ep700 - wd1e-2 ep500 isotropic-norm Let me also test lr1.5e-3 with wd1e-2. But each 8-seed config ~630s, 6 configs = ~63 min. That's a lot. Let me be selective: test wd1.5e-2, wd2e-2, ep700, isotropic , 4 configs, 8 seeds ~42 min. Acceptable but let me reduce to 6 seeds to speed (~470s each, ~31 min). Actually, I want to also leave time to build the final model and the save/verify pipeline. Let me do a focused 6-seed run of the most promising: wd1.5e-2 ep500, wd2e-2 ep500, wd1e-2 ep700, and isotropic(wd1e-2 ep500). For isotropic normalization, I need to modify the normalization in cv2. Currently fs = std per axis. Isotropic: use a single scalar (e.g., overall std or 0.25 for [-0.5,0.5] range). But feat_std must be length-3. I can set all three to the same value (e.g., the RMS over all axes). Let me add a cfg flag 'iso_norm' to cv2.oof's train_predict. Let me edit cv2.py to support it. Let me implement iso_norm option and also a lr override already exists. Let me edit cv2.py.

`wd1e-2 ep500` is the clear winner across **all three metrics**: single rel_L2 0.2088±0.0058, MAE 0.1241, maxae 0.6612 (best), ensemble 0.1995. Stronger regularization helps. Let me refine further: higher wd, ep700, and test isotropic normalization (which preserves true geometric proportions). I'll add an isotropic-norm option.

bash

cd /app && python - << 'PYEOF'
import re
s=open('cv2.py').read()
s=s.replace(
"    fm=xtr.mean((0,1)); fs=xtr.std((0,1)).clamp_min(1e-6)",
"    fm=xtr.mean((0,1)); fs=xtr.std((0,1)).clamp_min(1e-6)\n    if cfg.get('iso_norm',False):\n        fs=xtr.std().expand(3).clone().clamp_min(1e-6)")
open('cv2.py','w').write(s)
print('patched iso_norm' if 'iso_norm' in s else 'FAIL')
PYEOF
grep -n "iso_norm" cv2.py

output

patched iso_norm
15:    if cfg.get('iso_norm',False):

bash

cd /app && cat > /app/run7.py << 'PYEOF'
import cv2 as cv, lib_train as L, time, numpy as np
base=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-2,
          epochs=500,batch_size=16,warmup_frac=0.05,loss='mse',clip=1.0,yreflect=True)
SEEDS=tuple(range(6))
def ev(name,over):
    cfg=dict(base); cfg.update(over)
    t=time.time(); _,preds=cv.oof(cfg,seeds=SEEDS)
    per=[L.metrics(preds[s],cv.Y) for s in range(len(SEEDS))]
    rl=np.array([p['rel_l2'] for p in per])
    print('%-18s single rel_l2=%.4f±%.4f mae=%.4f maxae=%.4f | ens=%.4f (%.0fs)'%(
        name,rl.mean(),rl.std(),np.mean([p['mae'] for p in per]),np.mean([p['maxae'] for p in per]),
        L.metrics(preds.mean(0),cv.Y)['rel_l2'],time.time()-t),flush=True)
for name,over in [
  ('wd1e-2_ep500',{}),
  ('wd1.5e-2',{'weight_decay':1.5e-2}),
  ('wd2e-2',{'weight_decay':2e-2}),
  ('wd1e-2_ep700',{'epochs':700}),
  ('wd1e-2_iso',{'iso_norm':True}),
  ('wd1e-2_lr1.5e-3',{'lr':1.5e-3}),
]:
    ev(name,over)
PYEOF
nohup python run7.py > /app/run7.log 2>&1 & echo "pid $!"

output

pid 850

bash

sleep 295; cat /app/run7.log | grep -v Warning

output

<system>Tool ran without output or errors</system>

bash

sleep 200; cat /app/run7.log | grep -v Warning

output

<system>Tool ran without output or errors</system>

No output yet after ~500s. 6 seeds × 6 folds × ep500 ≈ 36 trainings × ~14s = ~500s for the first config. Should be close. Let me wait a bit more.

bash

sleep 150; cat /app/run7.log | grep -v Warning; echo "running:"; ps aux|grep run7.py|grep -v grep|wc -l

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
running:
1

wd1e-2_ep500 with 6 seeds: single 0.2097 ± 0.0061, consistent with the 8-seed run (0.2088). Good reproducibility. Let me wait for the variants.

Reproducible: `wd1e-2_ep500` gives single 0.2097±0.0061 (matches the 8-seed run). Let me wait for the higher-wd, ep700, isotropic, and lr variants.

bash

sleep 295; cat /app/run7.log | grep -v Warning

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)

bash

sleep 260; cat /app/run7.log | grep -v Warning

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)

wd1.5e-2: 0.2123 , slightly worse than wd1e-2 (0.2097). So wd1e-2 is near-optimal; higher wd starts hurting. Let me wait for the rest (wd2e-2, ep700, iso, lr1.5e-3).

bash

sleep 295; cat /app/run7.log | grep -v Warning | tail -4

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)

bash

sleep 240; cat /app/run7.log | grep -v Warning | tail -4

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)
wd2e-2             single rel_l2=0.2133±0.0056 mae=0.1268 maxae=0.7052 | ens=0.2051 (492s)

wd2e-2: 0.2133 , confirms wd1e-2 is the sweet spot. Let me wait for ep700, iso, lr1.5e-3.

Confirmed wd1e-2 is the sweet spot (wd1.5e-2 → 0.2123, wd2e-2 → 0.2133). Let me get the remaining three (ep700, isotropic norm, lr1.5e-3).

bash

sleep 295; cat /app/run7.log | grep -v Warning | tail -4

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)
wd2e-2             single rel_l2=0.2133±0.0056 mae=0.1268 maxae=0.7052 | ens=0.2051 (492s)

bash

sleep 260; cat /app/run7.log | grep -v Warning | tail -4

output

wd1e-2_ep500       single rel_l2=0.2097±0.0061 mae=0.1248 maxae=0.6774 | ens=0.2013 (517s)
wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)
wd2e-2             single rel_l2=0.2133±0.0056 mae=0.1268 maxae=0.7052 | ens=0.2051 (492s)
wd1e-2_ep700       single rel_l2=0.2119±0.0085 mae=0.1275 maxae=0.7050 | ens=0.2039 (682s)

ep700: 0.2119 ± 0.0085 , not better than ep500 (0.2097), and higher variance. So ep500 is good. Let me wait for iso and lr1.5e-3.

ep700 doesn't beat ep500 (0.2119 vs 0.2097). Let me get the isotropic-norm and lr variants.

bash

sleep 295; cat /app/run7.log | grep -v Warning | tail -3

output

wd1.5e-2           single rel_l2=0.2123±0.0053 mae=0.1272 maxae=0.6849 | ens=0.2021 (506s)
wd2e-2             single rel_l2=0.2133±0.0056 mae=0.1268 maxae=0.7052 | ens=0.2051 (492s)
wd1e-2_ep700       single rel_l2=0.2119±0.0085 mae=0.1275 maxae=0.7050 | ens=0.2039 (682s)

bash

sleep 230; cat /app/run7.log | grep -v Warning | tail -2

output

wd1e-2_ep700       single rel_l2=0.2119±0.0085 mae=0.1275 maxae=0.7050 | ens=0.2039 (682s)
wd1e-2_iso         single rel_l2=0.2083±0.0060 mae=0.1252 maxae=0.7052 | ens=0.1997 (482s)

wd1e-2_iso (isotropic normalization): single 0.2083 ± 0.0060, ens 0.1997 , slightly BETTER than per-axis (0.2097). maxae a bit higher (0.705 vs 0.677). The rel_l2 and ens are marginally better with isotropic normalization. This is within noise but consistent with the idea that preserving true geometric proportions helps slightly. Let me wait for lr1.5e-3, then finalize. The iso result is encouraging but marginal (0.2083 vs 0.2097, diff 0.0014, within noise std 0.006). I'll consider it. Let me get the last result.

Isotropic normalization is marginally better (single 0.2083 vs 0.2097, ens 0.1997). Let me get the last variant (lr1.5e-3).

bash

sleep 250; cat /app/run7.log | grep -v Warning | tail -2; echo "running:"; ps aux|grep run7.py|grep -v grep|wc -l

output

wd1e-2_ep700       single rel_l2=0.2119±0.0085 mae=0.1275 maxae=0.7050 | ens=0.2039 (682s)
wd1e-2_iso         single rel_l2=0.2083±0.0060 mae=0.1252 maxae=0.7052 | ens=0.1997 (482s)
running:
1

bash

sleep 180; cat /app/run7.log | grep -v Warning | tail -1; ps aux|grep run7.py|grep -v grep|wc -l

output

wd1e-2_lr1.5e-3    single rel_l2=0.2127±0.0048 mae=0.1270 maxae=0.7205 | ens=0.2029 (489s)
0

lr1.5e-3: 0.2127 , slightly worse than lr1e-3 (0.2097). So lr1e-3 is good. Final recipe decision: - yreflect=True - wd=1e-2 - epochs=500 - lr=1e-3, bs=16, cosine schedule with 5% warmup - loss=mse - clip=1.0 - normalization: isotropic gave marginally better (0.2083 vs 0.2097). I'll go with isotropic since it's consistently slightly better on rel_l2 and ens, and it preserves geometry. But maxae was slightly worse. Let me consider combining: the differences are within noise. Let me make a decision that's robust. Both per-axis and iso are essentially tied. I'll lean toward the one with better PRIMARY metric (rel_l2): iso (0.2083) and ens (0.1997). But maxae: per-axis 0.677 vs iso 0.705. Since primary is rel_l2, iso is marginally preferred, but the maxae regression is a small concern. Actually, let me reconsider: I could get the benefit of lower variance AND good metrics by picking the right final-model strategy. Let me now focus on the FINAL model: Plan for final model: 1. Train on ALL 148 labeled samples (train+val+public_test) with the chosen recipe. 2. Since single-model has seed variance (±0.006), train several seeds (say 12) on all 148 and select the "medoid" model , the one whose predictions on the 148 inputs are closest to the mean prediction across all seeds. This avoids unlucky seeds without needing held-out labels. This should give a representative model near the mean single performance (~0.208), possibly a touch better, and avoids shipping a bad-seed model. Actually, wait. Let me reconsider medoid selection. The medoid minimizes distance to the average prediction vector. The average prediction (ensemble) generalizes better (0.1997). The medoid model is the single model whose training-set predictions are most central. But does central-on-training translate to good generalization? Roughly, models that agree with the ensemble on training inputs are "typical" and likely have typical (good) generalization, avoiding outlier seeds that overfit weirdly. It's a reasonable, label-free heuristic. The risk: all models fit training data similarly well (near-memorization), so training-input predictions may not discriminate generalization. Given augmentation + wd1e-2, they don't fully memorize, so there's signal. Alternatively, a cleaner selection: use the 148 with a final round of cross-validation to estimate each seed's... no, seeds don't transfer. Actually, the simplest robust choice: train on all 148 with a FIXED seed that I've validated tends to be fine, OR train a few seeds and evaluate each on the FULL 148 training set (train error) , but that rewards overfitting (lower train error = more overfit = worse generalization potentially). So NOT train error. The medoid (consensus) approach is better than train-error selection. Let me also consider: evaluate candidate final models via their agreement with an ENSEMBLE's predictions on the 148 (which is the best available proxy for truth-like smoothness). Pick the model whose predictions correlate best / are closest to the ensemble average. That's the medoid idea essentially. Let me implement: train M=12 models on all 148 (recipe: iso, wd1e-2, ep500, yrefl), collect predictions on all 148 inputs (in eval mode). Compute mean prediction vector. Select the model minimizing ||pred_i - mean||. Save that model. But wait , there's an even better approach that's fully valid: Since I have labels for all 148, I can do a final hold-out-based selection that doesn't waste much data: Actually no, I want all 148 for training. Hmm, let me reconsider once more whether medoid actually helps vs just picking seed 0. The 8-seed std is 0.006. Best seed ~0.20, worst ~0.22. Medoid should land near median ~0.209. Picking seed 0 is a gamble within that range. Medoid reduces downside risk. It's worth doing. Actually, let me verify the medoid heuristic with an OOF-style experiment to make sure it's not worse than random seed. I'll do: for each fold, train M models on train-part, compute medoid by con …[truncated]

Recipe locked in: **yreflect, wd=1e-2, ep500, lr=1e-3, bs=16, cosine, MSE**, with isotropic normalization (marginally best: single 0.2083, ens 0.1997). lr1.5e-3 and higher wd were worse. Since I ship one model with ±0.006 seed variance, I want a label-free way to avoid an unlucky seed. Let me validate a **medoid/consensus selection** (pick the model whose predictions on training inputs are closest to the multi-seed mean) via OOF before using it.

bash

cd /app && cat > /app/medoid_test.py << 'PYEOF'
import torch, numpy as np, lib_train as L, math, time
from lib_train import DEV
import torch.nn as nn, torch.nn.functional as F
from candidate_model import PointNetRegressor
data=L.load_all()
X=torch.cat([data['train'][0],data['val'][0],data['public_test'][0]],0)
Y=torch.cat([data['train'][1],data['val'][1],data['public_test'][1]],0)
N=X.shape[0]
CFG=dict(feature_width=64,global_width=256,head_width=128,lr=1e-3,weight_decay=1e-2,
         epochs=500,batch_size=16,loss='mse',yreflect=True,iso_norm=True)
def train_one(xtr,ytr_n,nx,cfg,seed):
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    model=PointNetRegressor(cfg['feature_width'],cfg['global_width'],cfg['head_width']).to(DEV)
    opt=torch.optim.AdamW(model.parameters(),lr=cfg['lr'],weight_decay=cfg['weight_decay'])
    n=xtr.shape[0]; bs=cfg['batch_size']; epochs=cfg['epochs']
    total=epochs*math.ceil(n/bs); warm=int(0.05*total)
    lrf=lambda s: s/max(1,warm) if s<warm else 0.5*(1+math.cos(math.pi*(s-warm)/max(1,total-warm)))
    step=0
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=DEV)
        for st in range(0,n,bs):
            bidx=perm[st:st+bs]; xb=nx(L.augment(xtr[bidx],cfg))
            for g in opt.param_groups: g['lr']=cfg['lr']*lrf(step)
            opt.zero_grad(set_to_none=True); loss=F.mse_loss(model(xb),ytr_n[bidx])
            loss.backward(); nn.utils.clip_grad_norm_(model.parameters(),1.0); opt.step(); step+=1
    model.eval(); return model
def fold(Xtr,Ytr,Xte,M):
    Xtr=Xtr.to(DEV); Ytr=Ytr.to(DEV); Xte=Xte.to(DEV)
    fm=Xtr.mean((0,1)); fs=Xtr.std().expand(3).clone().clamp_min(1e-6); lm=Ytr.mean(0); ls=Ytr.std(0).clamp_min(1e-6)
    nx=lambda x:(x-fm)/fs; ny=lambda y:(y-lm)/ls; dy=lambda y:y*ls+lm
    ytr_n=ny(Ytr)
    models=[train_one(Xtr,ytr_n,nx,CFG,seed=s) for s in range(M)]
    with torch.no_grad():
        tr_pred=torch.stack([m(nx(Xtr)) for m in models],0)  # [M,n,1]
        te_pred=torch.stack([dy(m(nx(Xte))) for m in models],0)  # [M,nte,1]
    consensus=tr_pred.mean(0)
    dist=((tr_pred-consensus)**2).mean((1,2))  # [M]
    medoid=int(dist.argmin().item())
    return te_pred.cpu(), medoid
def run(M=9,kfold=6,folds_seed=123):
    g=torch.Generator().manual_seed(folds_seed); perm=torch.randperm(N,generator=g)
    fid=torch.zeros(N,dtype=torch.long)
    for i,idx in enumerate(perm): fid[idx]=i%kfold
    P=torch.zeros(M,N,1); MED=torch.zeros(N,1); 
    medrl=[]
    for f in range(kfold):
        te=(fid==f).nonzero().ravel(); tr=(fid!=f).nonzero().ravel()
        tep,med=fold(X[tr],Y[tr],X[te],M)
        P[:,te]=tep; MED[te]=tep[med]
    # per-seed
    per=[L.metrics(P[s],Y)['rel_l2'] for s in range(M)]
    print('per-seed rel_l2 mean=%.4f min=%.4f max=%.4f'%(np.mean(per),np.min(per),np.max(per)))
    print('ensemble rel_l2=%.4f'%L.metrics(P.mean(0),Y)['rel_l2'])
    m=L.metrics(MED,Y); print('MEDOID   rel_l2=%.4f mae=%.4f maxae=%.4f'%(m['rel_l2'],m['mae'],m['maxae']))
t=time.time(); run(M=9); print('(%.0fs)'%(time.time()-t))
PYEOF
nohup python medoid_test.py > /app/medoid.log 2>&1 & echo "pid $!"

output

pid 1002

bash

sleep 295; cat /app/medoid.log | grep -v Warning

output

<system>Tool ran without output or errors</system>

M=9 models × 6 folds = 54 trainings at ep500 ≈ 54 × ~14s = ~750s. Let me wait more.

The medoid test trains 54 models (9 seeds × 6 folds), so it takes a while. Let me wait.

bash

sleep 295; cat /app/medoid.log | grep -v Warning; echo "running:"; ps aux|grep medoid_test|grep -v grep|wc -l

output

running:
1

bash

sleep 230; cat /app/medoid.log | grep -v Warning; echo "running:"; ps aux|grep medoid_test|grep -v grep|wc -l

output

per-seed rel_l2 mean=0.2075 min=0.1903 max=0.2203
ensemble rel_l2=0.1977
MEDOID   rel_l2=0.1990 mae=0.1223 maxae=0.6243
(738s)
running:
0

Excellent! The medoid selection works very well: - per-seed rel_l2: mean 0.2075, min 0.1903, max 0.2203 - ensemble: 0.1977 - MEDOID: rel_l2 0.1990, mae 0.1223, maxae 0.6243 The medoid (0.1990) is MUCH better than the mean single seed (0.2075) and almost as good as the full ensemble (0.1977)! And maxae 0.6243 is the best I've seen, mae 0.1223 best too. The medoid selection captures nearly all the ensemble benefit in a SINGLE model, because it picks the most "central" model which tends to avoid idiosyncratic overfitting. This is a great result. So my final strategy: 1. Train M models (say 15-20 for a good consensus) on ALL 148 data with the recipe (iso norm, wd1e-2, ep500, yrefl). 2. Compute consensus predictions on the 148 training inputs. 3. Select the medoid model (closest to consensus). 4. Save that single model's state_dict + normalization constants. The medoid OOF estimate is rel_l2 ≈ 0.199, mae ≈ 0.122, maxae ≈ 0.624. Let me use more models for the final consensus (M=20) to get a more stable consensus and better medoid. Actually, more models → better consensus estimate → better medoid selection. Let me use M=20 or 25. But wait , for the FINAL model, I train on all 148, so there's no held-out to measure. I rely on the OOF estimate (0.199) as my expectation for the held-out grading set. Good. One more consideration: should I add a tiny linear recalibration baked into label_mean/label_std? Earlier recalibration helped only marginally (0.2244→0.2232). With the better model it's probably negligible and risky (could overfit the 148). I'll skip recalibration to avoid overfitting; the medoid already gives clean results. Actually, let me reconsider: the medoid selects based on consensus on TRAINING inputs. For the final model trained on all 148, I compute consensus on those same 148 and pick medoid. This mirrors exactly what the OOF test did (consensus on train-part, i.e., the fold's training data). Wait, in the OOF test, consensus was computed on the fold's TRAINING inputs (tr), and medoid selected by agreement there, then evaluated on held-out (te). For the final, consensus on all 148 (training), medoid selected, evaluated on external held-out. Consistent. Let me also double check: in the OOF medoid test, I selected medoid per-fold using that fold's train consensus , exactly analogous. Good, the 0.199 estimate is valid. Now let me also decide per-axis vs iso norm for the final. The medoid test used iso_norm=True and got great results (0.199). Let me stick with iso_norm. Let me now write the final training script that: 1. Loads all 148 samples. 2. Computes feat_mean (per-axis mean), feat_std (isotropic: single scalar std expanded to 3), label_mean, label_std from all 148. 3. Trains M=20 models with the recipe. 4. Selects medoid via consensus on the 148 inputs. 5. Saves /app/outputs/model.pt with required keys. Then verify it loads into the fixed architecture and produces [B,1]. Let me also make the training script robust and reproducible. Let me also verify the saved model reproduces predictions via the exact inference path the harness uses: load state_dict into PointNetRegressor, normalize with feat_mean/feat_std, forward, denorm with label_mean/label_std. Let me confirm how the harness applies normalization. The task says: feat_mean/feat_std "used to standardize the input coordinates before the forward pass" , so x_norm = (x - feat_mean)/feat_std. label_mean/label_std "used to standardize the target during training and de-standardize the model output at inference" , so pred = out*label_std + label_mean. This matches my convention. Good. feat_mean/feat_std are length-3, applied per-coordinate. For isotropic, feat_std = [s,s,s]. Fine. Let me write the final script. I'll train M=20 models. At ep500, 148 samples, bs16 → 10 batches/epoch × 500 = 5000 steps per model. ~20s each? Let me estimate: earlier per-fold train (123 samples, ep500) was ~14s. 148 samples similar, ~17s. M=20 → ~6 min. Fine. Let me write it and …[truncated]

The medoid selection is a big win: **MEDOID rel_L2 = 0.1990, MAE = 0.1223, maxae = 0.6243** , essentially matching the full ensemble (0.1977) and far better than the mean single seed (0.2075), all while being a single shippable model. It also gives the best MAE and worst-case numbers I've seen. Now I'll write the final training script: train 20 models on all 148 labeled samples with the locked recipe, select the medoid by consensus on the training inputs, and save the checkpoint.

str_replace_based_edit_tool

command: create
path: /app/final_train.py
file_text: from __future__ import annotations
import math, time, json
from pathlib import Path
import torch
import torch.nn as nn
import torch.nn.functional as F

from candidate_model import PointNetRegressor
import lib_train as L

DEV = torch.device("cuda" if torch.cuda.is_available() else "cpu")
APP = Path("/app")

# ----- Final recipe (selected via pooled out-of-fold cross-validation) -----
CFG = dict(
    feature_width=64, global_width=256, head_width=128,
    lr=1e-3, weight_decay=1e-2, epochs=500, batch_size=16,
    warmup_frac=0.05, loss="mse", clip=1.0,
    yreflect=True,     # mass-preserving y-axis mirror (brackets ~symmetric in y)
    iso_norm=True,     # isotropic coord scaling (preserves true proportions)
)
M_MODELS = 20           # train many seeds, then pick the consensus medoid


def load_pooled():
    xs, ys = [], []
    for s in ["train", "val", "public_test"]:
        x, y = L.load_split(s)
        xs.append(x); ys.append(y)
    return torch.cat(xs, 0), torch.cat(ys, 0)


def train_one(xtr, ytr_n, nx, cfg, seed):
    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)
    model = PointNetRegressor(cfg["feature_width"], cfg["global_width"], cfg["head_width"]).to(DEV)
    opt = torch.optim.AdamW(model.parameters(), lr=cfg["lr"], weight_decay=cfg["weight_decay"])
    n = xtr.shape[0]; bs = cfg["batch_size"]; epochs = cfg["epochs"]
    total = epochs * math.ceil(n / bs); warm = int(cfg["warmup_frac"] * total)

    def lrf(s):
        if s < warm:
            return s / max(1, warm)
        p = (s - warm) / max(1, total - warm)
        return 0.5 * (1 + math.cos(math.pi * p))

    step = 0
    for ep in range(epochs):
        model.train()
        perm = torch.randperm(n, device=DEV)
        for st in range(0, n, bs):
            bidx = perm[st:st + bs]
            xb = nx(L.augment(xtr[bidx], cfg))
            for g in opt.param_groups:
                g["lr"] = cfg["lr"] * lrf(step)
            opt.zero_grad(set_to_none=True)
            loss = F.mse_loss(model(xb), ytr_n[bidx])
            loss.backward()
            nn.utils.clip_grad_norm_(model.parameters(), cfg["clip"])
            opt.step(); step += 1
    model.eval()
    return model


def main():
    t0 = time.time()
    X, Y = load_pooled()
    print(f"[final] pooled samples={X.shape[0]} device={DEV}", flush=True)
    Xd = X.to(DEV); Yd = Y.to(DEV)

    feat_mean = Xd.mean(dim=(0, 1))
    if CFG["iso_norm"]:
        feat_std = Xd.std().expand(3).clone().clamp_min(1e-6)
    else:
        feat_std = Xd.std(dim=(0, 1)).clamp_min(1e-6)
    label_mean = Yd.mean(dim=0)
    label_std = Yd.std(dim=0).clamp_min(1e-6)

    nx = lambda x: (x - feat_mean) / feat_std
    ny = lambda y: (y - label_mean) / label_std
    dy = lambda y: y * label_std + label_mean
    ytr_n = ny(Yd)

    models = []
    for s in range(M_MODELS):
        m = train_one(Xd, ytr_n, nx, CFG, seed=s)
        models.append(m)
        print(f"[final] trained model seed={s} ({time.time()-t0:.0f}s)", flush=True)

    # Consensus medoid selection (label-free): pick the model whose predictions
    # on the training inputs are closest to the multi-seed mean prediction.
    with torch.no_grad():
        preds = torch.stack([m(nx(Xd)) for m in models], 0)  # [M, N, 1]
    consensus = preds.mean(0)
    dist = ((preds - consensus) ** 2).mean(dim=(1, 2))
    medoid = int(dist.argmin().item())
    print(f"[final] medoid seed={medoid} dists={[round(float(d),4) for d in dist]}", flush=True)

    # Report training-set agreement metrics for the chosen model (sanity only).
    with torch.no_grad():
        med_pred = dy(models[medoid](nx(Xd))).cpu()
    tr_m = L.metrics(med_pred, Y)
    print(f"[final] medoid TRAIN-fit metrics {tr_m}", flush=True)

    out_dir = APP / "outputs"; out_dir.mkdir(parents=True, exist_ok=True)
    payload = {
        "state_dict": {k: v.cpu().clone() for k, v in models[medoid].state_dict().items()},
        "feat_mean": feat_mean.de
…[truncated]

output

New file created successfully at /app/final_train.py

bash

cd /app && nohup python final_train.py > /app/final.log 2>&1 & echo "pid $!"

output

pid 1060

While the final model trains, let me write a standalone verification script that reloads `model.pt` into the fixed architecture exactly as the grading harness would, and evaluates the three metrics on the held-out-style splits to confirm the checkpoint is valid and well-formed.

str_replace_based_edit_tool

command: create
path: /app/verify_ckpt.py
file_text: from __future__ import annotations
from pathlib import Path
import torch
from candidate_model import build_model, PointNetRegressor
import lib_train as L

APP = Path("/app")


def load_config():
    import json
    return json.loads((APP / "task_config.json").read_text())


@torch.no_grad()
def predict(model, x, ck):
    # Replicate the inference path the grader uses.
    fm = ck["feat_mean"].view(1, 1, 3)
    fs = ck["feat_std"].view(1, 1, 3)
    xn = (x - fm) / fs
    out = model(xn)
    return out * ck["label_std"].view(1, -1) + ck["label_mean"].view(1, -1)


def main():
    ck = torch.load(APP / "outputs" / "model.pt", map_location="cpu", weights_only=True)
    # required keys present?
    for k in ["state_dict", "feat_mean", "feat_std", "label_mean", "label_std"]:
        assert k in ck, f"missing key {k}"
    for k in ["feat_mean", "feat_std"]:
        assert tuple(ck[k].shape) == (3,), (k, ck[k].shape)
    for k in ["label_mean", "label_std"]:
        assert ck[k].numel() == 1, (k, ck[k].shape)

    # Build the fixed architecture and load (strict) to guarantee compatibility.
    model = build_model(load_config())
    assert isinstance(model, PointNetRegressor)
    model.load_state_dict(ck["state_dict"], strict=True)
    model.eval()

    # Shape contract: [B,2048,3] -> [B,1]
    dummy = torch.randn(5, 2048, 3)
    assert tuple(predict(model, dummy, ck).shape) == (5, 1)

    print("checkpoint keys/shapes OK; strict load OK; [B,2048,3]->[B,1] OK")
    for split in ["train", "val", "public_test"]:
        x, y = L.load_split(split)
        p = predict(model, x, ck)
        m = L.metrics(p, y)
        print(f"{split:12s} rel_l2={m['rel_l2']:.4f} mae={m['mae']:.4f} maxae={m['maxae']:.4f}")


if __name__ == "__main__":
    main()

output

New file created successfully at /app/verify_ckpt.py

bash

sleep 180; cat /app/final.log | grep -v Warning

output

[final] pooled samples=148 device=cuda
[final] trained model seed=0 (20s)
[final] trained model seed=1 (36s)
[final] trained model seed=2 (52s)
[final] trained model seed=3 (68s)
[final] trained model seed=4 (84s)
[final] trained model seed=5 (100s)
[final] trained model seed=6 (116s)
[final] trained model seed=7 (133s)
[final] trained model seed=8 (149s)
[final] trained model seed=9 (165s)
[final] trained model seed=10 (181s)

~16s per model. 20 models → ~320s. Let me wait for it to finish (remaining ~9 models + consensus).

Training is progressing at ~16s/model. Let me wait for all 20 models plus medoid selection to finish.

bash

sleep 180; cat /app/final.log | grep -v Warning | tail -15

output

[final] trained model seed=8 (149s)
[final] trained model seed=9 (165s)
[final] trained model seed=10 (181s)
[final] trained model seed=11 (198s)
[final] trained model seed=12 (214s)
[final] trained model seed=13 (231s)
[final] trained model seed=14 (247s)
[final] trained model seed=15 (263s)
[final] trained model seed=16 (279s)
[final] trained model seed=17 (295s)
[final] trained model seed=18 (311s)
[final] trained model seed=19 (327s)
[final] medoid seed=16 dists=[0.0008, 0.0009, 0.0006, 0.0008, 0.0007, 0.0006, 0.0014, 0.001, 0.0011, 0.0006, 0.0012, 0.0007, 0.0008, 0.0008, 0.0007, 0.0011, 0.0005, 0.0007, 0.0006, 0.0009]
[final] medoid TRAIN-fit metrics {'rel_l2': 0.016492731869220734, 'mae': 0.010568831115961075, 'maxae': 0.05360269546508789}
[final] wrote /app/outputs/model.pt (328s)

The final model is trained and saved. Medoid seed=16 selected. Train-fit metrics are very low (rel_l2 0.016) , as expected, it fits training data well (near-memorization but with regularization). This is just a sanity check, not a generalization estimate. Now let me run the verification script to confirm the checkpoint loads into the fixed architecture and works. Note: train/val/public_test were all used in training, so their metrics here are optimistic (train-fit), not generalization. My generalization estimate comes from the OOF medoid test (0.199). Let me run verify_ckpt.py.

The final model is saved (medoid = seed 16). The train-fit metrics are optimistic since all pooled data was used for training , my true generalization estimate remains the OOF medoid result (rel_L2 ≈ 0.199). Let me run the verification script to confirm the checkpoint is valid and loads exactly as the grader expects.

bash

cd /app && python verify_ckpt.py 2>&1 | grep -v Warning

output

checkpoint keys/shapes OK; strict load OK; [B,2048,3]->[B,1] OK
train        rel_l2=0.0166 mae=0.0105 maxae=0.0538
val          rel_l2=0.0181 mae=0.0126 maxae=0.0404
public_test  rel_l2=0.0109 mae=0.0071 maxae=0.0179

The checkpoint loads correctly with strict=True, the shape contract holds, and all required keys/shapes are valid. The metrics on train/val/public_test are very low (rel_l2 ~0.01-0.02) but that's because all these were used for training (memorization). The real generalization estimate is the OOF ~0.199. Now, let me reconsider a concern: I used ALL 148 labeled samples (train+val+public_test) for training the final model. Is that legitimate? The task says the model is applied to a HELD-OUT set (separate from these). Using val and public_test for training is standard practice when they're provided with labels and the final grading is on a different hidden set. The OOF CV (which properly holds out folds) estimates ~0.199 generalization. This should transfer to the hidden held-out set assuming it's from the same distribution. However, there's a subtle risk: what if the grading harness's "held-out set" is actually the public_test or val? No , it says "held-out set of bracket geometries" compared against ground-truth, which is separate. The public_test is given to me with labels, so it's not the hidden grading set. Using it for training is fine. But to be safe, let me reconsider: is there any risk that training on public_test+val could HURT if the hidden set is more similar to train-only distribution? No, more data from the same pool helps. The splits are random draws from the same SimJEB dataset (manifest shows source_bracket_ids scattered). So pooling is beneficial. Let me double-check the data distributions are consistent across splits (they were: label means 0.75/0.76/0.73, similar ranges). Good. Now, let me reconsider whether I should be even more robust. The medoid OOF estimate (0.199) used M=9 models; the final uses M=20, which gives a better consensus and thus likely an equal or better medoid. Good. Let me also reconsider the worst-case (maxae). OOF medoid maxae was 0.624. On the hidden set, worst-case depends on whether there's an outlier bracket. Not much more I can do. Let me also sanity-check: is there any issue with the saved feat_std being isotropic (all three equal)? Let me inspect the saved tensors. Let me verify the saved payload contents and dtypes.

The checkpoint passes all structural checks: required keys present, correct shapes, strict load into the fixed `PointNetRegressor`, and the `[B,2048,3]→[B,1]` contract holds. (The low split metrics are just training-fit, since all pooled data was used; the honest generalization estimate is the OOF medoid ≈ 0.199.) Let me inspect the saved tensors to confirm they're clean.

bash

cd /app && python -c "
import torch
ck=torch.load('outputs/model.pt',weights_only=True)
print('keys:',list(ck.keys()))
for k in ['feat_mean','feat_std','label_mean','label_std']:
    t=ck[k]; print(f'{k:11s} shape={tuple(t.shape)} dtype={t.dtype} vals={[round(float(x),5) for x in t.flatten()]}')
sd=ck['state_dict']; print('state_dict tensors:',len(sd))
print('all finite:', all(torch.isfinite(v).all().item() for v in sd.values() if v.dtype.is_floating_point))
import os; print('file size KB:', round(os.path.getsize('outputs/model.pt')/1024,1))
" 2>&1 | grep -v Warning

output

keys: ['state_dict', 'feat_mean', 'feat_std', 'label_mean', 'label_std']
feat_mean   shape=(3,) dtype=torch.float32 vals=[-0.01178, 0.00743, -0.07199]
feat_std    shape=(3,) dtype=torch.float32 vals=[0.186, 0.186, 0.186]
label_mean  shape=(1,) dtype=torch.float32 vals=[0.7488]
label_std   shape=(1,) dtype=torch.float32 vals=[0.38614]
state_dict tensors: 41
all finite: True
file size KB: 375.7

Everything looks clean: - feat_mean: per-axis mean [-0.0118, 0.0074, -0.0720] - feat_std: isotropic [0.186, 0.186, 0.186] - label_mean: 0.7488, label_std: 0.3861 - state_dict: 41 tensors, all finite. - file size 375 KB. Now, let me do a final robustness consideration. I want to make sure the medoid approach for the FINAL model (trained on all 148) is sound. The OOF test validated it gives ~0.199. But there's a subtle point: in the final, I select medoid by consensus on the 148 TRAIN inputs. The OOF test selected medoid by consensus on the fold's train inputs and evaluated on held-out. So it's validated. Good. However, one more idea to potentially improve robustness further: instead of picking the single medoid, I showed the full ensemble (0.1977) is slightly better than medoid (0.1990). But I can only ship one model. The medoid is the best single-model proxy. Confirmed. Let me reconsider whether I could do better than medoid by a different selection. The medoid is already near-ensemble. Marginal. I'll keep it. Now let me reconsider the primary metric once more. Everything points to rel_l2 ~0.199 on held-out. Let me also re-examine: is the held-out set possibly larger and could the metrics differ? The grading computes rel_l2 over the full held-out set. My OOF over 148 is a good proxy. Let me also make sure I'm not missing an opportunity: what about combining y-reflection as TEST-TIME augmentation baked into weights? Not possible. Let me also reconsider: could I improve by training the final medoid selection with even more models or a better consensus (e.g., trimmed mean consensus)? Marginal. M=20 is fine. I think the solution is complete and robust. Let me do a final end-to-end sanity test simulating the grader more precisely: load model.pt fresh, build model from task_config, run on public_test, and confirm reasonable outputs and that predictions are in physical kg range. Actually I already did that in verify_ckpt.py. The public_test rel_l2 0.0109 is training-fit (optimistic). Let me now also produce a quick, honest held-out-style check to present a realistic number: I'll retrain a medoid model using a proper hold-out (train+val for training, public_test as a never-seen test) to see a realistic generalization number on data the final pipeline didn't memorize. Wait, but my final model DID train on public_test. To get an honest single number for reporting, the OOF medoid test (0.199) already serves. But let me do one clean confirmation: train medoid on (train+val)=132, test on public_test=16 (held out), to see a realistic number on a truly unseen split with the EXACT final pipeline (M=20 medoid). This gives an independent check beyond the 6-fold OOF. This is worth doing for confidence. Let me run it. It's M=20 models on 132 samples + consensus + eval on 16. ~20*15s=300s. Let me write a quick script reusing final_train logic but with a holdout. Actually, let me just do it inline quickly.

All saved tensors are clean and finite: isotropic `feat_std=[0.186]*3`, sensible `label_mean/std`, 41 state-dict tensors, 376 KB. For an independent confidence check (beyond the 6-fold OOF), let me run the exact final pipeline (20 models + medoid) on a true hold-out: train on train+val (132) and evaluate on the untouched public_test (16).

bash

cd /app && cat > /app/holdout_check.py << 'PYEOF'
import torch, time, lib_train as L
import final_train as FT
from lib_train import DEV
xtr,ytr=L.load_split('train'); xv,yv=L.load_split('val')
Xtr=torch.cat([xtr,xv],0); Ytr=torch.cat([ytr,yv],0)
Xte,Yte=L.load_split('public_test')
Xd=Xtr.to(DEV); Yd=Ytr.to(DEV); Xe=Xte.to(DEV)
fm=Xd.mean((0,1)); fs=Xd.std().expand(3).clone().clamp_min(1e-6)
lm=Yd.mean(0); ls=Yd.std(0).clamp_min(1e-6)
nx=lambda x:(x-fm)/fs; ny=lambda y:(y-lm)/ls; dy=lambda y:y*ls+lm
ytr_n=ny(Yd)
t=time.time(); models=[]
for s in range(FT.M_MODELS):
    models.append(FT.train_one(Xd,ytr_n,nx,FT.CFG,seed=s))
with torch.no_grad():
    trp=torch.stack([m(nx(Xd)) for m in models],0)
    tep=torch.stack([dy(m(nx(Xe))) for m in models],0)
cons=trp.mean(0); med=int(((trp-cons)**2).mean((1,2)).argmin())
per=[L.metrics(tep[i].cpu(),Yte)['rel_l2'] for i in range(len(models))]
import numpy as np
print('HOLDOUT public_test (trained on train+val, 20-seed):')
print('  per-seed rel_l2 mean=%.4f min=%.4f max=%.4f'%(np.mean(per),min(per),max(per)))
print('  ensemble', {k:round(v,4) for k,v in L.metrics(tep.mean(0).cpu(),Yte).items()})
print('  MEDOID(seed %d)'%med, {k:round(v,4) for k,v in L.metrics(tep[med].cpu(),Yte).items()})
print('(%.0fs)'%(time.time()-t))
PYEOF
nohup python holdout_check.py > /app/holdout.log 2>&1 & echo "pid $!"

output

pid 1127

bash

sleep 290; cat /app/holdout.log | grep -v Warning

output

<system>Tool ran without output or errors</system>

bash

sleep 60; cat /app/holdout.log | grep -v Warning; ps aux|grep holdout_check|grep -v grep|wc -l

output

HOLDOUT public_test (trained on train+val, 20-seed):
  per-seed rel_l2 mean=0.2945 min=0.2108 max=0.3396
  ensemble {'rel_l2': 0.286, 'mae': 0.1787, 'maxae': 0.5881}
  MEDOID(seed 3) {'rel_l2': 0.3021, 'mae': 0.1958, 'maxae': 0.4929}
(309s)
0

Hmm, the holdout check on public_test (16 samples) gives much higher rel_l2: per-seed mean 0.2945, medoid 0.3021, ensemble 0.286. This is notably worse than the OOF estimate (0.199). Why? The public_test set of 16 is apparently a "hard" subset (recall the very first baseline: test rel_l2 ~0.29-0.35 while val ~0.18-0.21). So public_test is harder than average. With only 16 samples, rel_l2 is noisy and this particular set is hard. Also, training on only 132 (train+val) vs 148 reduces data slightly. Also notice the medoid here (0.3021) was slightly WORSE than per-seed mean (0.2945) and ensemble (0.286). On this small/hard 16-sample set, medoid selection picked seed 3 which happened to be a bit worse. With only 16 test points, the medoid's advantage (which is about avoiding bad generalizers on average) is noisy. But note maxae for medoid (0.4929) was better than ensemble (0.5881). This is a useful reality check: the held-out grading number could be higher than 0.199 if the hidden set is "hard" like public_test. But the OOF over all 148 (which includes public_test in various folds) gave 0.199 for the medoid. The discrepancy is because: 1. public_test is a particularly hard 16-sample subset. 2. The OOF averages over all 148 including easier val-like samples. The true held-out grading set is unknown difficulty. My best unbiased estimate is the pooled OOF (0.199 medoid), since it's computed over the most data and mimics the grading (rel_l2 over a full set). The 16-sample public_test is too small and happens to be hard. Let me reconsider: should I worry that medoid selection might not robustly help? In the 6-fold OOF (148 samples), medoid (0.199) clearly beat mean-single (0.2075). In the 16-sample holdout, medoid (0.302) was slightly worse than mean-single (0.2945) , but 16 samples is very noisy, and medoid still gave best maxae. The 148-sample OOF is far more reliable. I'll trust it. But this does reveal the final model's performance depends heavily on the hidden set's difficulty. Nothing I can do about that except have the best average model. Now, an important realization: the final shipped model was trained on ALL 148 (including public_test). So it has seen those 16 hard samples during training. That's good , it means the shipped model won't be surprised by public_test-like geometries. The hidden held-out set is separate, but pooling all 148 gives the model the broadest exposure. Let me reconsider whether the medoid choice for the FINAL model (seed 16, trained on 148) is good. I can't evaluate it on truly held-out data. The 6-fold OOF validated the PROCEDURE (medoid on 148-pool gives 0.199). I'll trust that. Actually, let me reconsider one thing: the holdout test showed that on a hard set, the ensemble (0.286) beat the medoid (0.302) by a bit more than on the full OOF. Since I can only ship one model, medoid is my best bet. But given the ensemble consistently edges out the medoid, and the medoid occasionally underperforms on small hard sets, is there any way to ship something closer to the ensemble? One more idea I haven't fully explored: weight averaging of the multiple seeds IN A WAY THAT WORKS. Standard weight averaging across independent inits fails due to permutation symmetry of neurons. BUT there's a technique: align neurons across models (via permutation matching / "Git Re-Basin") then average. That's complex and risky to implement correctly here. Given the gain (0.199 medoid vs 0.197 ensemble) is tiny, not worth it. Another idea: Could I ship the ensemble by folding multiple models into the SAME architecture with wider... no, architecture is fixed width. So medoid is the right call. My final model is good. Let me reconsider the overall expected performance. The honest estimate range: - Pooled 6-fold OOF medoid: rel_l2 0.199, mae 0.122, maxae 0.624. - If hidden set is hard (public_test-like): rel_l2 could be ~0.28-0.30. - If hidden set is easy (val-like): rel_l2 could be ~0.12-0.15. The expected is around 0.20, …[truncated]

Useful reality check. On the 16-sample public_test held out, the pipeline gets rel_L2 ≈ 0.29–0.30, but that specific subset is known to be hard and 16 points is very noisy (my very first baselines showed val ≈ 0.18 vs test ≈ 0.32). The reliable estimate is the **pooled 6-fold OOF over 148 samples (medoid rel_L2 ≈ 0.199)**, which mirrors the grading (rel_L2 over a full set) and averages over far more data. Importantly, the shipped model was trained on all 148 (including those hard public_test geometries), so it has maximal exposure. Let me confirm no stray processes remain and do a final integrity re-check of the saved checkpoint.

bash

cd /app && ps aux | grep -E "python (run|final|distill|medoid|holdout|cv)" | grep -v grep | wc -l | xargs echo "stray training procs:"; echo "---"; ls -la outputs/; echo "--- final strict-load verify ---"; python verify_ckpt.py 2>&1 | grep -v Warning | head -2

output

stray training procs: 0
---
total 385
drwxr-sr-x 2 h2tagent h2tagent   4096 Jun 21 01:26 .
drwxrws--- 1 h2tagent h2tagent   4096 Jun 21 01:27 ..
-rw-r--r-- 1 h2tagent h2tagent 384685 Jun 21 01:26 model.pt
-rw-r--r-- 1 h2tagent h2tagent    602 Jun 21 01:26 train_meta.json
--- final strict-load verify ---
checkpoint keys/shapes OK; strict load OK; [B,2048,3]->[B,1] OK
train        rel_l2=0.0166 mae=0.0105 maxae=0.0538

Everything is finalized: - No stray processes. - /app/outputs/model.pt exists (384 KB) with train_meta.json. - Strict load works, shape contract holds. The solution is complete. Let me write a brief summary of what I did. Let me reconsider if there's anything else worth trying given remaining budget. I have ~89k tokens left. The main potential improvement would be getting closer to ensemble performance. But medoid already captures most of it. Let me reconsider one thing: the medoid selection for the final model picked seed 16. But the OOF validation of medoid used M=9 and consensus on train-part. For the final with M=20, the consensus is more stable. I'm confident. Actually, let me reconsider whether I should verify the medoid procedure once more with a cleaner, larger-scale OOF using the EXACT final settings (iso_norm, M=20) to get the most accurate generalization estimate and confirm medoid ≥ mean-single robustly. The earlier medoid OOF used M=9 and got 0.199. Let me not burn too much time, but a confirmation with M matching would be nice. However, M=20 × 6 folds = 120 trainings × 16s = ~32 min. That's a lot. The M=9 result (0.199) is sufficient validation. I'll skip. Actually, let me reconsider the possibility that the isotropic normalization choice, while marginally better on OOF rel_l2, had slightly worse maxae. Given the task emphasizes "do well on all three," let me double-check that iso vs per-axis doesn't hurt worst-case meaningfully. From run7: iso maxae 0.705 vs per-axis (run6 wd1e-2) 0.661. Hmm, per-axis had better maxae. But the medoid_test (which used iso) got maxae 0.624 , better than both, because medoid selection reduces worst-case too. So with medoid + iso, maxae 0.624 is good. The per-axis medoid might give even better maxae but slightly worse rel_l2. Since rel_l2 is primary and iso medoid gave 0.199 (best rel_l2) with maxae 0.624 (good), iso is the right choice. Actually, wait, let me reconsider. The medoid_test used iso_norm and gave rel_l2 0.199, mae 0.122, maxae 0.624 , all three are the best numbers I've obtained. So iso + medoid dominates. Good, I'm confident in the final choice. I think the solution is complete and well-validated. Let me write the summary. Let me produce a concise final summary of the approach, results, and deliverables. Let me also clean up the scratch files? They don't hurt. The deliverable is /app/outputs/model.pt. I'll leave the scripts as documentation. Actually, let me make sure none of my scratch files overwrote the provided files (candidate_model.py, dataset.py, train.py, configs). I created new files (lib_train.py, cv.py, cv2.py, etc.) and didn't modify the originals except... I didn't touch candidate_model.py. Good. Let me verify candidate_model.py is unchanged.

No stray processes; `/app/outputs/model.pt` (with `train_meta.json`) is present and re-verifies cleanly. Let me confirm I never modified the provided fixed files (especially `candidate_model.py`).

bash

cd /app && echo "candidate_model.py unchanged check (build_model + class present):"; grep -c "class PointNetRegressor" candidate_model.py; grep -c "def build_model" candidate_model.py; echo "--- git/orig intact: head of candidate_model.py ---"; head -5 candidate_model.py; echo "--- meta ---"; cat outputs/train_meta.json

output

candidate_model.py unchanged check (build_model + class present):
1
1
--- git/orig intact: head of candidate_model.py ---
from __future__ import annotations

from typing import Mapping

import torch
--- meta ---
{
  "cfg": {
    "feature_width": 64,
    "global_width": 256,
    "head_width": 128,
    "lr": 0.001,
    "weight_decay": 0.01,
    "epochs": 500,
    "batch_size": 16,
    "warmup_frac": 0.05,
    "loss": "mse",
    "clip": 1.0,
    "yreflect": true,
    "iso_norm": true
  },
  "m_models": 20,
  "medoid_seed": 16,
  "feat_mean": [
    -0.011780548840761185,
    0.007426036521792412,
    -0.07198601216077805
  ],
  "feat_std": [
    0.1860007345676422,
    0.1860007345676422,
    0.1860007345676422
  ],
  "label_mean": [
    0.7487966418266296
  ],
  "label_std": [
    0.38614240288734436
  ]
}

The provided architecture file is untouched, and the deliverable is complete and validated. Here's a summary of what I did and why. ## Approach **Problem framing.** This is small-data scalar regression: only 100 train (+32 val, +16 public_test) point clouds of `[2048,3]`, predicting bracket mass, through a *fixed* PointNet (shared MLP → max-pool → MLP head). I can only control normalization, the training procedure, and the single shipped weights , no architecture, inference-time TTA, or ensembling. **Key findings from data analysis.** - The y-axis always spans the full unit cube (extent ≈ 1.0 for every sample), so scale is consistently encoded and mass is a genuine deterministic function of the normalized shape (not scale-noise-limited). - Chamfer analysis showed brackets are approximately symmetric under a **y-axis mirror** (and mirrors are exactly mass-preserving and stay in-distribution) , this became my most valuable augmentation. **Establishing the ceiling.** I built a pooled **6-fold out-of-fold CV over all 148 labeled samples** (mirrors the grading's "rel-L2 over the full set" and is far less noisy than the tiny val/test). Kernel-ridge and k-NN on rich hand-crafted shape descriptors did *worse* (≥0.236) than the PointNet (~0.224), confirming the architecture isn't the bottleneck , data quantity is. **Recipe selection (by OOF).** Systematic sweeps gave: y-reflection augmentation (big win), weight decay = 1e-2, 500 epochs, cosine LR (1e-3) with warmup, MSE, and isotropic coordinate normalization. EMA/SWA and distillation gave no gain; higher/lower wd, jitter, longer epochs, and larger batch were worse. **The main trick , consensus medoid.** Since I can ship only one model but an ensemble generalizes ~0.01 better, I validated (via OOF) that selecting the **medoid** , the model whose predictions on the training inputs are closest to the multi-seed mean , recovers almost all of the ensemble benefit in a single checkpoint: OOF **rel-L2 0.199, MAE 0.122, max-abs-err 0.624**, beating the mean single seed (0.2075) on all three metrics simultaneously. **Final model.** Trained 20 seeds on all 148 pooled samples with the chosen recipe, selected the medoid (seed 16), and saved `/app/outputs/model.pt` with `state_dict`, `feat_mean`, `feat_std`, `label_mean`, `label_std`. ## Verification - Required keys present; `feat_mean/feat_std` length-3, `label_mean/label_std` length-1; all tensors finite (float32). - Loads **strict=True** into the untouched `PointNetRegressor`; `[B,2048,3] → [B,1]` confirmed. - An independent hold-out (train+val → public_test) corroborated the pipeline; that 16-sample subset is intrinsically hard/noisy, so the 148-sample OOF (~0.199) is my reliable generalization estimate. One honest caveat worth flagging: the true held-out score will track the hidden set's difficulty. My estimate (~0.20 rel-L2) comes from pooled CV, which is the most trustworthy proxy available, and the shipped model was trained on all provided labeled data (including the hard public_test geometries) for maximum exposure.

Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward

tests/test.sh

# SyncVals normalized verifier summary
# task: simjeb-bracket-fea-mass-prediction-real
# attempt: 5  status: completed  raw_score: 0.5
# reward rule: continuous: score 0.5 >= 0.5
# => reward = 1.0  (PASS)
#
# Note: these tasks score via an external hidden grader; no pytest/verifier
# block is embedded in the agent transcript. The block below is the last
# sizable tool-output captured in the run (supporting evidence).
----------------------------------------------------------------------
candidate_model.py unchanged check (build_model + class present):
1
1
--- git/orig intact: head of candidate_model.py ---
from __future__ import annotations

from typing import Mapping

import torch
--- meta ---
{
  "cfg": {
    "feature_width": 64,
    "global_width": 256,
    "head_width": 128,
    "lr": 0.001,
    "weight_decay": 0.01,
    "epochs": 500,
    "batch_size": 16,
    "warmup_frac": 0.05,
    "loss": "mse",
    "clip": 1.0,
    "yreflect": true,
    "iso_norm": true
  },
  "m_models": 20,
  "medoid_seed": 16,
  "feat_mean": [
    -0.011780548840761185,
    0.007426036521792412,
    -0.07198601216077805
  ],
  "feat_std": [
    0.1860007345676422,
    0.1860007345676422,
    0.1860007345676422
  ],
  "label_mean": [
    0.7487966418266296
  ],
  "label_std": [
    0.38614240288734436
  ]
}

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_5c5ed690cfd0411b. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_5c5ed690cfd0411b · verifier authoritative; classifier explanatory.