SyncVals · Trajectory

simjeb-bracket-fea-mass-prediction-real

claude-code claude-opus-4-8 ✗ failed GOOD_FAILURE ↑ View task

Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.

Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.

Classification , post-hoc; cannot change the reward

GOOD_FAILUREHonest miss, the agent ran correctly but couldn't solve it. Expected for a hard task; the task is sound.

SubtypeInsufficient Exploration / Incomplete Understanding

EvidenceAgent's model achieves raw score 0.42 (primary metric mass_rel_l2_score), below the 0.50 threshold needed for pass. Test output confirms: 'FINAL DELIVERABLE OK' but 'reward=0.0 (FAIL) score 0.42 < 0.5'. The deliverable format is correct (all required keys present, loads into PointNetRegressor, maps [B,2048,3]->[B,1]). The golden solution (solve.sh) trains for 2200 epochs with EMA, data augmentation, and combined loss, proving strong performance is achievable with proper hyperparameter tuning.

Root causeAgent trained the model using insufficient hyperparameter optimization, likely just running the starter train.py code (1 epoch default) without increasing epochs, adding augmentation, or tuning learning rates. This is a small-data regression problem (100 training samples) requiring substantial training effort and exploration, which the agent did not invest.

RecommendationN/A - task is fine. This is a hard machine learning problem that requires agents to explore hyperparameters, training duration, regularization, and augmentation strategies. The instruction is clear, the verifier is fair, the data contract is well-specified, and the oracle proves strong solutions exist. Agent failure is expected for this complexity level."

Trajectory

Tool-by-tool agent trajectory

280 tool calls · 3 tool types · 280 steps

You are given point-cloud samples of 3D jet-engine bracket designs along with each bracket's FEA-derived mass (kg). Each point cloud is an unordered set of 2048 surface points in R^3; coordinates are pre-centered and rescaled so the axis-aligned bounding box of each bracket fits inside the unit cube. The mapping from geometry to mass is not given to you - you have to learn it from the data. Train a model that takes a `[2048, 3]` point cloud and predicts the scalar mass value. Your model is then applied to a held-out set of bracket geometries and the predictions are compared against the ground-truth FEA-simulated mass values. ## Files - `/app/data/{train,val,public_test}/samples.pt` - each file is a dict with keys `features` (shape `[N, 2048, 3]`) and `labels` (shape `[N, 1]`), both float16 on disk. Load with `torch.load(path, weights_only=True)`. - `/app/task_config.json` - architecture sizes and training defaults. - `/app/feature_schema.json` - input / label spec. - `/app/candidate_model.py` - the fixed PointNet-style architecture. Keep it as is; your saved checkpoint has to load back into this same architecture. - `/app/dataset.py` - a minimal loader for `samples.pt`. - `/app/train.py` - a small starter training loop you can extend or replace. ## What you must produce Write your trained checkpoint to `/app/outputs/model.pt` as a dict with the following required keys: - `state_dict` - your model's `state_dict()` mapping. - `feat_mean`, `feat_std` - length-3 tensors used to standardize the input coordinates before the forward pass. The same normalization is applied at inference time, so these are required. - `label_mean`, `label_std` - length-1 tensors used to standardize the target during training and de-standardize the model output at inference. The model's output is mapped back to physical units using these, so they are required too. The model must consume a `[B, 2048, 3]` tensor and produce a `[B, 1]` output. ## What you are optimizing Your predictions are measured against the true FEA-simulated mass labels with three error numbers, all lower is better: the relative L2 over the full set (the primary one), the mean absolute error, and the worst-case per-sample absolute error. A strong model needs to do well on all three, so do not trade one off against the others.

Let me start by exploring the environment to understand what I'm working with.

I'll start by exploring the environment to understand the provided files and data.

bash

cd /app && ls -la && echo "---CONFIG---" && cat task_config.json && echo "---SCHEMA---" && cat feature_schema.json

output

total 14
drwxrws--- 3 h2tagent h2tagent  138 Jun 20 18:13 .
drwxr-xr-x 1 root     root     4096 Jun 20 20:56 ..
-rw-rw---- 1 h2tagent h2tagent 1639 Jun 20 18:10 candidate_model.py
drwxrws--- 5 h2tagent h2tagent   58 Jun 20 18:10 data
-rw-rw---- 1 h2tagent h2tagent 1183 Jun 20 18:10 dataset.py
-rw-rw---- 1 h2tagent h2tagent  499 Jun 20 18:10 feature_schema.json
-rw-rw---- 1 h2tagent h2tagent  581 Jun 20 18:10 task_config.json
-rw-rw---- 1 h2tagent h2tagent 3117 Jun 20 18:10 train.py
---CONFIG---
{
  "task_id": "simjeb-bracket-fea-mass-prediction-real",
  "dataset_name": "simjeb-bracket-fea-mass-pilot",
  "n_points": 2048,
  "point_dim": 3,
  "label_dim": 1,
  "label_per_point": false,
  "seed": 20260518,
  "model": {
    "feature_width": 64,
    "global_width": 256,
    "head_width": 128
  },
  "training": {
    "epochs": 1,
    "batch_size": 16,
    "lr": 0.001,
    "weight_decay": 0.0001,
    "loss": "MSE"
  },
  "evaluation": {
    "primary_metric": "mass_rel_l2_score",
    "secondary_metrics": [
      "mass_mae_score",
      "mass_max_abs_err_score"
    ]
  }
}
---SCHEMA---
{
  "version": "simjeb-bracket-fea-mass-pilot-v1",
  "input": {
    "type": "point_cloud",
    "n_points": 2048,
    "dim": 3,
    "coord_range": [
      -0.5,
      0.5
    ],
    "description": "Unordered surface point sample of a jet-engine bracket; each bracket mesh is centered at its bounding-box centroid and rescaled so the axis-aligned bounding box fits inside the unit cube."
  },
  "label": {
    "type": "scalar",
    "dim": 1,
    "description": "FEA-derived bracket mass in kg."
  }
}

Let me look at the model architecture, dataset loader, and training loop.

Let me look at the model architecture, dataset, and training code.

bash

cd /app && echo "===candidate_model.py===" && cat candidate_model.py && echo "===dataset.py===" && cat dataset.py && echo "===train.py===" && cat train.py

output

===candidate_model.py===
from __future__ import annotations

from typing import Mapping

import torch
import torch.nn as nn


class SharedPointMLP(nn.Module):
    def __init__(self, in_dim: int, widths: tuple[int, ...]) -> None:
        super().__init__()
        layers: list[nn.Module] = []
        prev = in_dim
        for w in widths:
            layers.append(nn.Conv1d(prev, w, kernel_size=1))
            layers.append(nn.BatchNorm1d(w))
            layers.append(nn.GELU())
            prev = w
        self.net = nn.Sequential(*layers)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.net(x)


class PointNetRegressor(nn.Module):
    def __init__(self, feature_width: int, global_width: int, head_width: int) -> None:
        super().__init__()
        self.feature = SharedPointMLP(3, (feature_width, feature_width, feature_width))
        self.lift = SharedPointMLP(feature_width, (feature_width * 2, global_width))
        self.head = nn.Sequential(
            nn.Linear(global_width, head_width),
            nn.GELU(),
            nn.Linear(head_width, head_width // 2),
            nn.GELU(),
            nn.Linear(head_width // 2, 1),
        )

    def forward(self, points: torch.Tensor) -> torch.Tensor:
        x = points.transpose(1, 2).contiguous()
        x = self.feature(x)
        x = self.lift(x)
        x = x.max(dim=2).values
        return self.head(x)


def build_model(config: Mapping) -> PointNetRegressor:
    m = config["model"]
    return PointNetRegressor(
        feature_width=int(m["feature_width"]),
        global_width=int(m["global_width"]),
        head_width=int(m["head_width"]),
    )
===dataset.py===
from __future__ import annotations

from pathlib import Path
from typing import Iterable

import torch
from torch.utils.data import Dataset


class SampleStore(Dataset):
    def __init__(self, split_dir: str | Path) -> None:
        self.dir = Path(split_dir)
        payload = torch.load(self.dir / "samples.pt", map_location="cpu", weights_only=True)
        self.features: torch.Tensor = payload["features"].float()
        self.labels: torch.Tensor = payload["labels"].float()
        if self.features.shape[0] != self.labels.shape[0]:
            raise ValueError("features and labels must have the same number of rows")

    def __len__(self) -> int:
        return self.features.shape[0]

    def __getitem__(self, idx: int) -> tuple[torch.Tensor, torch.Tensor]:
        return self.features[idx], self.labels[idx]


def iter_batches(
    x: torch.Tensor,
    y: torch.Tensor,
    batch_size: int,
    shuffle: bool = True,
) -> Iterable[tuple[torch.Tensor, torch.Tensor]]:
    n = x.shape[0]
    idx = torch.randperm(n) if shuffle else torch.arange(n)
    for start in range(0, n, batch_size):
        chunk = idx[start : start + batch_size]
        yield x[chunk], y[chunk]
===train.py===
from __future__ import annotations

import json
import os
import sys
from pathlib import Path

import torch
import torch.nn.functional as F

APP_DIR = Path(os.environ.get("APP_DIR", "/app"))
sys.path.insert(0, str(APP_DIR))

from candidate_model import build_model  # noqa: E402
from dataset import SampleStore, iter_batches  # noqa: E402


def _device() -> torch.device:
    if torch.cuda.is_available():
        return torch.device("cuda")
    if torch.backends.mps.is_available():
        return torch.device("mps")
    return torch.device("cpu")


def _load_config() -> dict:
    with (APP_DIR / "task_config.json").open() as fh:
        return json.load(fh)


def main() -> None:
    cfg = _load_config()
    device = _device()
    torch.manual_seed(int(cfg["seed"]))

    train = SampleStore(APP_DIR / "data" / "train")
    val = SampleStore(APP_DIR / "data" / "val")

    feat_mean = train.features.mean(dim=(0, 1), keepdim=True)
    feat_std = train.features.std(dim=(0, 1), keepdim=True).clamp_min(1e-6)
    label_mean = train.labels.mean(dim=(0,), keepdim=True)
    label_std = train.labels.std(dim=(0,), keepdi
…[truncated]

Now let me examine the data. Let me check the sizes, distributions, etc.

Now let me examine the data itself , sizes, shapes, and label distributions.

bash

cd /app && ls -la data && echo "---" && for d in train val public_test; do echo "== $d =="; ls -la data/$d; done

output

total 3
drwxrws--- 5 h2tagent h2tagent  58 Jun 20 18:10 .
drwxrws--- 3 h2tagent h2tagent 138 Jun 20 18:13 ..
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 public_test
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 train
drwxrws--- 2 h2tagent h2tagent  55 Jun 20 18:10 val
---
== train ==
total 1210
drwxrws--- 2 h2tagent h2tagent      55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent      58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent    6483 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 1230821 Jun 20 18:10 samples.pt
== val ==
total 389
drwxrws--- 2 h2tagent h2tagent     55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent     58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent   2011 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 395109 Jun 20 18:10 samples.pt
== public_test ==
total 197
drwxrws--- 2 h2tagent h2tagent     55 Jun 20 18:10 .
drwxrws--- 5 h2tagent h2tagent     58 Jun 20 18:10 ..
-rw-rw---- 1 h2tagent h2tagent   1134 Jun 20 18:10 manifest.jsonl
-rw-rw---- 1 h2tagent h2tagent 198437 Jun 20 18:10 samples.pt

bash

cd /app && python3 -c "
import torch, json
for split in ['train','val','public_test']:
    p = torch.load(f'data/{split}/samples.pt', weights_only=True)
    f = p['features']; l = p['labels']
    print(f'== {split} ==')
    print('features', tuple(f.shape), f.dtype, 'labels', tuple(l.shape), l.dtype)
    lf = l.float()
    print('label min/max/mean/std', float(lf.min()), float(lf.max()), float(lf.mean()), float(lf.std()))
    ff = f.float()
    print('feat min/max', float(ff.min()), float(ff.max()))
    print('feat per-axis min', ff.amin(dim=(0,1)).tolist())
    print('feat per-axis max', ff.amax(dim=(0,1)).tolist())
    print('feat per-axis mean', ff.mean(dim=(0,1)).tolist())
    print('feat per-axis std', ff.std(dim=(0,1)).tolist())
"

output

== train ==
features (100, 2048, 3) torch.float16 labels (100, 1) torch.float16
label min/max/mean/std 0.207763671875 2.21875 0.7475781440734863 0.3894648551940918
feat min/max -0.5 0.5
feat per-axis min [-0.320556640625, -0.5, -0.2279052734375]
feat per-axis max [0.322998046875, 0.5, 0.22705078125]
feat per-axis mean [-0.013504491187632084, 0.007462748326361179, -0.0709712877869606]
feat per-axis std [0.14211857318878174, 0.2701786458492279, 0.0871192067861557]
== val ==
features (32, 2048, 3) torch.float16 labels (32, 1) torch.float16
label min/max/mean/std 0.293701171875 2.072265625 0.7621994018554688 0.41264674067497253
feat min/max -0.5 0.5
feat per-axis min [-0.3095703125, -0.5, -0.2279052734375]
feat per-axis max [0.31005859375, 0.5, 0.2275390625]
feat per-axis mean [-0.005372888408601284, 0.005196526646614075, -0.07424016296863556]
feat per-axis std [0.1418035924434662, 0.2656756639480591, 0.08837021142244339]
== public_test ==
features (16, 2048, 3) torch.float16 labels (16, 1) torch.float16
label min/max/mean/std 0.1983642578125 1.44921875 0.7296066284179688 0.327779084444046
feat min/max -0.5 0.5
feat per-axis min [-0.309326171875, -0.5, -0.1865234375]
feat per-axis max [0.309326171875, 0.5, 0.1864013671875]
feat per-axis mean [-0.013821225613355637, 0.011655600741505623, -0.07381976395845413]
feat per-axis std [0.14321814477443695, 0.27090150117874146, 0.08569176495075226]

Key observations: - Train: 100 samples, Val: 32 samples, public_test: 16 samples. Very small dataset! - Point clouds: [2048, 3], coordinates in [-0.5, 0.5]. - Labels: mass in kg, range roughly 0.2 to 2.2, mean ~0.75. This is a small-data regression problem. The architecture is fixed (PointNet-style). I need to train a model that maps [2048,3] -> [1]. Let me look at the manifests to understand the data better.

This is a small-data regression problem: only 100 training samples. Let me look at the manifests for additional context.

bash

cd /app && echo "=== train manifest (first 3) ===" && head -n 3 data/train/manifest.jsonl && echo "=== val manifest (first 2) ===" && head -n 2 data/val/manifest.jsonl && echo "=== public_test manifest (first 2) ===" && head -n 2 data/public_test/manifest.jsonl && echo "=== counts ===" && wc -l data/*/manifest.jsonl

output

=== train manifest (first 3) ===
{"id": "train_0000", "source_bracket_id": 214, "n_points": 2048}
{"id": "train_0001", "source_bracket_id": 538, "n_points": 2048}
{"id": "train_0002", "source_bracket_id": 546, "n_points": 2048}
=== val manifest (first 2) ===
{"id": "val_0000", "source_bracket_id": 418, "n_points": 2048}
{"id": "val_0001", "source_bracket_id": 441, "n_points": 2048}
=== public_test manifest (first 2) ===
{"id": "public_test_0000", "source_bracket_id": 461, "n_points": 2048}
{"id": "public_test_0001", "source_bracket_id": 440, "n_points": 2048}
=== counts ===
  16 data/public_test/manifest.jsonl
 100 data/train/manifest.jsonl
  32 data/val/manifest.jsonl
 148 total

bash

cd /app && python3 -c "
import torch
# Check for duplicate source brackets across splits
import json
ids = {}
for split in ['train','val','public_test']:
    s = set()
    with open(f'data/{split}/manifest.jsonl') as f:
        for line in f:
            d = json.loads(line)
            s.add(d['source_bracket_id'])
    ids[split] = s
    print(split, 'n_unique_brackets', len(s))
print('train&val', ids['train'] & ids['val'])
print('train&test', ids['train'] & ids['public_test'])
print('val&test', ids['val'] & ids['public_test'])
"

output

train n_unique_brackets 100
val n_unique_brackets 32
public_test n_unique_brackets 16
train&val set()
train&test set()
val&test set()

So there are no overlapping brackets across splits , each split has unique source brackets. This is a true generalization task. The dataset is small (100 train). The held-out test is different from public_test presumably. Let me check the environment: is there a GPU? What PyTorch version?

No bracket overlap across splits , this is a genuine generalization task. Let me check the compute environment and PyTorch version.

bash

cd /app && python3 -c "
import torch, platform
print('torch', torch.__version__)
print('cuda available', torch.cuda.is_available())
print('python', platform.python_version())
import os
print('cpus', os.cpu_count())
" && free -h 2>/dev/null | head -2

output

torch 2.5.1+cu124
cuda available True
python 3.11.10
cpus 3
               total        used        free      shared  buff/cache   available
Mem:            30Gi        64Mi        29Gi          0B        29Mi        29Gi

bash

cd /app && nvidia-smi --query-gpu=name,memory.total,memory.free --format=csv 2>/dev/null || echo "nvidia-smi not available"

output

name, memory.total [MiB], memory.free [MiB]
NVIDIA H100 80GB HBM3, 81559 MiB, 80015 MiB

I have an H100 GPU. The dataset is small (100 train, 2048 points each). This is very manageable. Let me think about the problem carefully. **The task:** Predict scalar mass (kg) from a [2048,3] point cloud. Fixed architecture: PointNet-style regressor. I must save a checkpoint that loads into `PointNetRegressor` with the config sizes (feature_width=64, global_width=256, head_width=128). **Key insight about the physics:** Mass = density × volume. The point clouds are surface samples, pre-centered and rescaled so the bounding box fits in the unit cube. Wait , important subtlety. The coordinates are rescaled so each bracket's bounding box fits in the unit cube. This means the *scale* information is partially lost , if all brackets are normalized to fit in the unit cube, then the absolute size is gone. But the mass varies from 0.2 to 2.2 kg. So the mass must be inferred from the *shape* (aspect ratio, how much material/volume the shape occupies within its bounding box, wall thickness, etc.). Actually wait, let me reconsider. The rescaling: "coordinates are pre-centered and rescaled so the axis-aligned bounding box of each bracket fits inside the unit cube." Does each bracket get its OWN scale factor, or is there a single global scale? If each bracket is independently rescaled to fit the unit cube, then absolute scale is lost. But let me check: the per-axis max values are around 0.32, 0.5, 0.22 for train. The y-axis spans the full [-0.5, 0.5] = range 1.0. So at least one axis hits the full unit cube range. Hmm, but it seems like for ALL splits, the y-axis (axis 1) hits exactly -0.5 to 0.5. And x is around ±0.32, z around ±0.22. This suggests a consistent aspect ratio across brackets, OR a global scaling where the largest dimension across all brackets is along y. Let me check per-sample bounding boxes to understand the rescaling better. If each sample independently fills the unit cube, then for each sample, max over axes of (max-min) should be 1.0. Let me verify.

I have an H100 GPU and a small dataset. Let me understand the rescaling carefully , whether each bracket is independently scaled (losing absolute size) or globally scaled. This is physically crucial since mass relates to volume.

bash

cd /app && python3 -c "
import torch
p = torch.load('data/train/samples.pt', weights_only=True)
f = p['features'].float(); l = p['labels'].float()
# per-sample bounding box extents
mn = f.amin(dim=1)  # [N,3]
mx = f.amax(dim=1)  # [N,3]
ext = mx - mn       # [N,3] extent per axis
print('per-sample extent stats:')
print('  axis max extent per sample (should be ~1 if each fills unit cube):')
maxext = ext.max(dim=1).values
print('   min', float(maxext.min()), 'max', float(maxext.max()), 'mean', float(maxext.mean()))
print('  which axis is largest (count):', torch.bincount(ext.argmax(dim=1), minlength=3).tolist())
print('extent axis0: min/max', float(ext[:,0].min()), float(ext[:,0].max()))
print('extent axis1: min/max', float(ext[:,1].min()), float(ext[:,1].max()))
print('extent axis2: min/max', float(ext[:,2].min()), float(ext[:,2].max()))
"

output

per-sample extent stats:
  axis max extent per sample (should be ~1 if each fills unit cube):
   min 0.9912109375 max 1.0 mean 0.9991015791893005
  which axis is largest (count): [0, 100, 0]
extent axis0: min/max 0.502197265625 0.6435546875
extent axis1: min/max 0.9912109375 1.0
extent axis2: min/max 0.2757568359375 0.4549560546875

So each sample's largest extent is ~1.0 (fills the unit cube), and it's always along axis 1 (y). This confirms each bracket is independently rescaled so its bounding box fits in the unit cube, with the longest dimension along y always being ~1.0. This means absolute scale information is LOST. The model must predict mass purely from the normalized shape. The mass depends on: 1. The aspect ratio (extent in x and z relative to y). 2. The "fill fraction" / geometry , how much material is in the shape. 3. The original absolute scale is gone, BUT it's correlated with shape in the dataset (these are real bracket designs with physical constraints). Wait, but if the absolute scale is lost, then two brackets with identical shape but different sizes would have different masses but identical point clouds → impossible to distinguish. However, since these are real designs normalized, the mass correlates with the normalized shape + aspect ratio. The relationship is learnable because the dataset presumably has consistent physical scale relationships... Actually, let me reconsider. The SimJEB dataset is a collection of jet engine bracket designs. They all share the same mounting interface (4 bolt holes + a load application point) with FIXED physical dimensions. So the absolute scale IS implicitly fixed by the common mounting interface! When normalized to the unit cube, brackets with more material (bulkier) will have... hmm, actually the normalization would change based on the bounding box. Let me think about this differently. The key is: there's a learnable mapping from normalized shape to mass. The aspect ratio is preserved (x extent and z extent relative to the fixed y=1.0). And the distribution of points reveals the shape/volume. The relative L2 is the primary metric: sqrt(sum((pred-true)^2) / sum(true^2)). MAE and max abs error are secondary. Let me think about what features predict mass well: - Volume-like features: The surface point distribution can indicate volume. - The aspect ratio (x and z extents). Let me do a quick exploratory analysis: can simple geometric features predict mass well? This gives me a baseline and sanity check. Let me compute features like bounding box extents, point cloud spread (std per axis), etc., and fit a linear/GBM model mentally. Actually let me just explore correlations between simple features and mass.

Each bracket is independently rescaled (longest extent ≈1.0 always along y-axis). So absolute scale is mostly normalized away, and mass must be inferred from normalized shape + aspect ratio. Let me check how predictive simple geometric features are, to establish a baseline understanding.

bash

cd /app && python3 -c "
import torch, numpy as np
def feats(f):
    mn=f.amin(dim=1); mx=f.amax(dim=1); ext=mx-mn
    std=f.std(dim=1)
    mean=f.mean(dim=1)
    # pairwise distance-based 'size' proxies
    feats = torch.cat([ext, std, mean.abs()], dim=1)
    return feats
tr=torch.load('data/train/samples.pt',weights_only=True)
ftr=tr['features'].float(); ltr=tr['labels'].float().squeeze(1)
X=feats(ftr).numpy(); y=ltr.numpy()
names=['ext_x','ext_y','ext_z','std_x','std_y','std_z','|mean_x|','|mean_y|','|mean_z|']
print('Correlation of features with mass:')
for i,n in enumerate(names):
    c=np.corrcoef(X[:,i],y)[0,1]
    print(f'  {n:10s}: {c:+.3f}')
# product of extents = bbox volume; since y~1, ~ext_x*ext_z
bboxvol = X[:,0]*X[:,1]*X[:,2]
print('  bbox_vol  :', f'{np.corrcoef(bboxvol,y)[0,1]:+.3f}')
print('  ext_x*ext_z:', f'{np.corrcoef(X[:,0]*X[:,2],y)[0,1]:+.3f}')
"

output

Correlation of features with mass:
  ext_x     : +0.234
  ext_y     : +0.143
  ext_z     : +0.177
  std_x     : +0.635
  std_y     : +0.149
  std_z     : +0.250
  |mean_x|  : -0.401
  |mean_y|  : +0.102
  |mean_z|  : -0.183
  bbox_vol  : +0.253
  ext_x*ext_z: +0.249

Some correlation exists. std_x has 0.635 correlation. Let me fit a quick gradient boosting / random forest with richer features to see how much of the mass variance can be explained by hand-crafted features. This tells me the ceiling of a "shape statistics" approach and gives context for what the PointNet should achieve. Actually, let me be more systematic. Let me build a richer feature set and use a simple regressor with cross-validation. But sklearn may not be installed. Let me check. Actually, the real goal is training the PointNet. But this exploration helps me understand difficulty. Let me check if sklearn is available. Actually, let me focus. The task requires using the fixed PointNet architecture. Let me think about how to get the best performance out of it. **Challenges:** 1. Very small training set (100 samples). Overfitting is a huge risk. 2. BatchNorm with batch_size=16 and only 100 samples. 3. Need good generalization. **Strategy:** 1. Data augmentation: Since point clouds are unordered sets, I can augment with: - Random point subsampling/shuffling (order invariant anyway due to max pooling, but resampling helps). - Small jitter (Gaussian noise on points). - Random point dropout. - **Important consideration about symmetry augmentation:** Can I flip/rotate? The brackets have a specific orientation. The mass is rotation/reflection invariant (mass doesn't change if you flip the shape). BUT the normalization is axis-aligned bounding box based. If I rotate arbitrarily, the bounding box changes and the point cloud would no longer be "normalized so bounding box fits unit cube" consistently , the test data follows the specific normalization. So arbitrary rotations would take inputs off-distribution. - However, **reflections** along axes preserve the axis-aligned bounding box! Reflecting x -> -x keeps the bounding box the same (just mirrored). Mass is reflection-invariant. So reflection augmentation along each axis is safe IF the bracket designs have that symmetry in distribution. Let me check: jet engine brackets in SimJEB , are they symmetric? The mounting interface typically has symmetry. Let me check the data distribution , is it roughly symmetric about x=0, z=0? Let me verify the symmetry of the point distributions to decide which reflections are valid augmentations. Looking at means: mean_x ~ -0.0135, mean_z ~ -0.071. Not perfectly centered but close-ish for x. The centering is by bounding-box centroid, so mean isn't necessarily 0. Let me think about which reflections keep the data on-distribution. The safest: reflection that maps the distribution to itself. Let me examine by checking if the point cloud is approximately symmetric under x -> -x. Actually, a cleaner approach: reflection along an axis is a valid augmentation for a regression target that is reflection-invariant, provided the *population* of shapes is symmetric under that reflection (so augmented samples look like plausible other samples). Even if not perfectly, it's a reasonable regularizer. But to be safe and principled, let me check the actual symmetry. Let me examine the geometry more , maybe visualize the bracket structure by checking point density. Let me step back and be pragmatic given limited tokens. Key decisions: 1. Use the fixed architecture. 2. Normalize inputs (feat_mean/std) and labels (label_mean/std). 3. Train with heavy augmentation + regularization to combat overfitting. 4. Use an ensemble? The checkpoint must load into a single PointNetRegressor. So I can only save ONE model's state_dict. No ensembling at inference (unless I average weights, which is valid , weight averaging / SWA). Wait , can I ensemble? The spec says the checkpoint's state_dict must load into PointNetRegressor. So it's a single model. I could use SWA (stochastic weight averaging) which produces a single set of weights. That's a good regularizer and fully compatible. 5. Loss: The primary metric is relative L2 over the full set = sqrt(sum (p …[truncated]

Some geometric features correlate with mass (e.g. std_x at 0.64), confirming the mapping is learnable from normalized shape. Let me define the exact evaluation metrics and compute simple baselines (constant predictor) to anchor my expectations.

bash

cd /app && python3 -c "
import torch, numpy as np
def metrics(pred, true):
    pred=np.asarray(pred,dtype=np.float64); true=np.asarray(true,dtype=np.float64)
    rel_l2 = np.sqrt(np.sum((pred-true)**2))/np.sqrt(np.sum(true**2))
    mae = np.mean(np.abs(pred-true))
    maxe = np.max(np.abs(pred-true))
    return rel_l2, mae, maxe
tr=torch.load('data/train/samples.pt',weights_only=True)['labels'].float().squeeze(1).numpy()
va=torch.load('data/val/samples.pt',weights_only=True)['labels'].float().squeeze(1).numpy()
te=torch.load('data/public_test/samples.pt',weights_only=True)['labels'].float().squeeze(1).numpy()
print('Constant(train mean) predictor:')
for name,arr in [('val',va),('test',te)]:
    c=np.full_like(arr, tr.mean())
    print(f'  {name}: rel_l2={metrics(c,arr)[0]:.4f} mae={metrics(c,arr)[1]:.4f} maxe={metrics(c,arr)[2]:.4f}')
print('train mean',tr.mean(),'std',tr.std())
"

output

Constant(train mean) predictor:
  val: rel_l2=0.4706 mae=0.3024 maxe=1.3247
  test: rel_l2=0.3995 mae=0.2607 maxe=0.7016
train mean 0.74757814 std 0.38751265

So a constant predictor gives rel_l2 ~0.47 on val, ~0.40 on test. I need to do much better. Let me establish what a strong hand-crafted feature baseline can achieve to set a target, then build the PointNet. Let me check if sklearn is available for a quick feature-based baseline (GBM/RF) to understand the achievable ceiling.

A constant predictor gives rel_L2 ≈ 0.40–0.47. Let me check whether sklearn is available and what a rich hand-crafted-feature model can achieve , this tells me the achievable ceiling before committing to the neural approach.

bash

cd /app && python3 -c "import sklearn; print('sklearn', sklearn.__version__)" 2>&1 | head -1; python3 -c "import scipy; print('scipy', scipy.__version__)" 2>&1 | head -1

output

Traceback (most recent call last):
Traceback (most recent call last):

No sklearn or scipy. Let me do a quick feature-based baseline with a numpy ridge regression to estimate the ceiling. Let me build richer features and do ridge regression + evaluate on val. Actually, let me build a quick feature-based analysis using numpy linear regression (ridge). I'll construct a bunch of shape descriptors. Features to try: - bbox extents (x,z since y~1) - std per axis - mean abs per axis - covariance / PCA eigenvalues of the point cloud - occupancy on a coarse voxel grid (surface coverage) - moments Actually the most physically meaningful for mass: the point cloud samples the surface. The volume enclosed relates to mass. Estimating volume from surface points is hard, but voxel occupancy of a coarse grid gives a proxy of the "bounding shape" volume. Surface area also relates. Let me just try ridge regression with extents+std+PCA eigenvalues+voxel occupancy count and see val performance. This gives me a sense of the ceiling.

Let me build a quick hand-crafted-feature ridge regression baseline with NumPy (voxel occupancy, PCA eigenvalues, moments) to estimate the achievable ceiling.

bash

cd /app && python3 -c "
import torch, numpy as np
def metrics(pred, true):
    pred=np.asarray(pred,np.float64); true=np.asarray(true,np.float64)
    return (np.sqrt(np.sum((pred-true)**2))/np.sqrt(np.sum(true**2)),
            np.mean(np.abs(pred-true)), np.max(np.abs(pred-true)))
def make_feats(f):
    N=f.shape[0]; out=[]
    mn=f.amin(1); mx=f.amax(1); ext=mx-mn
    std=f.std(1); mean=f.mean(1)
    out+= [ext,std,mean]
    # PCA eigenvalues of centered cloud
    fc=f-f.mean(1,keepdim=True)
    cov=torch.einsum('npi,npj->nij',fc,fc)/f.shape[1]
    evals=torch.linalg.eigvalsh(cov)  # [N,3] ascending
    out.append(evals)
    # voxel occupancy at resolution R in [-0.5,0.5]
    for R in (8,16):
        idx=((f+0.5).clamp(0,0.999999)*R).long()  # [N,2048,3]
        lin=idx[...,0]*R*R+idx[...,1]*R+idx[...,2]
        occ=torch.zeros(N, R*R*R)
        for n in range(N):
            occ[n, torch.unique(lin[n])]=1.0
        out.append(occ.sum(1,keepdim=True)/(R*R*R))  # occupancy fraction
    return torch.cat(out,1).numpy()
tr=torch.load('data/train/samples.pt',weights_only=True)
va=torch.load('data/val/samples.pt',weights_only=True)
Xtr=make_feats(tr['features'].float()); ytr=tr['labels'].float().squeeze(1).numpy()
Xva=make_feats(va['features'].float()); yva=va['labels'].float().squeeze(1).numpy()
mu=Xtr.mean(0); sd=Xtr.std(0)+1e-8
Xtr=(Xtr-mu)/sd; Xva=(Xva-mu)/sd
Xtr=np.c_[Xtr,np.ones(len(Xtr))]; Xva=np.c_[Xva,np.ones(len(Xva))]
for lam in [0.1,1.0,10.0]:
    A=Xtr.T@Xtr+lam*np.eye(Xtr.shape[1]); b=Xtr.T@ytr
    w=np.linalg.solve(A,b)
    p=Xva@w
    r=metrics(p,yva)
    print(f'ridge lam={lam}: val rel_l2={r[0]:.4f} mae={r[1]:.4f} maxe={r[2]:.4f}')
"

output

ridge lam=0.1: val rel_l2=0.2062 mae=0.1444 maxe=0.4492
ridge lam=1.0: val rel_l2=0.2097 mae=0.1446 maxe=0.4972
ridge lam=10.0: val rel_l2=0.2203 mae=0.1496 maxe=0.5998

A simple ridge regression on hand-crafted features achieves val rel_l2 ≈ 0.206, much better than the constant 0.47. This tells me the mapping is learnable and a target of rel_l2 < 0.2 is reasonable, likely much better with a good PointNet that learns richer features. Voxel occupancy features clearly help (they approximate volume). The PointNet with max-pooling should be able to learn similar and better features. Now let me build the real training pipeline for the fixed PointNet. Key elements: 1. Input normalization (feat_mean, feat_std) , per-axis standardization. The train.py computes these over all points. I'll use that. 2. Label standardization. 3. Data augmentation (critical for 100 samples): - Random resampling of points (with replacement or subsample from 2048) , but we only have 2048 fixed. I can randomly drop points and duplicate (point dropout), or sample subsets each step. - Jitter: add small Gaussian noise. - Random scaling? NO , scaling changes the bounding box normalization and mass relationship. Actually scaling the whole cloud uniformly would change extents; since mass is inferred from normalized shape, uniform scaling would break the normalization convention (test data is normalized to unit cube). Avoid global scaling. Could do tiny anisotropic scaling as augmentation but risky. Avoid. - Reflection along x and z axes: need to verify symmetry. Let me check whether reflecting keeps distribution similar. Reflection preserves axis-aligned bbox so stays on-distribution in terms of normalization. Mass is invariant. Even if shapes aren't perfectly symmetric, a mirrored bracket is a plausible bracket (brackets can be mirrored). This is a strong, safe augmentation. Let me check the x-mirror symmetry of the dataset to be sure it's reasonable. - Small rotations about the y? Hmm, rotation changes bbox. Avoid large; maybe tiny rotations. Avoid for safety. - Point jitter + dropout + reflection are the safe set. Let me verify reflection symmetry: compare distribution of points vs mirrored. Actually, I'll just check whether the bracket is roughly symmetric about the x=center plane and z=center plane by measuring how well the mirrored cloud matches via a coarse voxel IoU. But this takes time. Let me reason: SimJEB brackets attach to an engine with a symmetric bolt pattern; many are symmetric left-right. The safe augmentation that never hurts distribution-wise: reflection about the plane perpendicular to an axis, recentered. Because after reflecting x->-x and the bounding box is symmetric if originally the centering is by bbox centroid. Since centered by bbox centroid, bbox is [min,max] symmetric-ish? Not exactly. Let me just check: does mirroring x produce a cloud whose per-axis stats match the original distribution? If the data has left-right symmetry, mirror augmentation is clearly valid. Let me test quickly with voxel IoU between each cloud and its x-mirror. Actually, a cleaner and very safe augmentation given these are bracket designs: reflect along x (left-right). Let me verify symmetry level.

A NumPy ridge baseline on hand-crafted features (voxel occupancy, PCA eigenvalues, moments) reaches val rel_L2 ≈ 0.206 , so the target is clearly learnable and a good PointNet should beat it. Let me verify reflection symmetry of the brackets to decide which augmentations are safe (reflections preserve the axis-aligned bounding box, so they stay on-distribution).

bash

cd /app && python3 -c "
import torch
tr=torch.load('data/train/samples.pt',weights_only=True)
f=tr['features'].float()
N=f.shape[0]
def voxset(pts,R=16):
    idx=((pts+0.5).clamp(0,0.999999)*R).long()
    return set((idx[:,0]*R*R+idx[:,1]*R+idx[:,2]).tolist())
def iou_mirror(axis,R=16):
    ious=[]
    for n in range(N):
        p=f[n].clone()
        # recenter axis about its bbox center, mirror, recenter back
        c=(p[:,axis].max()+p[:,axis].min())/2
        pm=p.clone(); pm[:,axis]=2*c-pm[:,axis]
        a=voxset(p,R); b=voxset(pm,R)
        ious.append(len(a&b)/len(a|b))
    return sum(ious)/len(ious)
for ax,name in [(0,'x'),(1,'y'),(2,'z')]:
    print(f'mean voxel IoU original vs {name}-mirror: {iou_mirror(ax):.3f}')
"

output

mean voxel IoU original vs x-mirror: 0.453
mean voxel IoU original vs y-mirror: 0.760
mean voxel IoU original vs z-mirror: 0.189

Interesting. The y-mirror has the highest IoU (0.76), meaning the brackets are most symmetric about the y-midplane (front-back?). x-mirror IoU 0.45, z-mirror 0.19 (least symmetric). None are perfectly symmetric (IoU=1 would be perfect). y-mirror at 0.76 is fairly symmetric. But wait , for augmentation purposes, I don't necessarily need the individual shape to be symmetric. I need the mirrored shape to be a *plausible member of the population*. Even if a specific bracket isn't self-symmetric, its mirror image is still a valid bracket shape with the same mass (mass is reflection-invariant), and it respects the same bounding-box normalization. So reflection is a label-preserving augmentation that generates plausible new samples, expanding the dataset. This is standard and safe for this kind of problem. The main caveat: if test brackets are only ever in one "handedness," a mirrored training sample might be slightly off-distribution, but since mass is invariant and the network sees both, it learns a mirror-invariant function which is appropriate for a mass predictor (mass doesn't depend on handedness). This should help generalization. I'll use reflection augmentation on all three axes (each with prob 0.5), combined with jitter and point resampling/dropout. Reflection about the bbox center of each axis. Actually simpler: since data is centered (roughly), reflect about 0. But centering is by bbox centroid, so reflecting about 0 = reflecting about bbox center approximately. Let me reflect about the per-sample bbox center to be precise and keep the bbox identical. Actually, to keep things consistent with normalization (coords in [-0.5,0.5], bbox centered), reflecting about the coordinate such that the bbox maps to itself: new = (min+max) - old. This keeps bbox identical. Then re-standardize with feat_mean/std. Good. Hmm, but if I reflect about bbox center, the points get mirrored but feat_mean/std normalization is global (same for all). The mirrored cloud has same bbox. Fine. Actually, let me reconsider: is it simpler and equally valid to just reflect about 0 (negate the coordinate)? The data is centered at bbox centroid, so min ≈ -max per axis, meaning bbox center ≈ 0. Let me check: earlier, train per-axis min/max: axis0 min -0.32, max 0.32 (symmetric!). axis1 -0.5,0.5. axis2 -0.228,0.227. These are symmetric about 0! So centering is by bbox centroid → bbox center = 0 exactly. So reflecting about 0 (negation) keeps the bbox. So augmentation reflect: x -> -x is exact. Great, simple negation works. Now, the normalization feat_mean is near 0 but not exactly (mean of points, not bbox center). Reflecting about 0 then applying (x-feat_mean)/feat_std is fine. Let me now design the full training. Let me write a training script. **Augmentations (applied on raw coords in [-0.5,0.5] before normalization):** 1. Random reflection per axis (negate) with p=0.5 each. 2. Jitter: add Gaussian noise sigma~0.005-0.01, clipped. 3. Point resampling: randomly sample 2048 points with replacement from the 2048 (bootstrap) , or random shuffle + subsample. Since max-pool is permutation invariant, shuffling doesn't matter. Point dropout (randomly drop some points, replace by duplicates) adds robustness. I'll randomly select a subset (e.g., keep 1024-2048) and pad by resampling to keep 2048. Actually simpler: with some probability, resample indices with replacement. 4. Maybe tiny anisotropic scaling (0.95-1.05) then re-normalize? This changes aspect ratio which is informative for mass , risky. I'll skip or use very mild. Actually, scaling then re-normalizing to unit cube could simulate different brackets but changes the label relationship. Skip. Let me also consider small random rotations about y-axis (the long axis)? Rotating about y changes x,z bounding box. After rotation, re-normalize? This changes the normalized shape. Risky. Skip. Keep augmentations conservative and label-preserving: reflections (exact), jitter (small), point dropout/resample. …[truncated]

Reflections are essentially exact bbox-preserving symmetries here (min≈−max per axis), and mass is reflection-invariant, so reflection is a safe label-preserving augmentation. Now let me build the training pipeline for the fixed PointNet. I'll write a reusable module with augmentation, training, SWA, and metric evaluation.

bash

cd /app && cat > /app/lib_train.py << 'PY'
from __future__ import annotations
import json, os, sys, copy
from pathlib import Path
import torch, torch.nn as nn, torch.nn.functional as F
import numpy as np

APP = Path('/app'); sys.path.insert(0, str(APP))
from candidate_model import build_model

def load_cfg():
    with (APP/'task_config.json').open() as fh: return json.load(fh)

def load_split(name):
    p = torch.load(APP/'data'/name/'samples.pt', weights_only=True)
    return p['features'].float(), p['labels'].float()

def metrics(pred, true):
    pred=np.asarray(pred,np.float64).ravel(); true=np.asarray(true,np.float64).ravel()
    rel=np.sqrt(np.sum((pred-true)**2))/np.sqrt(np.sum(true**2))
    mae=np.mean(np.abs(pred-true)); mx=np.max(np.abs(pred-true))
    return rel, mae, mx

def augment(xb, cfg):
    # xb: [B,N,3] raw coords in [-0.5,0.5]
    B,N,_ = xb.shape
    dev = xb.device
    if cfg.get('reflect',True):
        signs = (torch.randint(0,2,(B,1,3),device=dev).float()*2-1)
        xb = xb*signs
    if cfg.get('jitter',0)>0:
        xb = xb + torch.randn_like(xb)*cfg['jitter']
    if cfg.get('resample_p',0)>0:
        do = torch.rand(B,device=dev) < cfg['resample_p']
        if do.any():
            idx = torch.randint(0,N,(B,N),device=dev)
            xb_rs = torch.gather(xb, 1, idx.unsqueeze(-1).expand(-1,-1,3))
            xb = torch.where(do.view(B,1,1), xb_rs, xb)
    if cfg.get('scale_jit',0)>0:
        s = 1.0 + (torch.rand(B,1,3,device=dev)*2-1)*cfg['scale_jit']
        xb = xb*s
    return xb

def update_bn(model, x_raw, fmean, fstd, bs=64):
    # recompute BatchNorm running stats over data (normalized)
    for m in model.modules():
        if isinstance(m, nn.BatchNorm1d):
            m.reset_running_stats(); m.momentum=None
    model.train()
    with torch.no_grad():
        for i in range(0, x_raw.shape[0], bs):
            xb=((x_raw[i:i+bs]-fmean)/fstd)
            model(xb)
    model.eval()

def train_once(Xtr_raw, ytr, Xva_raw, yva, cfg, device='cuda', verbose=False):
    torch.manual_seed(cfg.get('seed',0)); np.random.seed(cfg.get('seed',0))
    base=load_cfg()
    fmean = Xtr_raw.reshape(-1,3).mean(0); fstd = Xtr_raw.reshape(-1,3).std(0).clamp_min(1e-6)
    lmean = ytr.mean(0); lstd = ytr.std(0).clamp_min(1e-6)
    fmean=fmean.to(device); fstd=fstd.to(device); lmean=lmean.to(device); lstd=lstd.to(device)
    Xtr_raw=Xtr_raw.to(device); ytr=ytr.to(device)
    Xva_raw=Xva_raw.to(device); yva_n=((yva.to(device)-lmean)/lstd)
    model=build_model(base).to(device)
    epochs=cfg['epochs']; bs=cfg['batch_size']
    opt=torch.optim.AdamW(model.parameters(), lr=cfg['lr'], weight_decay=cfg['wd'])
    warm=cfg.get('warmup',5)
    def lr_at(ep):
        if ep<warm: return (ep+1)/warm
        t=(ep-warm)/max(1,epochs-warm); return 0.5*(1+np.cos(np.pi*t))
    sched=torch.optim.lr_scheduler.LambdaLR(opt, lr_at)
    n=Xtr_raw.shape[0]
    swa_model=None; swa_n=0; swa_start=int(epochs*cfg.get('swa_frac',0.75))
    best={'rel':1e9}; best_state=None
    huber=cfg.get('huber',0.0)
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=device)
        for s in range(0,n,bs):
            idx=perm[s:s+bs]
            xb=Xtr_raw[idx]; yb=((ytr[idx]-lmean)/lstd)
            xb=augment(xb,cfg)
            xb=(xb-fmean)/fstd
            opt.zero_grad(set_to_none=True)
            pred=model(xb)
            if huber>0: loss=F.huber_loss(pred,yb,delta=huber)
            else: loss=F.mse_loss(pred,yb)
            loss.backward(); opt.step()
        sched.step()
        # SWA accumulate
        if ep>=swa_start and (ep%cfg.get('swa_every',1)==0):
            if swa_model is None: swa_model=copy.deepcopy(model); swa_n=1
            else:
                with torch.no_grad():
                    for ps,pm in zip(swa_model.parameters(),model.parameters()):
                        ps.mul_(swa_n/(swa_n+1)).add_(pm/(swa_n+1))
                    for bs_,bm in zip(swa_model.buffers(),model.buffers()):
        
…[truncated]

output

written lib_train.py

Now let me run a first experiment: train on train, validate on val, with a baseline augmentation config.

Now let me run a first experiment comparing augmentation settings, training on `train` and evaluating on `val`.

bash

cd /app && python3 -c "
import torch, numpy as np
from lib_train import load_split, train_once
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=150,warmup=5,swa_frac=0.7,seed=0,
          reflect=True,jitter=0.01,resample_p=0.5,scale_jit=0.0,huber=0.0)
import time
for name,over in [('no_aug',dict(reflect=False,jitter=0,resample_p=0)),
                  ('reflect_only',dict(jitter=0,resample_p=0)),
                  ('full_aug',dict())]:
    cfg=dict(base); cfg.update(over)
    t=time.time(); r=train_once(Xtr,ytr,Xva,yva,cfg,verbose=False)
    b=r['best']; s=r['swa']
    print(f'{name:14s} best ep{b[\"ep\"]:3d} rel{b[\"rel\"]:.4f} mae{b[\"mae\"]:.4f} mx{b[\"mx\"]:.4f} | SWA rel{s[0]:.4f} mae{s[1]:.4f} mx{s[2]:.4f} | {time.time()-t:.1f}s')
"

output

[stderr]
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/app/lib_train.py", line 107
    print(f'  ep{ep:3d} lr{opt.param_groups[0][\"lr\"]:.1e} val rel{r[0]:.4f} mae{r[1]:.4f} mx{r[2]:.4f}')
                                                                                                         ^
SyntaxError: f-string expression part cannot include a backslash

I need to fix an f-string with backslashes in the library. Let me correct that line.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:         if verbose and (ep%max(1,epochs//10)==0 or ep==epochs-1):
            print(f'  ep{ep:3d} lr{opt.param_groups[0][\"lr\"]:.1e} val rel{r[0]:.4f} mae{r[1]:.4f} mx{r[2]:.4f}')
new_str:         if verbose and (ep%max(1,epochs//10)==0 or ep==epochs-1):
            cur_lr=opt.param_groups[0]['lr']
            print('  ep%3d lr%.1e val rel%.4f mae%.4f mx%.4f'%(ep,cur_lr,r[0],r[1],r[2]))

output

The file /app/lib_train.py has been edited successfully.

bash

cd /app && python3 -c "
import torch, numpy as np, time
from lib_train import load_split, train_once
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=150,warmup=5,swa_frac=0.7,seed=0,
          reflect=True,jitter=0.01,resample_p=0.5,scale_jit=0.0,huber=0.0)
for name,over in [('no_aug',dict(reflect=False,jitter=0,resample_p=0)),
                  ('reflect_only',dict(jitter=0,resample_p=0)),
                  ('full_aug',dict())]:
    cfg=dict(base); cfg.update(over)
    t=time.time(); r=train_once(Xtr,ytr,Xva,yva,cfg,verbose=False)
    b=r['best']; s=r['swa']
    print('%-14s best ep%3d rel%.4f mae%.4f mx%.4f | SWA rel%.4f mae%.4f mx%.4f | %.1fs'%(name,b['ep'],b['rel'],b['mae'],b['mx'],s[0],s[1],s[2],time.time()-t))
"

output

no_aug         best ep124 rel0.1874 mae0.1388 mx0.3726 | SWA rel0.1963 mae0.1458 mx0.3881 | 7.3s
reflect_only   best ep135 rel0.2747 mae0.1886 mx0.5796 | SWA rel0.5498 mae0.3925 mx0.9438 | 3.6s
full_aug       best ep124 rel0.2980 mae0.1998 mx0.7535 | SWA rel0.3615 mae0.2409 mx0.8913 | 4.9s

Interesting , surprisingly, no_aug is BEST here (rel 0.187), and augmentation HURTS. This is counterintuitive. Possible reasons: 1. The reflection augmentation takes samples off-distribution because the brackets are NOT symmetric, and the mass mapping depends on the specific (non-mirrored) shape. Since val/test are all in canonical orientation, training on mirrored versions confuses the model. 2. Jitter/resample may be too aggressive. Wait, but reflection shouldn't change mass... Let me reconsider. The issue is the model has limited capacity and the augmentation makes the learning problem harder (more variance), and with only 150 epochs it underfits the augmented distribution. Also, SWA is hurting a lot with augmentation , maybe because BN stats recomputed on raw (non-augmented) train data but model trained on augmented → mismatch. Actually update_bn uses raw training data normalized (no augmentation), which matches inference distribution, so that's correct. Hold on , the bigger issue: maybe reflections genuinely hurt because the mass-shape mapping is sensitive to orientation in this normalized space. But physically mass is reflection invariant... unless the point SAMPLING or the dataset's canonical alignment encodes scale info. Actually here's a subtle thing: these are surface point samples. The mass depends on volume & density. Reflection preserves volume, so mass truly invariant. So why does reflect hurt? Let me reconsider: With reflection, the effective dataset is larger/harder. With only 150 epochs and strong reflection (prob 0.5 each axis = 8 combos), the model sees each original config less often. Underfitting. Maybe need more epochs. But no_aug already converges to 0.187 and augmented versions are worse even at best epoch. Let me reconsider whether reflection is actually valid. Let me test: train no_aug longer and see if it improves or overfits. And test lighter augmentation (just jitter small, no reflect; just resample). Let me also try more epochs for augmented. Actually, the key question is generalization to the TRUE held-out test, not val (32 samples, noisy estimate). val with 32 samples , a single hard sample dominates max error. rel 0.187 on val might be optimistic or pessimistic. I'm worried about overfitting to val by picking "best epoch" on val (124). That's selection bias , best epoch chosen by val means val metric is optimistically biased. For a fair estimate I should use SWA or a fixed epoch, OR use cross-validation across train+val. Given only 100+32 samples, I should do **k-fold cross-validation** on the combined 132 samples to get a robust estimate of generalization and to choose hyperparameters. That's the right methodology here. Let me implement k-fold CV. But first, the augmentation puzzle. Let me investigate more carefully with CV rather than single val split, because the single val split is noisy. Let me set up 4-fold or 6-fold CV over the combined train+val (132), measuring the metrics using a fixed training budget (no val-based epoch selection , instead use final model or SWA). That removes selection bias. Let me reconsider the "best epoch" selection: In train_once I select best_state by val rel. For CV, I should NOT peek , I'll report the SWA result and the final-model result, and maybe a fixed-epoch result. Actually SWA gives a principled single model. Let me rely on SWA or final-epoch. But SWA hurt above. Let me debug SWA: maybe the issue is swa_frac 0.7 with cosine LR going to ~0 at the end , SWA typically uses a high constant or cyclic LR. With cosine decaying to 0, the last 30% of epochs have very low LR, so SWA averages nearly-identical late models ≈ final model. That should be fine, not a 0.55 disaster. The disaster suggests BN update issue or that the averaged model is bad. Hmm, SWA rel 0.55 for reflect_only is worse than constant predictor. Something's off with update_bn. Let me check: I set m.momentum=None which makes BN use cumulative average , good. reset_running_stats r …[truncated]

Surprising , augmentation hurt and SWA behaved poorly here. But a 32-sample val split is noisy and "best epoch" selection is biased. I need a robust, unbiased signal. Let me set up k-fold cross-validation over the combined 132 samples and evaluate the *final* model (no peeking) to make decisions on solid ground.

bash

cd /app && cat > /app/cv.py << 'PY'
import torch, numpy as np, time, sys
from lib_train import load_split, train_once, metrics
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0)
N=X.shape[0]
g=torch.Generator().manual_seed(1234); perm=torch.randperm(N,generator=g)
K=6
folds=[perm[i::K] for i in range(K)]  # stratified-ish by shuffle

def run(cfg, report='final'):
    rels=[];maes=[];mxs=[]
    allp=[];allt=[]
    for k in range(K):
        va_idx=folds[k]; tr_idx=torch.cat([folds[j] for j in range(K) if j!=k])
        r=train_once(X[tr_idx],y[tr_idx],X[va_idx],y[va_idx],cfg)
        # pick model
        import torch as T
        from candidate_model import build_model
        from lib_train import load_cfg
        fmean=r['fmean'].cuda();fstd=r['fstd'].cuda();lmean=r['lmean'].cuda();lstd=r['lstd'].cuda()
        if report=='swa' and r['swa_model'] is not None:
            m=r['swa_model']
        elif report=='best':
            m=build_model(load_cfg()).cuda(); m.load_state_dict(r['best_state']); m.eval()
        else:
            m=r['final_model']
        m.eval()
        with T.no_grad():
            vp=(m((X[va_idx].cuda()-fmean)/fstd)*lstd+lmean).cpu()
        allp.append(vp); allt.append(y[va_idx])
    allp=torch.cat(allp).numpy().ravel(); allt=torch.cat(allt).numpy().ravel()
    return metrics(allp,allt)

if __name__=='__main__':
    base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=150,warmup=5,swa_frac=0.7,seed=0,
              reflect=False,jitter=0.0,resample_p=0.0,scale_jit=0.0,huber=0.0)
    import json
    configs=json.loads(sys.argv[1]) if len(sys.argv)>1 else [{}]
    for over in configs:
        cfg=dict(base); cfg.update(over)
        t=time.time()
        rf=run(cfg,'final'); 
        print('FINAL %-40s relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(over),rf[0],rf[1],rf[2],time.time()-t),flush=True)
PY
python3 cv.py '[{"reflect":false,"jitter":0.0,"resample_p":0.0},{"reflect":true},{"jitter":0.01},{"resample_p":0.5},{"reflect":true,"jitter":0.01,"resample_p":0.5}]'

output

FINAL {'reflect': False, 'jitter': 0.0, 'resample_p': 0.0} relL2=0.2752 mae=0.1734 maxe=0.8235 (24s)
FINAL {'reflect': True}                        relL2=0.3066 mae=0.1928 maxe=0.8836 (21s)
FINAL {'jitter': 0.01}                         relL2=0.3115 mae=0.1981 maxe=0.8161 (22s)
FINAL {'resample_p': 0.5}                      relL2=0.2619 mae=0.1694 maxe=0.7273 (33s)
FINAL {'reflect': True, 'jitter': 0.01, 'resample_p': 0.5} relL2=0.3685 mae=0.2315 maxe=1.0310 (40s)

With unbiased CV (final model, 6-fold over 132 samples): - no_aug: rel 0.275 - reflect: 0.307 (worse) - jitter: 0.311 (worse) - resample_p 0.5: 0.262 (slightly better) - all: 0.369 (worse) So the CV estimate of generalization is ~0.26-0.28, noticeably worse than the single val's 0.187 (confirming the val "best epoch" was optimistic/biased). The true test performance is likely around 0.26-0.30 with this setup. Reflection and jitter hurt. resample helps slightly. Interesting. So the model is struggling. The final-model CV rel ~0.27 is worse than the ridge baseline (0.206 on val). Hmm. The ridge on hand features got 0.206 on val. Let me CV the ridge too for fair comparison. Actually, this tells me the PointNet as-is is underperforming hand-crafted features. I need to improve the PointNet training substantially. Let me think about WHY it's underperforming: 1. The architecture uses max-pooling over a global feature. With only 100 samples, it may underfit the complex geometry→mass mapping, OR overfit. 2. The relative L2 is dominated by worst cases (maxe ~0.8). Large errors on a few samples. Key insight: The hand-crafted features that worked best were voxel occupancy (volume proxy) and std_x. The PointNet with max-pooling captures extents (max coordinates) well but may not capture VOLUME/occupancy well, since max-pooling only gets extremes, not density/counts. Mass ∝ volume, and volume needs density/counting info that max-pooling discards! This is a known PointNet limitation: global max-pooling captures the "silhouette/extent" but not how much volume is filled. Two shapes with same bounding extent but different fill would give similar max-pooled features. But I can't change the architecture. However, I CAN change the INPUT representation. The model does `points.transpose(1,2)` then conv1d. The input is [B,2048,3]. I must feed [B,2048,3]. But what if I augment the point cloud in a way that encodes density... no, input is fixed as coords. Hmm, wait. Actually I can't change input dim (must be [B,2048,3]) and the grader feeds raw normalized coords. So I can't add features. So within these constraints, how to get the best out of max-pooled PointNet? The max-pool captures, per feature channel, the max over points of a learned function of (x,y,z). With 64→64→64 then 128→256 channels, the network can learn many localized "detectors" whose max activation indicates presence of points in certain regions. The collection of 256 maxed features can encode a coarse occupancy/shape signature (like a soft voxelization): each channel fires if any point falls in a learned region, and max≈1 if present. This can approximate occupancy patterns → volume. So it CAN represent volume-ish features, but learning this from 100 samples is hard. Ideas to improve: 1. **More training / better regularization** to reduce the worst-case errors. 2. **Better loss**: maybe the issue is a few outliers. Could weight them. 3. **Ensemble via SWA or multi-seed weight averaging** , but weight averaging across seeds doesn't work. However, I could train multiple models and... no, single checkpoint. Wait , actually, can I make the SINGLE model an ensemble internally? No, it must be exactly PointNetRegressor. 4. **Input normalization choice**: Currently per-axis standardization by std. This stretches z (small std 0.087) more than y. After standardization, coords have unit std per axis. This might distort geometry. Alternative: use a single global scale (isotropic) to preserve aspect ratio, OR no normalization (coords already in [-0.5,0.5]). Let me reconsider. The feat_mean/std approach stretches each axis to unit variance, destroying the aspect-ratio information that's PREDICTIVE of mass (ext_x, ext_z correlate with mass). Wait no , standardization is affine per-axis; it rescales but the relative differences between samples are preserved. Sample A with larger ext_x still has larger ext_x after dividing by the (common) std. So aspect ratio info across samples is preser …[truncated]

The unbiased CV estimate (~0.26–0.28) is notably worse than the biased single-val number (0.187), confirming selection bias. The PointNet is also underperforming the ridge baseline. Let me get an honest CV number for the ridge baseline as my target, then tune the network harder.

bash

cd /app && python3 -c "
import torch, numpy as np
from lib_train import load_split, metrics
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0).squeeze(1)
def make_feats(f):
    out=[]; mn=f.amin(1);mx=f.amax(1);ext=mx-mn;std=f.std(1);mean=f.mean(1)
    out+=[ext,std,mean]
    fc=f-f.mean(1,keepdim=True); cov=torch.einsum('npi,npj->nij',fc,fc)/f.shape[1]
    out.append(torch.linalg.eigvalsh(cov))
    for R in (8,16):
        idx=((f+0.5).clamp(0,0.999999)*R).long(); lin=idx[...,0]*R*R+idx[...,1]*R+idx[...,2]
        occ=torch.zeros(f.shape[0],R*R*R)
        for n in range(f.shape[0]): occ[n,torch.unique(lin[n])]=1.0
        out.append(occ.sum(1,keepdim=True)/(R*R*R))
    return torch.cat(out,1).numpy()
Xf=make_feats(X); yv=y.numpy()
N=len(Xf); g=torch.Generator().manual_seed(1234); perm=torch.randperm(N,generator=g).numpy()
K=6; folds=[perm[i::K] for i in range(K)]
for lam in [0.3,1.0,3.0,10.0]:
    allp=np.zeros(N)
    for k in range(K):
        va=folds[k]; tr=np.concatenate([folds[j] for j in range(K) if j!=k])
        mu=Xf[tr].mean(0); sd=Xf[tr].std(0)+1e-8
        A=(Xf[tr]-mu)/sd; B=(Xf[va]-mu)/sd
        A=np.c_[A,np.ones(len(A))]; B=np.c_[B,np.ones(len(B))]
        w=np.linalg.solve(A.T@A+lam*np.eye(A.shape[1]),A.T@yv[tr])
        allp[va]=B@w
    r=metrics(allp,yv)
    print('ridge CV lam=%.1f relL2=%.4f mae=%.4f maxe=%.4f'%(lam,r[0],r[1],r[2]))
"

output

ridge CV lam=0.3 relL2=0.2244 mae=0.1522 maxe=0.5145
ridge CV lam=1.0 relL2=0.2240 mae=0.1508 maxe=0.5448
ridge CV lam=3.0 relL2=0.2257 mae=0.1505 maxe=0.5708
ridge CV lam=10.0 relL2=0.2404 mae=0.1565 maxe=0.6337

Ridge CV: rel_l2 ≈ 0.224, mae 0.15, maxe 0.51. This is a solid target. My PointNet final-model CV is 0.26-0.28, worse. Now I want the PointNet to beat 0.224. Let me think harder. The voxel-occupancy features are doing a lot of work (volume proxy). PointNet max-pooling struggles with this. But a well-trained PointNet can encode occupancy via many localized detectors. The problem is learning from 100 samples. Important idea: Since the network struggles, maybe I can help it by choosing the input normalization to make spatial structure more learnable. Currently anisotropic standardization. Alternative idea , and this is powerful: I can combine the neural net with the hand-crafted insight by NOT changing architecture but improving training. But the cleanest win: ensemble multiple PointNets... but single checkpoint only. Wait, let me reconsider the ensembling constraint. The saved state_dict must load into PointNetRegressor. One model. BUT, I could train many models with different seeds and augmentations and select the single best by CV? No, I still deploy one model, and its individual generalization is ~0.27. Hmm, what about weight averaging of independently trained models (like "model soup")? Model soups work when models are fine-tuned from the same initialization. If I train from the same init with different data orderings/augmentation, averaging weights can work (since they stay in the same basin). This is basically SWA across runs. Worth trying: train several models from the same seed init but different augmentation RNG, average weights → a soup. This could reduce variance and improve generalization, giving a single deployable model. But first, let me improve the base single-model training to beat ridge. Let me sweep hyperparameters with CV. Let me try more epochs, different wd, lr, batch size, resample aug. Let me also reconsider whether per-axis anisotropic normalization is good. Let me try isotropic normalization (single scale) , preserves shape. Actually the starter computes per-axis std. Let me test both in the sweep by adding a flag. But lib_train computes feat per-axis always. Let me add an option. Actually, let me reconsider a more fundamental point. The max-pool PointNet maps each point through an MLP and takes max. For predicting a volume/mass, the most useful statistic would be something like counting. Max-pool can't count. But the network can still use the spatial extent and shape silhouette. The ridge used extents+occupancy. PointNet gets extents easily (max of linear = max coordinate). Occupancy (volume fill) is the hard part. Given the architectural constraint, let me just push training quality and regularization, and use a weight-averaged soup + SWA properly. Let me also test larger epochs. Let me run a focused sweep. I'll add an 'iso' normalization option and a 'center_only' option to lib_train. Let me modify train_once to accept norm mode. Let me keep it simple: test epochs and wd and batch with resample aug (best so far). Let me sweep. Time per 6-fold run ~25-40s. Let me run a sweep of ~6 configs in one go (~3-4 min). Let me be strategic. Config sweep 1 (all with resample_p=0.5, reflect=False, jitter=0): - epochs 300, lr 1e-3, wd 1e-4, bs16 - epochs 300, lr 1e-3, wd 1e-3, bs16 - epochs 300, lr 2e-3, wd 1e-3, bs32 - epochs 500, lr 1e-3, wd 1e-3, bs16 - epochs 300, lr 1e-3, wd 3e-3, bs16 Let me run.

Ridge CV gives rel_L2 ≈ 0.224 , that's my target to beat. The PointNet is underperforming, likely because global max-pooling captures extent but not volume/fill well. Within the fixed architecture, I'll push training quality. Let me sweep epochs, weight decay, LR, and batch size (with the resample augmentation that helped) via CV.

bash

cd /app && timeout 600 python3 cv.py '[
{"resample_p":0.5,"epochs":300,"lr":1e-3,"wd":1e-4,"batch_size":16},
{"resample_p":0.5,"epochs":300,"lr":1e-3,"wd":1e-3,"batch_size":16},
{"resample_p":0.5,"epochs":300,"lr":2e-3,"wd":1e-3,"batch_size":32},
{"resample_p":0.5,"epochs":500,"lr":1e-3,"wd":1e-3,"batch_size":16},
{"resample_p":0.5,"epochs":300,"lr":1e-3,"wd":3e-3,"batch_size":16}
]'

output

[stderr]
FINAL {'resample_p': 0.5, 'epochs': 300, 'lr': 0.001, 'wd': 0.0001, 'batch_size': 16} relL2=0.2470 mae=0.1557 maxe=0.6795 (88s)
FINAL {'resample_p': 0.5, 'epochs': 300, 'lr': 0.001, 'wd': 0.001, 'batch_size': 16} relL2=0.2593 mae=0.1662 maxe=0.7995 (83s)
FINAL {'resample_p': 0.5, 'epochs': 300, 'lr': 0.002, 'wd': 0.001, 'batch_size': 32} relL2=0.2478 mae=0.1562 maxe=0.6662 (66s)Shell command timed out.
Tool interaction exceeded 300 seconds.

The tool timed out at 300s. Let me restart and run fewer configs per call, in the background, to avoid timeouts. Results so far: epochs 300 with resample, lr1e-3 wd1e-4 → 0.247. lr2e-3 wd1e-3 bs32 → 0.248. These are better than before (0.26) but still worse than ridge 0.224. Let me restart the tool and be more careful about timeouts , run in background with nohup and poll.

The tool timed out. Let me restart it and run experiments in the background to avoid the 300s limit.

bash

restart: true

output

<system>tool has been restarted.</system>

bash

cd /app && nohup python3 cv.py '[
{"resample_p":0.5,"epochs":600,"lr":1e-3,"wd":1e-3,"batch_size":16},
{"resample_p":0.5,"epochs":400,"lr":1.5e-3,"wd":3e-4,"batch_size":16}
]' > /app/sweep1.log 2>&1 &
echo "started PID $!"

output

started PID 312

While that runs, let me think more deeply about the architecture's limitation and how to overcome it. The core issue: max-pooling PointNet captures extent but not volume/density well. But the ridge's strongest features included voxel occupancy. Let me reconsider. Maybe the bottleneck isn't capacity but generalization variance with 100 samples. The ridge is heavily regularized (linear). The PointNet has ~few hundred K params. Let me count params. Actually, a better idea to reduce variance and improve generalization: train an ENSEMBLE and average into a model soup (same init). Let me test whether a soup of models (same init, different augmentation/ordering seeds) generalizes better. If individual models are ~0.25 but decorrelated errors, a soup (weight average) might approach ~0.20. But weight-averaging only works if models stay in the same loss basin (same init, not too divergent). With 300+ epochs and different data orderings, they may diverge too much for weight averaging. An alternative that ALWAYS works for reducing variance is output-averaging an ensemble , but I can only deploy one model. Unless... I distill the ensemble into a single PointNet! Train an ensemble of N models (output-averaged predictions on train+val, and even on augmented copies), then train a single PointNet to match the ensemble's predictions (knowledge distillation). The single distilled model can absorb much of the ensemble's generalization. But distillation target is limited to available inputs; I could generate many augmented point clouds and label them with the ensemble's averaged prediction, giving the student a richer training signal. This is promising but complex. Actually, simpler and quite effective: Since I can't ensemble at inference, but I CAN use all data (train+val = 132) for the final model, and use SWA along the trajectory (which is a legit single model). Let me get SWA working properly since it earlier failed , probably a bug or LR schedule issue. Let me reconsider: The earlier SWA disaster (0.55) was with reflect augmentation. Let me check SWA for no-aug / resample configs specifically. Let me also reconsider the LR schedule for SWA: cosine to near-zero means late epochs barely move , SWA ≈ final. So SWA shouldn't hurt for no-aug. The 0.55 was likely the reflect augmentation making the averaged model bad + BN mismatch. Let me verify SWA on resample config via the log (cv.py reports only final). Let me add SWA reporting. Let me wait for sweep1 and think about the biggest lever. Actually, let me reconsider the problem from the data scale. The true held-out test: I don't know its size but public_test is 16. The grading uses a held-out set. My CV on 132 samples is the best proxy. The ridge at 0.224 uses volume features. To beat it with PointNet, I likely need the PointNet to capture volume. Let me think: does the PointNet even have access to volume info? The input is 2048 surface points. Volume isn't directly given, but the DENSITY of surface points might correlate... Actually all clouds have exactly 2048 points regardless of size. Surface area sampling: if points are uniformly sampled over the surface, denser regions (per unit area) would indicate... no, uniform sampling gives uniform density. Hmm. The number of points is fixed at 2048. So local density ~ 2048/surface_area. But surface_area in normalized coords... Actually the mass correlates with how "bulky/filled" the normalized shape is. A chunky bracket (high volume fraction within bbox) vs a thin one. Surface points reveal the silhouette and concavities. Max-pool captures the convex extent mainly. Let me reconsider: maybe I should reduce model overfitting via strong weight decay AND more epochs AND averaging. Let me also try the model soup idea quickly once I have a good base config. Let me check sweep progress. Actually, let me reconsider whether a better input normalization helps. Let me test isotropic scaling. And test "no normalization beyond centering" (coords …[truncated]

While the sweep runs, let me improve the harness: report SWA too, support isotropic normalization, and separate the init seed from the augmentation seed (needed for "model soup" weight-averaging). Let me check params and the running sweep first.

bash

cd /app && python3 -c "
from lib_train import load_cfg
from candidate_model import build_model
m=build_model(load_cfg()); print('params', sum(p.numel() for p in m.parameters()))
" && sleep 60 && echo "---sweep1 so far---" && cat sweep1.log

output

params 92289
---sweep1 so far---

92K params, 100-132 training samples. That's a lot of params relative to data, but point clouds provide many "samples" per shape (2048 points) for the shared MLP. Still, overfitting risk is real. sweep1.log is empty still (600 epochs takes a while, ~2-3 min per config × 6 folds). Let me wait more. Meanwhile let me enhance lib_train for norm mode and separate seeds. Let me edit train_once to: - accept cfg['init_seed'] for model init (default cfg['seed']) - accept cfg['norm'] in {'aniso','iso','none'} - return everything needed. Let me add.

92K params vs ~100 samples , overfitting risk is real, though each cloud gives 2048 point-samples to the shared MLP. Let me enhance `lib_train` to support isotropic normalization and a separate init-seed (for model soups), while the sweep continues.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:     torch.manual_seed(cfg.get('seed',0)); np.random.seed(cfg.get('seed',0))
    base=load_cfg()
    fmean = Xtr_raw.reshape(-1,3).mean(0); fstd = Xtr_raw.reshape(-1,3).std(0).clamp_min(1e-6)
new_str:     base=load_cfg()
    fmean = Xtr_raw.reshape(-1,3).mean(0); fstd = Xtr_raw.reshape(-1,3).std(0).clamp_min(1e-6)
    nm=cfg.get('norm','aniso')
    if nm=='iso':
        fstd = fstd.mean().repeat(3)
    elif nm=='none':
        fmean=torch.zeros(3); fstd=torch.ones(3)

output

The file /app/lib_train.py has been edited successfully.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:     Xva_raw=Xva_raw.to(device); yva_n=((yva.to(device)-lmean)/lstd)
    model=build_model(base).to(device)
new_str:     Xva_raw=Xva_raw.to(device); yva_n=((yva.to(device)-lmean)/lstd)
    torch.manual_seed(cfg.get('init_seed',cfg.get('seed',0)))
    model=build_model(base).to(device)
    torch.manual_seed(cfg.get('seed',0)); np.random.seed(cfg.get('seed',0))

output

The file /app/lib_train.py has been edited successfully.

bash

cd /app && sleep 120 && cat sweep1.log

output

FINAL {'resample_p': 0.5, 'epochs': 600, 'lr': 0.001, 'wd': 0.001, 'batch_size': 16} relL2=0.2428 mae=0.1585 maxe=0.6666 (136s)

600 epochs, resample, lr1e-3, wd1e-3: 0.2428. Only marginal improvement over 300 epochs (0.247). Diminishing returns. Still worse than ridge 0.224. Let me think differently. The PointNet is stuck around 0.24-0.25. The ridge with volume features is at 0.224. Big idea: What if I help the PointNet learn volume by a better approach , but I can't change architecture or input. Alternative big idea: **Distillation from the ridge/ensemble**, OR **augment training data with the ensemble**. Actually, let me reconsider. Maybe the issue is that max-pool PointNet genuinely can't easily represent volume, but CAN represent it if I give it enough of the right training signal. Let me test: can a PointNet OVERFIT the training set well? If it can drive train error to ~0, then it has capacity and the issue is generalization (need regularization/more data/ensembling). If it can't even fit train, then representational limits. Let me check train error for a config. Let me quickly train on all 132 and check fit on training data. Actually more useful: let me pursue the model soup / multi-seed ensemble distillation, which addresses generalization variance , likely the main issue. Let me first quantify: what's the CV performance of an OUTPUT-averaged ensemble of K models (different init seeds)? If that's much better (say 0.20), then distilling or soup-ing is worth it. Let me test ensemble-of-final-models via output averaging in CV. I'll write cv_ens.py. This is the key experiment: does ensembling help? If yes, I'll find a way to deploy a single model capturing it (soup or distillation). Let me write cv_ens that for each fold trains K models (different init_seed and seed), averages their outputs, and computes metrics. Also test weight-soup (average weights) vs output-average. Let me be mindful of time: K=4 models × 6 folds × (say 200 epochs) could be slow. Let me use epochs=250, resample aug, K=4. Each model ~ let me estimate: 300 epochs took ~14s/fold earlier (88s/6). So 250 epochs ~12s × 4 models × 6 folds = ~290s. Too close to timeout for foreground. Run in background. Let me write cv_ens.py.

600 epochs barely helped (0.243). The network plateaus around 0.24, just above ridge. The key question is whether the bottleneck is *generalization variance* , if so, ensembling will help a lot. Let me test whether an output-averaged ensemble and a weight-soup improve CV, which would justify deploying a single distilled/averaged model.

bash

cd /app && cat > /app/cv_ens.py << 'PY'
import torch, numpy as np, time, sys, json, copy
from lib_train import load_split, train_once, metrics, update_bn, load_cfg
from candidate_model import build_model
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0)
N=X.shape[0]
g=torch.Generator().manual_seed(1234); perm=torch.randperm(N,generator=g)
K=6; folds=[perm[i::K] for i in range(K)]

def eval_model(m,Xe,fmean,fstd,lmean,lstd):
    m.eval()
    with torch.no_grad():
        return (m((Xe.cuda()-fmean)/fstd)*lstd+lmean).cpu()

def run(cfg, M=4):
    p_ens=[]; p_soup=[]; t_all=[]
    for k in range(K):
        va=folds[k]; tr=torch.cat([folds[j] for j in range(K) if j!=k])
        models=[]; norm=None
        for s in range(M):
            c=dict(cfg); c['seed']=100+s; c['init_seed']=200+s
            r=train_once(X[tr],y[tr],X[va],y[va],c)
            models.append(r['final_model'])
            norm=(r['fmean'].cuda(),r['fstd'].cuda(),r['lmean'].cuda(),r['lstd'].cuda())
        fmean,fstd,lmean,lstd=norm
        # output ensemble
        preds=torch.stack([eval_model(m,X[va],*norm) for m in models],0).mean(0)
        p_ens.append(preds)
        # weight soup
        soup=copy.deepcopy(models[0])
        sd={kk: torch.stack([m.state_dict()[kk].float() for m in models],0).mean(0) for kk in soup.state_dict()}
        soup.load_state_dict(sd)
        update_bn(soup, X[tr].cuda(), fmean, fstd)
        p_soup.append(eval_model(soup,X[va],*norm))
        t_all.append(y[va])
    t=torch.cat(t_all).numpy().ravel()
    pe=torch.cat(p_ens).numpy().ravel(); ps=torch.cat(p_soup).numpy().ravel()
    return metrics(pe,t), metrics(ps,t)

if __name__=='__main__':
    base=dict(batch_size=16,lr=1e-3,wd=1e-3,epochs=300,warmup=5,swa_frac=0.7,
              reflect=False,jitter=0.0,resample_p=0.5,scale_jit=0.0,huber=0.0)
    cfgs=json.loads(sys.argv[1])
    M=int(sys.argv[2]) if len(sys.argv)>2 else 4
    for over in cfgs:
        cfg=dict(base); cfg.update(over)
        t=time.time(); (re,ps)=run(cfg,M)
        print('ENS(M=%d) %-30s relL2=%.4f mae=%.4f maxe=%.4f | SOUP relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(M,str(over),re[0],re[1],re[2],ps[0],ps[1],ps[2],time.time()-t),flush=True)
PY
nohup python3 cv_ens.py '[{"epochs":300}]' 4 > /app/ens.log 2>&1 &
echo "started $!"

output

started 371

While that runs, let me reconsider the problem. Let me think about whether there's a smarter approach to representational limits. Actually, wait. Let me reconsider the claim that PointNet can't capture volume. Let me check how well the best hand features correlate and whether the PointNet is missing something simple like the point DENSITY distribution. Actually, let me reconsider the whole thing. Perhaps anisotropic normalization is hurting. Let me think: feat_std per axis = [0.142, 0.270, 0.087]. Dividing z by 0.087 multiplies z by ~11.5, while y by 1/0.27=3.7, x by 1/0.14=7. So after normalization, the point cloud is stretched to roughly unit variance per axis , heavily distorting aspect ratios that were informative. Wait, but as I argued, across samples the relative differences are preserved (linear scaling). However, the ABSOLUTE aspect ratio within each shape is distorted to ~1:1:1 variance, which might make it harder for the conv filters to detect meaningful geometric patterns consistently. More importantly, the anisotropic scaling is the SAME for all samples (global), so it's just a fixed linear transform of the input space. The network could in principle learn to undo it. It shouldn't fundamentally hurt. But it might affect optimization. Let me test iso and none normalization in CV. Let me wait for ens.log. Meanwhile let me reason about the soup feasibility , if soup works nearly as well as output-ensemble, I can deploy the soup as a single model. That's the cleanest path to a single deployable model that captures ensemble benefits. Let me wait.

Let me wait for the ensemble/soup experiment and think about normalization in parallel.

bash

cd /app && sleep 180 && cat ens.log

output

<system>Tool ran without output or errors</system>

bash

cd /app && sleep 90 && cat ens.log && echo "=== proc ===" && ps aux | grep cv_ens | grep -v grep | head

output

=== proc ===
h2tagent   371 96.3 16.0 20161408 5058808 ?    Rl   21:12   4:28 python3 cv_ens.py [{"epochs":300}] 4

Still running (4 models × 6 folds × 300 epochs). Let me wait more. It's been ~4.5 min. Should finish soon.

bash

cd /app && sleep 120 && cat ens.log

output

ENS(M=4) {'epochs': 300}                relL2=0.2628 mae=0.1601 maxe=0.7442 | SOUP relL2=0.4723 mae=0.3030 maxe=1.5169 (285s)

Interesting and important results: - Output ensemble (M=4): relL2=0.2628 , NOT better than single model (0.247)! Actually worse than the best single. Wait, that's odd. Output averaging usually helps. But here wd=1e-3, epochs=300, resample , single was 0.2593 for wd1e-3. So ensemble 0.2628 ≈ same as single. No benefit from output ensembling! - Soup: 0.47 , terrible (weights in different basins, can't average across different init seeds). The fact that output ensembling of 4 models doesn't help means the errors are NOT just high-variance noise , they're systematic/correlated across seeds. The models make the SAME errors. This means the bottleneck is BIAS/representation, not variance. The PointNet consistently mispredicts certain samples regardless of seed. This strongly suggests a representational limitation: the max-pool PointNet cannot capture the feature needed for those samples (likely volume/fill). Ensembling correlated-error models doesn't help. So to improve, I need to give the model better information or a better inductive bias , but architecture and input are fixed. Wait , the input is fixed as [B,2048,3] coordinates, BUT I control the feat_mean/feat_std transform AND I control training. The grader does: model((x - feat_mean)/feat_std)*label_std+label_mean. feat_mean/feat_std are length-3. So the input transform is constrained to per-axis affine. I can't inject volume features. Hmm. So how to capture volume with max-pool over per-point MLP features? Let me reconsider: is volume really the missing ingredient, or is it something the PointNet SHOULD capture but isn't due to optimization? Let me check the ceiling: can the PointNet FIT the training data well (low train error)? If train error is low but val high → overfitting/variance (but ensembling didn't help, so not variance... unless all seeds overfit the same way). If train error is also high → underfitting/representation. Let me check train-set fit. Let me also reconsider: maybe the resample augmentation is capping performance. The no-aug single was 0.275, resample 0.262. Let me check no-aug ensemble and train fit. Actually the most informative quick test: train one model on ALL 132 with no augmentation for many epochs, measure TRAIN rel_l2. Let me do that now (fast, foreground).

Critical finding: output-ensembling 4 seeds gives **no** improvement (0.263 vs single 0.25), and the soup is terrible. That means errors are *correlated across seeds* , the bottleneck is representational bias, not variance. Let me verify by checking whether the model can even fit the training data (low train error would imply variance; high train error implies a representation/optimization limit).

bash

cd /app && timeout 200 python3 -c "
import torch, numpy as np
from lib_train import load_split, train_once, metrics
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0)
# train on ALL, measure train fit (no aug), capacity probe
for aug in [dict(resample_p=0.0),dict(resample_p=0.5)]:
    cfg=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=400,warmup=5,swa_frac=0.9,seed=0,
             reflect=False,jitter=0.0,scale_jit=0.0,huber=0.0); cfg.update(aug)
    r=train_once(X,y,X,y,cfg)  # eval on train itself
    m=r['final_model']; fmean=r['fmean'].cuda();fstd=r['fstd'].cuda();lmean=r['lmean'].cuda();lstd=r['lstd'].cuda()
    m.eval()
    with torch.no_grad(): p=(m((X.cuda()-fmean)/fstd)*lstd+lmean).cpu()
    tr=metrics(p,y)
    print('aug',aug,'TRAIN-fit relL2=%.4f mae=%.4f maxe=%.4f'%tr)
"

output

aug {'resample_p': 0.0} TRAIN-fit relL2=0.0137 mae=0.0096 maxe=0.0332
aug {'resample_p': 0.5} TRAIN-fit relL2=0.0258 mae=0.0170 maxe=0.0637

The model fits training data extremely well (train rel_l2 = 0.014 without aug, 0.026 with resample). So it has plenty of capacity , the problem is GENERALIZATION (overfitting), not representation bias per se. But wait , if it's overfitting (high variance), why didn't output-ensembling help? Ensembling reduces variance. Unless the overfitting is to systematic features that don't generalize, and all seeds learn the same non-generalizing features (because the training signal pushes them all the same way). Reconciling: The model memorizes train (low train error). On val, all seeds make similar errors (ensembling doesn't help). This means the models aren't differing by random initialization noise in a way that decorrelates val errors , they converge to similar functions that overfit similarly. This is "bias" in the sense of a consistent wrong extrapolation. So the real issue: with 100-132 samples and 92K params, the model finds a solution that fits train but extrapolates poorly and consistently. Ensembling identical-bias models doesn't help. What reduces this kind of overfitting? 1. Stronger regularization (but wd=1e-3 already tried, marginal). 2. MORE DATA via augmentation , but my augmentations (reflect, jitter) HURT. Why would reflection hurt if it's valid? Let me reconsider reflection. Earlier reflect alone in CV gave 0.307 (worse than 0.275 no-aug). If reflection were a valid label-preserving transform, it should help or be neutral. It hurt. Why? Hypothesis: The brackets are NOT reflection-symmetric in a way that matters. When I reflect x->-x, I create a shape that doesn't exist in the real bracket distribution (brackets have a specific handedness due to mounting). The val/test are all "correct handedness." Training on reflected (wrong-handedness) shapes forces the model to be reflection-invariant, but the TRUE mass function over the real (single-handedness) data manifold might use handedness-correlated features. By forcing invariance, we remove useful features → worse. OR reflected shapes are just off-manifold and waste capacity. Hmm, but mass IS physically reflection-invariant. A mirrored bracket has identical mass. So forcing invariance is correct and shouldn't remove useful info... unless the features that predict mass are correlated with handedness in the dataset (spurious correlation), and the model exploits that spurious correlation to do better on val (which shares the same spurious correlation). That would be "cheating" that happens to work on val/test because they're from the same distribution. In that case reflection aug removes the spurious signal and hurts val/test. Given val AND test come from the same generation process (same normalization, same design family), exploiting such correlations is actually fine for this task. So I should NOT force invariances that break them. Lesson: avoid reflection aug. Similarly jitter hurt , because it blurs the precise surface, and the model relies on precise geometry. So augmentation is mostly harmful here. The data is "clean" and consistent; the model should learn the precise mapping. So how to improve generalization without augmentation? The issue is overfitting with consistent bias. Options: A) **Reduce effective capacity / add regularization that improves extrapolation.** E.g., higher weight decay, early stopping, dropout. Let me tune wd more aggressively and add dropout (as training-only, no param change). Actually dropout in the head could help generalization. I can subclass and add functional dropout in forward without changing state_dict keys (dropout has no params). But the deployed model (plain PointNetRegressor) has no dropout , that's fine, dropout is identity at eval. But I train with a modified forward. The state_dict keys match. This is valid! B) **Input jitter via resample helped slightly (0.262 vs 0.275)** , it's a mild regularizer that doesn't distort geometry (just resamples existing points). Keep it. C) **Ensemble distillation won't help** …[truncated]

The model fits training data almost perfectly (train rel_L2 ≈ 0.014) , so this is an **overfitting/generalization** problem, not capacity. And since seeds make correlated errors, the fix is better regularization, not ensembling. Augmentation that distorts geometry (reflect/jitter) removes signal the consistent val/test distribution shares. Let me instrument the trainer to record the val-prediction trajectory across epochs so I can find the CV-optimal epoch and regularization cheaply in one run each. First let me add dropout (training-only, no new params) and trajectory logging.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:     huber=cfg.get('huber',0.0)
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=device)
        for s in range(0,n,bs):
            idx=perm[s:s+bs]
            xb=Xtr_raw[idx]; yb=((ytr[idx]-lmean)/lstd)
            xb=augment(xb,cfg)
            xb=(xb-fmean)/fstd
            opt.zero_grad(set_to_none=True)
            pred=model(xb)
new_str:     huber=cfg.get('huber',0.0)
    drop=cfg.get('dropout',0.0)
    traj=[]; rec_every=cfg.get('rec_every',0)
    def fwd(xb):
        if drop<=0: return model(xb)
        x=xb.transpose(1,2).contiguous()
        x=model.feature(x); x=model.lift(x); x=x.max(dim=2).values
        x=F.dropout(x,p=drop,training=model.training)
        return model.head(x)
    for ep in range(epochs):
        model.train(); perm=torch.randperm(n,device=device)
        for s in range(0,n,bs):
            idx=perm[s:s+bs]
            xb=Xtr_raw[idx]; yb=((ytr[idx]-lmean)/lstd)
            xb=augment(xb,cfg)
            xb=(xb-fmean)/fstd
            opt.zero_grad(set_to_none=True)
            pred=fwd(xb)

output

The file /app/lib_train.py has been edited successfully.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:         if verbose and (ep%max(1,epochs//10)==0 or ep==epochs-1):
new_str:         if rec_every and ((ep+1)%rec_every==0 or ep==epochs-1):
            traj.append((ep, vp.detach().cpu().clone()))
        if verbose and (ep%max(1,epochs//10)==0 or ep==epochs-1):

output

The file /app/lib_train.py has been edited successfully.

str_replace_based_edit_tool

command: str_replace
path: /app/lib_train.py
old_str:     return {'best':best,'best_state':best_state,'swa':swa_res,'swa_model':swa_model,
new_str:     return {'best':best,'best_state':best_state,'swa':swa_res,'swa_model':swa_model,'traj':traj,

output

The file /app/lib_train.py has been edited successfully.

Now let me write a CV script that uses the trajectory to find the optimal epoch across folds (pooled), for different regularization settings. This trains once per fold per config and evaluates all epochs. cv_traj.py: - For each config, for each fold, train with rec_every=20, collect traj (list of (ep, val_pred)). - Pool: for each recorded epoch index, concatenate val_preds across folds and compute rel_l2 vs true. - Report best epoch and its metrics. Note: folds have different sizes? perm[i::K] → sizes differ by at most 1. For pooling per epoch, I need the same epoch indices across folds , they are (same schedule). Good. But the concatenation order must match true labels order per fold. I'll store true per fold. Let me write it.

Now a CV script that finds the CV-optimal epoch from the trajectory in a single training run per fold, across regularization settings.

bash

cd /app && cat > /app/cv_traj.py << 'PY'
import torch, numpy as np, time, sys, json
from lib_train import load_split, train_once, metrics
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0)
N=X.shape[0]
g=torch.Generator().manual_seed(1234); perm=torch.randperm(N,generator=g)
K=6; folds=[perm[i::K] for i in range(K)]

def run(cfg):
    trajs=[]; trues=[]; eps=None
    for k in range(K):
        va=folds[k]; tr=torch.cat([folds[j] for j in range(K) if j!=k])
        r=train_once(X[tr],y[tr],X[va],y[va],cfg)
        trajs.append(r['traj']); trues.append(y[va].numpy().ravel())
        eps=[e for e,_ in r['traj']]
    # pool per epoch index
    best=None
    for ei in range(len(eps)):
        preds=np.concatenate([trajs[k][ei][1].numpy().ravel() for k in range(K)])
        tr=np.concatenate(trues)
        m=metrics(preds,tr)
        if best is None or m[0]<best[1][0]: best=(eps[ei],m)
    return best

if __name__=='__main__':
    base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=400,warmup=5,swa_frac=0.9,seed=0,
              reflect=False,jitter=0.0,resample_p=0.0,scale_jit=0.0,huber=0.0,rec_every=20)
    cfgs=json.loads(sys.argv[1])
    for over in cfgs:
        cfg=dict(base); cfg.update(over)
        t=time.time(); ep,m=run(cfg)
        print('%-55s best_ep=%3d relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(over),ep,m[0],m[1],m[2],time.time()-t),flush=True)
PY
nohup python3 cv_traj.py '[
{"wd":1e-4,"resample_p":0.0},
{"wd":1e-3,"resample_p":0.0},
{"wd":3e-3,"resample_p":0.0},
{"wd":1e-2,"resample_p":0.0},
{"wd":3e-3,"resample_p":0.5},
{"wd":3e-3,"dropout":0.2,"resample_p":0.0}
]' > /app/traj.log 2>&1 &
echo "started $!"

output

started 465

This trains 6 configs × 6 folds × 400 epochs. That's a lot. ~14s/fold for 300ep earlier with no aug. 400 ep ~18s × 6 folds × 6 configs = ~650s. Too long for one wait but it's in background. Let me wait and poll, reading partial results. While waiting, let me reconsider the big picture and whether there's a fundamentally better approach. Key realization: output-ensembling didn't help, meaning seeds are correlated. But that was with the SAME config. The errors might be dominated by a few val samples that are genuinely hard (outliers in shape). Let me look at WHICH samples have high error , maybe a few outliers dominate rel_l2 and maxe. Let me examine per-sample errors from a trained model to understand the error structure. Actually, let me wait for traj.log partial results first. Let me also reconsider: the primary metric is rel_l2 over the FULL set. If a few samples dominate, reducing their error matters most. Ridge had maxe 0.51, PointNet maxe ~0.67-0.74. The PointNet has worse worst-case. Idea: maybe a hybrid , but I can only deploy the PointNet. Another idea: What if the PointNet's systematic error correlates with a simple feature (like volume) that it fails to capture? If so, I could bias-correct... but I can't post-process at inference (grader does fixed de-normalization). So no. Wait, can I? The grader computes pred = model(norm_x) * label_std + label_mean. I control label_std, label_mean, and the model weights. I cannot add a separate correction term. So the model itself must be good. Let me reconsider making the PointNet capture volume. The issue might be that with max-pooling, volume (a count-like quantity) isn't naturally represented. But I showed it FITS training perfectly, so it CAN represent whatever distinguishes training samples. The problem is generalization. So, pure generalization improvement. Let me think about what regularizer best improves point-cloud regression generalization with tiny data: - Weight decay (tuning now) - Dropout (tuning now) - Data augmentation that stays ON-manifold. The challenge: reflect/jitter go off-manifold. What about resample (on-manifold, helped slightly)? - Label smoothing? Not for regression. - Mixup! Mixup for point clouds: interpolate between two point clouds? Hard for sets. But "PointMixup" exists. Alternatively, mixup in a simpler sense doesn't apply to coordinates directly. Actually, there's a clever augmentation that stays on-manifold and is label-consistent: since all brackets share the SAME mounting interface and are normalized the same way, maybe small perturbations... no. Let me reconsider reflection more carefully. It hurt at wd=1e-4. But maybe with the RIGHT regularization, forcing reflection-invariance is actually good for generalization (it's a true physical symmetry of mass). The earlier test conflated things. But physically, mass IS reflection invariant, so a reflection-invariant model is CORRECT and should generalize better IF val/test don't rely on handedness spurious correlations. The fact it hurt suggests val/test DO share handedness info. But handedness can't affect true mass... unless the point SAMPLING or normalization encodes something. Hmm, actually here's a real possibility: the brackets are NOT reflection symmetric, and reflecting creates shapes far from any real bracket. The model, forced to predict the same mass for real and mirrored shapes, must learn a more "averaged" function, which increases bias on the real manifold. Since test is on the real manifold, this hurts. So reflection = off-manifold augmentation here. Avoid. Confirmed reasoning. OK so my best bet is regularization tuning + optimal early stopping. Let me also consider: combine train+val for final (132 samples) , more data always helps generalization. For the final model I'll use all 132. My CV estimates generalization for the recipe; final model trained on all 132 should be slightly better than CV folds (which train on 110). Let me wait for traj results.

This sweep trains 6 configs × 6 folds. Let me wait and poll. Meanwhile, let me examine the per-sample error structure , whether a few hard outliers dominate the relative-L2 and worst-case error.

bash

cd /app && sleep 150 && cat traj.log

output

{'wd': 0.0001, 'resample_p': 0.0}                       best_ep=119 relL2=0.2564 mae=0.1594 maxe=0.7497 (72s)
{'wd': 0.001, 'resample_p': 0.0}                        best_ep=119 relL2=0.2655 mae=0.1646 maxe=0.8156 (63s)

With optimal early stopping (~120 epochs), wd=1e-4 gives 0.256 (no aug). wd=1e-3 slightly worse. Let me wait for the rest (higher wd, dropout, resample). So best-epoch early stopping helps a bit (0.256 vs 0.275 at fixed 150). Let me keep waiting. Let me think about the per-sample error analysis while waiting. Let me run it in parallel in a separate quick process? The GPU is busy but can handle it. Actually let me just wait to avoid interfering. Let me wait for more results.

bash

cd /app && sleep 180 && cat traj.log

output

{'wd': 0.0001, 'resample_p': 0.0}                       best_ep=119 relL2=0.2564 mae=0.1594 maxe=0.7497 (72s)
{'wd': 0.001, 'resample_p': 0.0}                        best_ep=119 relL2=0.2655 mae=0.1646 maxe=0.8156 (63s)
{'wd': 0.003, 'resample_p': 0.0}                        best_ep=119 relL2=0.2602 mae=0.1668 maxe=0.7474 (69s)
{'wd': 0.01, 'resample_p': 0.0}                         best_ep=119 relL2=0.2715 mae=0.1682 maxe=0.7747 (61s)

Weight decay doesn't help much; all around 0.256-0.27. Let me wait for the dropout and resample results. But it seems like regularization tuning isn't moving the needle much. The PointNet is stuck ~0.25-0.27. I'm now fairly convinced the architecture+input constraints cap this around 0.25 with straightforward training, while ridge on volume features gets 0.224. Let me think about a fundamentally better idea that respects constraints: **Key insight**: I can precompute feat_mean/feat_std however I want (they're just applied as (x-mean)/std). But what if the per-axis std-based normalization isn't ideal and a different LINEAR transform helps? No, it's constrained to per-axis diagonal. Minor. **Bigger idea , distillation from ridge/voxel model into the PointNet.** Even though PointNet errors are correlated across seeds, that's under the SAME training signal (true labels). If I instead train the PointNet to regress a BETTER target , e.g., the ridge's volume-feature-based prediction blended with true labels , I could inject the volume knowledge. But the PointNet still needs to compute something correlated with volume from points. It fits train perfectly already, so it can fit any target on train; the question is generalization. If I train it to predict the ridge's SMOOTH function (which generalizes at 0.224), the PointNet might inherit that smoother mapping and generalize better! This is "distilling a well-generalizing simple model into the neural net." The neural net, trained to match ridge predictions (which are smooth in feature space), might generalize better than when trained on raw noisy labels. But the PointNet would at best match ridge (0.224), probably worse due to approximation. Not clearly better. Alternatively, train on true labels but ensemble PointNet + ridge? Can't deploy ridge. **Another idea: augment the dataset using the ridge model as an oracle on perturbed point clouds.** Generate many augmented point clouds (reflections, jitters, resamples), label them with... true label (known, since mass invariant under reflection/jitter/resample). This gives more (cloud, mass) pairs. But reflection/jitter hurt (off-manifold). Resample stays on-manifold. So augmentation is limited. Hmm. Let me reconsider WHY reflect/jitter hurt , maybe it's because they're applied too strongly or because BN stats get confused. Actually let me revisit: maybe MILD jitter (0.003) or reflection ONLY on the most-symmetric axis (y, IoU 0.76) helps. The y-axis is the long axis and most symmetric. Let me test reflect-y-only. But these are second-order. Let me wait for dropout/resample results first, then decide. Let me also reconsider: maybe I should just accept ~0.24-0.25 and maximize robustness. But I'd like to beat ridge. Actually, let me reconsider the possibility that combining PointNet with volume info IS possible through the input. The grader feeds (x - feat_mean)/feat_std where x is [B,2048,3]. I cannot change dimensionality. BUT , what if feat_std is set such that... no. Let me reconsider the architecture once more: ``` feature: Conv1d(3->64,k1),BN,GELU x3 lift: Conv1d(64->128),BN,GELU; Conv1d(128->256),BN,GELU max over points head: Linear(256->128),GELU,Linear(128->64),GELU,Linear(64->1) ``` Max-pool over 256 channels. Each channel = max over points of a learned function of (x,y,z) (composition of per-point MLPs , note: all layers are per-point since kernel=1, so it's a per-point MLP from R^3 to R^256, then max-pool). So the global feature = max over points of phi(point), a 256-dim per-point embedding. This captures the "extremes" of each embedding dimension over the point set. This can represent things like bounding extents in learned directions, and presence/absence of points in learned regions (via ReLU-like detectors: a channel that's high only for points in region R → max is high if any point in R). So it can build a coarse occupancy signature → approximate volume. With 256 channels, ~256 region detectors. It CAN approx …[truncated]

Weight decay barely moves the needle (all ~0.26). Let me see the dropout and resample results, then examine the error structure.

bash

cd /app && sleep 140 && cat traj.log

output

{'wd': 0.0001, 'resample_p': 0.0}                       best_ep=119 relL2=0.2564 mae=0.1594 maxe=0.7497 (72s)
{'wd': 0.001, 'resample_p': 0.0}                        best_ep=119 relL2=0.2655 mae=0.1646 maxe=0.8156 (63s)
{'wd': 0.003, 'resample_p': 0.0}                        best_ep=119 relL2=0.2602 mae=0.1668 maxe=0.7474 (69s)
{'wd': 0.01, 'resample_p': 0.0}                         best_ep=119 relL2=0.2715 mae=0.1682 maxe=0.7747 (61s)
{'wd': 0.003, 'resample_p': 0.5}                        best_ep=299 relL2=0.2583 mae=0.1599 maxe=0.8394 (120s)
{'wd': 0.003, 'dropout': 0.2, 'resample_p': 0.0}        best_ep=179 relL2=0.2774 mae=0.1783 maxe=0.8049 (89s)

Dropout hurt (0.277). Resample with wd3e-3 at ep299: 0.258. Nothing beats the wd=1e-4 no-aug early-stopped 0.256. So the PointNet is firmly stuck at ~0.256 CV, vs ridge 0.224. Let me now seriously pursue DISTILLATION: train the PointNet to match a well-generalizing target (ridge predictions on volume features), possibly blended with true labels. The hypothesis: the ridge provides a smooth, well-generalizing function; if the PointNet learns to approximate it (plus true labels), it may generalize better than learning raw labels. But actually, a cleaner framing: The ridge uses hand-crafted volume features the PointNet struggles to infer from points. If I train the PointNet on a TARGET that's a blend α·ridge_pred + (1-α)·true_label, the ridge_pred is computable from the point cloud (I compute features then ridge). At inference, though, the PointNet only sees points , it must have learned to replicate. Since ridge_pred IS a function of the point cloud (via features), the PointNet CAN learn it (it fits train perfectly). The benefit: ridge_pred is a denoised/smoothed label that generalizes. However, the PointNet learning to match ridge on TRAIN then generalizing is still limited by PointNet's own generalization of that function. Might not beat ridge. Let me test it empirically via CV: - In each fold, fit ridge on train features, get ridge preds for train and val. - Train PointNet to regress blend target on train. - Evaluate PointNet on val (its own forward). Compare to ridge-alone and pointnet-alone. Actually, the cleaner and likely-better approach: **ensemble PointNet and ridge by distillation** , but let me first check the simplest thing: is the PointNet's error correlated or complementary to ridge's error? If complementary, an average of PointNet+ridge predictions would be much better. Even though I can only deploy the PointNet, if avg is much better, then distilling that average into the PointNet is worthwhile. Let me check correlation of errors: compute CV out-of-fold predictions for both ridge and a PointNet, then see if averaging helps. This is the crucial experiment. Let me produce out-of-fold predictions for both models on the 132 samples and analyze. Let me write this. I'll train one PointNet per fold (best config: wd1e-4, ep~120, no aug), get OOF preds, and compute ridge OOF preds, then evaluate individual + average + check per-sample. Let me do it.

Dropout hurt; nothing beats the simple wd=1e-4 early-stopped model at ~0.256. The PointNet plateaus above ridge (0.224). The decisive question: are PointNet errors *complementary* to ridge's? If averaging the two helps a lot, distilling that blend into the PointNet is worthwhile. Let me compute out-of-fold predictions for both and analyze.

bash

cd /app && cat > /app/oof.py << 'PY'
import torch, numpy as np
from lib_train import load_split, train_once, metrics
Xtr,ytr=load_split('train'); Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0); y=torch.cat([ytr,yva],0); yv=y.numpy().ravel()
N=X.shape[0]
g=torch.Generator().manual_seed(1234); perm=torch.randperm(N,generator=g)
K=6; folds=[perm[i::K] for i in range(K)]
def make_feats(f):
    out=[];mn=f.amin(1);mx=f.amax(1);ext=mx-mn;std=f.std(1);mean=f.mean(1);out+=[ext,std,mean]
    fc=f-f.mean(1,keepdim=True);cov=torch.einsum('npi,npj->nij',fc,fc)/f.shape[1]
    out.append(torch.linalg.eigvalsh(cov))
    for R in (8,16):
        idx=((f+0.5).clamp(0,0.999999)*R).long();lin=idx[...,0]*R*R+idx[...,1]*R+idx[...,2]
        occ=torch.zeros(f.shape[0],R*R*R)
        for n in range(f.shape[0]): occ[n,torch.unique(lin[n])]=1.0
        out.append(occ.sum(1,keepdim=True)/(R*R*R))
    return torch.cat(out,1).numpy()
Xf=make_feats(X)
oof_pn=np.zeros(N); oof_rg=np.zeros(N)
cfg=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=120,warmup=5,swa_frac=0.9,seed=0,
         reflect=False,jitter=0.0,resample_p=0.0,scale_jit=0.0,huber=0.0)
for k in range(K):
    va=folds[k].numpy(); tr=torch.cat([folds[j] for j in range(K) if j!=k]).numpy()
    # ridge
    mu=Xf[tr].mean(0);sd=Xf[tr].std(0)+1e-8
    A=np.c_[(Xf[tr]-mu)/sd,np.ones(len(tr))]; B=np.c_[(Xf[va]-mu)/sd,np.ones(len(va))]
    w=np.linalg.solve(A.T@A+1.0*np.eye(A.shape[1]),A.T@yv[tr]); oof_rg[va]=B@w
    # pointnet
    r=train_once(X[torch.tensor(tr)],y[torch.tensor(tr)],X[torch.tensor(va)],y[torch.tensor(va)],cfg)
    m=r['final_model'];fm=r['fmean'].cuda();fs=r['fstd'].cuda();lm=r['lmean'].cuda();ls=r['lstd'].cuda();m.eval()
    with torch.no_grad(): oof_pn[va]=(m((X[torch.tensor(va)].cuda()-fm)/fs)*ls+lm).cpu().numpy().ravel()
np.save('/app/oof_pn.npy',oof_pn); np.save('/app/oof_rg.npy',oof_rg); np.save('/app/oof_y.npy',yv)
print('PointNet OOF  relL2=%.4f mae=%.4f maxe=%.4f'%metrics(oof_pn,yv))
print('Ridge    OOF  relL2=%.4f mae=%.4f maxe=%.4f'%metrics(oof_rg,yv))
err_pn=oof_pn-yv; err_rg=oof_rg-yv
print('err corr PN vs RG:',np.corrcoef(err_pn,err_rg)[0,1])
for a in [0.3,0.4,0.5,0.6,0.7]:
    bl=a*oof_pn+(1-a)*oof_rg
    print('blend a=%.1f relL2=%.4f mae=%.4f maxe=%.4f'%(a,*metrics(bl,yv)))
PY
nohup python3 oof.py > /app/oof.log 2>&1 &
echo "started $!"

output

started 504

bash

cd /app && sleep 120 && cat oof.log

output

PointNet OOF  relL2=0.2800 mae=0.1777 maxe=0.8995
Ridge    OOF  relL2=0.2240 mae=0.1508 maxe=0.5448
err corr PN vs RG: 0.5500745809405904
blend a=0.3 relL2=0.2147 mae=0.1399 maxe=0.6326
blend a=0.4 relL2=0.2171 mae=0.1409 maxe=0.6708
blend a=0.5 relL2=0.2222 mae=0.1431 maxe=0.7089
blend a=0.6 relL2=0.2298 mae=0.1467 maxe=0.7470
blend a=0.7 relL2=0.2396 mae=0.1526 maxe=0.7851

Error correlation PN vs RG = 0.55 (moderately correlated, but not fully). Blending helps: a=0.3 (30% PointNet, 70% ridge) gives relL2=0.2147, better than ridge alone (0.224) and much better than PointNet (0.28). So the PointNet and ridge are partially complementary. A blend at ~0.3 PointNet gives 0.215. But I can only deploy the PointNet! So I need the PointNet to REPLICATE the blend (0.3·PN + 0.7·ridge). Via distillation: train the PointNet to predict the blend target. But the blend includes the PointNet's own prediction... circular. Better: The deployable model must be a single PointNet. The best achievable is to distill the RIDGE (or the blend of an independent teacher) into the PointNet. But distilling ridge into PointNet: the PointNet learns to mimic ridge. Since PointNet fits train perfectly, on train it matches ridge exactly; on val it generalizes its approximation. Would it generalize as well as ridge (0.224)? Possibly worse, since PointNet's generalization of the mimicked function may differ. Alternatively, distill the BLEND where the teacher's PointNet-component is a SEPARATE, already-trained PointNet (teacher), and the student PointNet learns to match (0.3·teacherPN + 0.7·ridge). The student then approximates a 0.215-quality function. If the student generalizes this target well, it could reach ~0.22. Hmm, but this is getting complicated and the gain is modest (0.224 → 0.215). Let me reconsider. Actually, simpler and possibly better: **distill ridge alone into the PointNet using heavy augmentation.** Here's the key insight: ridge is a function of hand-crafted features (extent, std, occupancy) that are computable from ANY point cloud, including augmented/reflected/jittered ones. The ridge's prediction is smooth and generalizes. If I generate a HUGE augmented dataset (reflections, jitters, resamples, even random rotations) and label each augmented cloud with the RIDGE prediction computed on that augmented cloud, the PointNet can learn to replicate ridge across a much broader input distribution. This gives the PointNet effectively unlimited training data with a consistent teacher, which should let it approximate ridge very well and generalize like ridge. Wait, but ridge computed on reflected cloud = same as original (features are reflection-invariant: extent, std, occupancy all invariant to reflection). So reflection augmentation with ridge labels teaches reflection-invariance consistently , good. Jitter changes features slightly, ridge recomputes accordingly , teaches robustness. This could work well! But the end result is bounded by ridge quality (~0.224) plus the blend benefit is lost. Actually the student mimicking ridge would at best hit ~0.224, likely a bit worse. The blend (0.215) needs the PointNet's independent signal. Let me reconsider the target. The blend 0.3·PN+0.7·RG=0.215. What if I distill this blend, using a FIXED teacher PN (trained on true labels) and ridge? The student learns target T = 0.3·PN_teacher(cloud) + 0.7·ridge(cloud), both computable on any augmented cloud. Student approximates T. Student's OWN generalization of T might land near 0.22. It's bounded by how well student mimics T and how well T generalizes (0.215 OOF). Actually, there's an even better approach to combine: since I can compute ridge features from the point cloud, and the PointNet also processes the point cloud, what if the teacher is the BEST possible blend and I make the student match it. But the fundamental ceiling is the teacher quality. Let me reconsider the ceiling. The OOF blend 0.215 is the realistic ceiling for a PN+ridge combo. Can I get a better teacher? Let me improve the teacher: 1. Better ridge features (more volume descriptors, nonlinear). 2. Better feature model (e.g., gradient boosting , but no sklearn; I can implement a simple one, or use more features + ridge with interactions). 3. Ensemble of diverse feature-based models. If I can push the feature-based model to ~0.18-0.20, then the distilled P …[truncated]

Key result: PointNet errors are only partially correlated with ridge (r=0.55), and a blend (0.3·PN + 0.7·ridge) reaches **0.215 OOF** , beating both. Since I can only deploy the PointNet, the path forward is **distillation**: build the strongest feature-based teacher, then train the PointNet to replicate it (with augmentation providing unlimited teacher-labeled data). First, let me strengthen the teacher with richer features.

bash

cd /app && cat > /app/feat.py << 'PY'
import torch, numpy as np
from lib_train import load_split, metrics
def rich_feats(f, res=(4,6,8,12,16), grids=(6,8)):
    N=f.shape[0]; out=[]
    mn=f.amin(1);mx=f.amax(1);ext=mx-mn;std=f.std(1);mean=f.mean(1)
    out+=[ext,std,mean,mean.abs()]
    # quantiles per axis
    qs=torch.tensor([0.1,0.25,0.5,0.75,0.9])
    for ax in range(3):
        out.append(torch.quantile(f[:,:,ax],qs.to(f),dim=1).T)  # [N,5]
    # PCA eigenvalues
    fc=f-f.mean(1,keepdim=True);cov=torch.einsum('npi,npj->nij',fc,fc)/f.shape[1]
    ev=torch.linalg.eigvalsh(cov); out.append(ev)
    out.append((ev[:,:1]/ (ev.sum(1,keepdim=True)+1e-9)))  # anisotropy
    # radial distance stats
    rad=f.norm(dim=2); out.append(torch.stack([rad.mean(1),rad.std(1),rad.amax(1)],1))
    # occupancy fraction at multiple res
    for R in res:
        idx=((f+0.5).clamp(0,0.999999)*R).long();lin=idx[...,0]*R*R+idx[...,1]*R+idx[...,2]
        occ=torch.zeros(N,R*R*R)
        for n in range(N): occ[n,torch.unique(lin[n])]=1.0
        out.append(occ.sum(1,keepdim=True)/(R*R*R))
    # full occupancy grids (spatial) at small res
    for R in grids:
        idx=((f+0.5).clamp(0,0.999999)*R).long();lin=idx[...,0]*R*R+idx[...,1]*R+idx[...,2]
        occ=torch.zeros(N,R*R*R)
        for n in range(N): occ[n,torch.unique(lin[n])]=1.0
        out.append(occ)
    return torch.cat(out,1).numpy()

def ridge_oof(Xf,yv,folds,lam):
    N=len(yv);oof=np.zeros(N)
    for k in range(len(folds)):
        va=folds[k].numpy();tr=np.concatenate([folds[j].numpy() for j in range(len(folds)) if j!=k])
        mu=Xf[tr].mean(0);sd=Xf[tr].std(0)+1e-8
        A=np.c_[(Xf[tr]-mu)/sd,np.ones(len(tr))];B=np.c_[(Xf[va]-mu)/sd,np.ones(len(va))]
        w=np.linalg.solve(A.T@A+lam*np.eye(A.shape[1]),A.T@yv[tr]);oof[va]=B@w
    return oof

if __name__=='__main__':
    Xtr,ytr=load_split('train');Xva,yva=load_split('val')
    X=torch.cat([Xtr,Xva],0);y=torch.cat([ytr,yva],0);yv=y.numpy().ravel()
    N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
    K=6;folds=[perm[i::K] for i in range(K)]
    Xf=rich_feats(X); print('feat dim',Xf.shape[1])
    np.save('/app/richfeat.npy',Xf)
    for lam in [1,3,10,30,100]:
        oof=ridge_oof(Xf,yv,folds,lam)
        print('lam=%5.0f relL2=%.4f mae=%.4f maxe=%.4f'%(lam,*metrics(oof,yv)))
PY
python3 feat.py

output

feat dim 767
lam=    1 relL2=0.2783 mae=0.1785 maxe=0.8402
lam=    3 relL2=0.2677 mae=0.1743 maxe=0.8191
lam=   10 relL2=0.2610 mae=0.1744 maxe=0.6979
lam=   30 relL2=0.3062 mae=0.2172 maxe=0.7722
lam=  100 relL2=0.4893 mae=0.3724 maxe=1.0388

The full occupancy grids (767 dims) made ridge WORSE (0.26 best) , too many dims, overfitting. The simpler feature set (earlier, ~20 dims) was better (0.224). So high-dim spatial grids hurt ridge. Let me go back to the compact, informative features and tune. The earlier compact set (ext, std, mean, PCA eigenvalues, occupancy fraction at 8,16) gave 0.224. Let me refine that compact set: add quantiles, radial, anisotropy, more occupancy-fraction resolutions, but NOT full grids. Let me test. Let me modify to drop full grids and keep compact rich features.

The 767-dim grids overfit ridge. The compact feature set worked better (0.224). Let me refine a *compact* rich feature set (no full grids) and tune.

bash

cd /app && python3 -c "
import torch, numpy as np
from feat import rich_feats, ridge_oof
from lib_train import load_split, metrics
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);y=torch.cat([ytr,yva],0);yv=y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
# compact: no full grids
Xf=rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=())
print('dim',Xf.shape[1])
np.save('/app/compactfeat.npy',Xf)
for lam in [0.3,1,2,3,5,10]:
    oof=ridge_oof(Xf,yv,folds,lam)
    print('lam=%5.1f relL2=%.4f mae=%.4f maxe=%.4f'%(lam,*metrics(oof,yv)))
"

output

dim 43
lam=  0.3 relL2=0.2347 mae=0.1615 maxe=0.5321
lam=  1.0 relL2=0.2290 mae=0.1603 maxe=0.5360
lam=  2.0 relL2=0.2246 mae=0.1571 maxe=0.5239
lam=  3.0 relL2=0.2223 mae=0.1548 maxe=0.5114
lam=  5.0 relL2=0.2207 mae=0.1519 maxe=0.4900
lam= 10.0 relL2=0.2259 mae=0.1546 maxe=0.4980

Compact rich features (43 dims) with lam=5 gives relL2=0.2207, mae=0.152, maxe=0.490 , slightly better than before (0.224) and notably better maxe. Now the teacher is ~0.22. The PointNet OOF was 0.28. The blend (0.3 PN + 0.7 ridge) gave 0.215. Let me recompute the blend with this improved ridge and the PointNet OOF I saved. Let me also consider a teacher = blend of ridge + PN_teacher. But for distillation I want the best possible teacher function computable on any cloud. Actually, let me reconsider the distillation target. The teacher I distill must be computable on arbitrary (augmented) clouds. Both ridge (on features) and a trained PN_teacher are computable. The blend 0.3·PN+0.7·ridge gave 0.215 OOF. But if I distill this blend into a NEW student PN, the student is itself a PN , so effectively the final model = student PN that mimics (0.3·PN_teacher + 0.7·ridge). But here's a subtlety: why not just distill ridge (0.22) and also ADD the student's own capacity? The distillation student minimizes distance to teacher; it won't exceed teacher quality. So the teacher quality is the ceiling. Best teacher I have is the blend (0.215). Let me improve the teacher blend further: - Better ridge (0.2207). - Better PN teacher? PN OOF 0.28. - Optimal blend weight with improved ridge. Let me also consider blending MULTIPLE feature models (ridge at different lambdas/feature subsets) , but they're correlated. Let me reconsider: maybe I should distill ONLY the ridge (simpler, robust, 0.22) and see if the student PN can match it. If the student reaches ~0.22, that already beats plain PN (0.256). Then optionally blend teacher with PN. Actually, here's a cleaner and likely-better idea for the TEACHER: instead of PN trained on true labels (0.28), use the ridge as teacher but let the student ALSO see true labels (multi-task: match teacher + match true label). This combines ridge's smoothness with true-label signal. The student might then generalize between ridge (0.22) and pushing toward truth. This is like the blend but learned. Let me just distill and experiment. Let me reconsider the whole plan efficiency. Given tokens/time, let me commit to: **Teacher T = 0.35·PN_teacher + 0.65·ridge** (or tuned), computable on any cloud. **Student**: PN trained to regress T on train clouds + augmentations, maybe + small weight on true labels. But building PN_teacher per fold for the teacher, plus student per fold, is 2× training. Doable. Actually, let me simplify and maximize robustness. Let me first test: distill ridge-only teacher into student PN, via CV, with augmentation. See what OOF the student gets. If student ≈ 0.22, good. Then test adding true-label term and PN_teacher blend. Let me reconsider augmentation for distillation. The teacher (ridge) is a function of features. For augmented clouds, I recompute teacher features → teacher label. Reflection: features are reflection-invariant (extent, std abs? mean is NOT reflection invariant , mean flips sign; I used mean and mean.abs(). mean would flip. But ridge uses mean with some weight. For a reflected cloud, true mass is same, ridge computes mean→flipped sign→different ridge prediction. That's teacher inconsistency. To keep teacher consistent under reflection, I should either (a) use reflection-invariant features only, or (b) recompute teacher on the augmented cloud so student matches whatever teacher says (self-consistent, even if teacher varies). For distillation, (b) is fine , student mimics teacher exactly as teacher behaves. But if teacher is inconsistent under reflection (predicts different mass for mirrored cloud), that's teacher noise. Better to make teacher reflection-invariant by using |mean| or dropping signed mean. Actually signed mean had decent correlation (mean_x corr -0.40). Hmm, but that's likely spurious (handedness). For generalization to test (same handedness), signed mean might help. This is getting subtle. Let me step back and be pragmatic: For distillation augmentatio …[truncated]

I realize a subtlety: a teacher fit on the fold-train set predicts those same train clouds almost perfectly, so naive distillation on train points just reproduces the true labels (no gain). The gain must come from training the student on a *broader* set of clouds (augmented copies) with teacher labels **recomputed** per copy , transferring the ridge's smooth, well-generalizing function. Let me implement and CV-test this.

bash

cd /app && cat > /app/distill.py << 'PY'
import torch, numpy as np, time, sys, json, copy
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn
from feat import rich_feats
from candidate_model import build_model
from lib_train import load_cfg
dev='cuda'
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]

def fit_ridge(Xf_tr, y_tr, lam=5.0):
    mu=Xf_tr.mean(0);sd=Xf_tr.std(0)+1e-8
    A=np.c_[(Xf_tr-mu)/sd,np.ones(len(Xf_tr))]
    w=np.linalg.solve(A.T@A+lam*np.eye(A.shape[1]),A.T@y_tr)
    return (mu,sd,w)
def ridge_pred(rg,Xf):
    mu,sd,w=rg; B=np.c_[(Xf-mu)/sd,np.ones(len(Xf))]; return B@w

def aug_cloud(c, reflect=True, jitter=0.006, resample=True):
    # c: [P,3]
    P=c.shape[0]; out=c.clone()
    if reflect:
        s=(torch.randint(0,2,(1,3),device=c.device).float()*2-1); out=out*s
    if resample:
        idx=torch.randint(0,P,(P,),device=c.device); out=out[idx]
    if jitter>0: out=out+torch.randn_like(out)*jitter
    return out

def build_distill_set(clouds, rg, n_copies, aug_kw):
    # clouds: [M,P,3] on gpu; returns Xaug [M*n,P,3] cpu, teacher labels [M*n,1]
    Xs=[]; 
    for _ in range(n_copies):
        batch=torch.stack([aug_cloud(clouds[i],**aug_kw) for i in range(clouds.shape[0])],0)
        Xs.append(batch)
    Xaug=torch.cat(Xs,0)
    Xf=rich_feats(Xaug.cpu())
    tl=ridge_pred(rg,Xf).astype(np.float32)
    return Xaug.cpu(), torch.tensor(tl).unsqueeze(1)

def train_student(Xd, Yd, Xva_e, yva_e, Xtrue=None, Ytrue=None, lam_true=0.0,
                  epochs=120, lr=1e-3, wd=1e-4, bs=64, seed=0):
    torch.manual_seed(seed); np.random.seed(seed)
    fmean=Xd.reshape(-1,3).mean(0).to(dev); fstd=Xd.reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Yd.mean(0).to(dev); lstd=Yd.std(0).clamp_min(1e-6).to(dev)
    Xd=Xd.to(dev); Yd=((Yd.to(dev)-lmean)/lstd)
    if Xtrue is not None: Xtrue=Xtrue.to(dev); Ytrue=((Ytrue.to(dev)-lmean)/lstd)
    model=build_model(load_cfg()).to(dev)
    opt=torch.optim.AdamW(model.parameters(),lr=lr,weight_decay=wd)
    sched=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:0.5*(1+np.cos(np.pi*e/epochs)))
    n=Xd.shape[0]; best=(1e9,)
    for ep in range(epochs):
        model.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs]
            xb=(Xd[idx]-fmean)/fstd; yb=Yd[idx]
            opt.zero_grad(set_to_none=True)
            loss=F.mse_loss(model(xb),yb)
            if lam_true>0 and Xtrue is not None:
                ti=torch.randint(0,Xtrue.shape[0],(min(bs,Xtrue.shape[0]),),device=dev)
                loss=loss+lam_true*F.mse_loss(model((Xtrue[ti]-fmean)/fstd),Ytrue[ti])
            loss.backward();opt.step()
        sched.step()
    model.eval()
    with torch.no_grad(): p=(model((Xva_e.to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
    return p, model, (fmean,fstd,lmean,lstd)

if __name__=='__main__':
    cfg=json.loads(sys.argv[1]) if len(sys.argv)>1 else {}
    n_copies=cfg.get('n_copies',20); lam_true=cfg.get('lam_true',0.0)
    aug_kw=dict(reflect=cfg.get('reflect',True),jitter=cfg.get('jitter',0.006),resample=cfg.get('resample',True))
    oof=np.zeros(N)
    t0=time.time()
    for k in range(K):
        va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
        Xf_tr=rich_feats(X[tr]); rg=fit_ridge(Xf_tr,Y[tr].numpy().ravel(),lam=cfg.get('lam',5.0))
        Xd,Yd=build_distill_set(X[tr].to(dev),rg,n_copies,aug_kw)
        p,_,_=train_student(Xd,Yd,X[va],Y[va].numpy().ravel(),
                            Xtrue=X[tr],Ytrue=Y[tr],lam_true=lam_true,
                            epochs=cfg.get('epochs',120),wd=cfg.get('wd',1e-4),seed=0)
        oof[va]=p
    print('DISTILL %s relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(cfg),*metrics(oof,yv),time.time()-t0),flu
…[truncated]

output

started 551

This trains a student to mimic ridge on augmented clouds. Let me wait for results. The build_distill_set computes rich_feats on 20×110=2200 clouds per fold, which includes occupancy python loops over 2200 samples × 9+2 resolutions , could be slow. Let me wait and check. While waiting, let me reconsider: if the student mimics ridge well, OOF should approach ridge's 0.22. If it's better (because the student also has access to raw geometry and the broad augmentation), maybe <0.22. If lam_true>0 later, it can blend toward truth. Let me wait ~60s and check progress.

Let me wait for the distillation CV and check progress.

bash

cd /app && sleep 90 && cat distill.log && echo "--- still running? ---" && ps aux|grep distill.py|grep -v grep|wc -l

output

--- still running? ---
1

bash

cd /app && sleep 120 && cat distill.log && echo "--- running? ---" && ps aux|grep distill.py|grep -v grep|wc -l

output

--- running? ---
1

Still running after ~3.5 min and no output. The feature computation on 2200 clouds × 6 folds might be slow. Let me check where it's spending time. Let me wait a bit more but also consider that rich_feats occupancy loop is the bottleneck. Let me check by timing rich_feats on 2200 clouds. Actually let me just wait a bit more , first fold might take a while.

bash

cd /app && sleep 120 && cat distill.log && echo "--- running? ---" && ps aux|grep distill.py|grep -v grep|wc -l

output

--- running? ---
1

Over 5.5 minutes and no output for even the first fold. The feature computation is likely the bottleneck (rich_feats with occupancy python loops over 2200 clouds, and quantile computation on GPU/CPU). Let me check. Let me profile rich_feats quickly in a separate process. Actually the running process is using GPU. Let me just time rich_feats on CPU for 2200 clouds. Actually, let me kill it and profile, because waiting blindly wastes time.

No output after 5+ minutes , feature computation on 2200 clouds per fold is likely the bottleneck. Let me profile `rich_feats` to confirm.

bash

cd /app && kill 551 2>/dev/null; sleep 2; python3 -c "
import torch, time
from feat import rich_feats
from lib_train import load_split
X,_=load_split('train')
c=X[:110]
import time
t=time.time(); f=rich_feats(c); print('110 clouds: %.1fs dim %d'%(time.time()-t, f.shape[1]))
big=c.repeat(20,1,1); print('big',big.shape)
t=time.time(); f=rich_feats(big); print('2200 clouds: %.1fs'%(time.time()-t))
"

output

110 clouds: 0.1s dim 767
big torch.Size([2000, 2048, 3])
2200 clouds: 2.9s
[stderr]
[5]+  Terminated              nohup python3 distill.py '{"n_copies":20,"lam_true":0.0,"reflect":true,"jitter":0.006}' > /app/distill.log 2>&1

rich_feats is fast (2.9s for 2200). Wait , it returned dim 767?? That means the default rich_feats still uses grids=(6,8). In distill.py I called rich_feats(Xaug.cpu()) with DEFAULT args (res=(4,6,8,12,16), grids=(6,8)) → 767 dims. But fit_ridge was fit on rich_feats(X[tr]) also default (767 dims). So the teacher is the 767-dim ridge (0.26), NOT the good 43-dim one (0.22)! Bug: I used default features everywhere, which is the WORSE feature set. But that still shouldn't make it hang. The feature computation is fast. So why no output? Let me reconsider , maybe the training (120 epochs × 2200 samples, bs64 → ~35 batches/epoch × 120 = 4200 steps/fold × 6 folds) plus the lam_true path. That should be fast on H100 (~30s/fold). 6 folds ~3 min. Plus features. Should've finished by ~4 min. It ran 5.5 min... maybe it was just slow and close to done. Let me not guess. Let me fix the feature set to the good compact one (43-dim, lam=5 → 0.22) and reduce epochs, and rerun with timing per fold printed. Let me modify distill.py to use compact features and print per-fold progress. Let me patch: define a compact feature wrapper.

`rich_feats` is fast, but I spotted a bug: it defaulted to the 767-dim grid features (the *worse* 0.26 teacher) instead of the compact 43-dim set (0.22). Let me fix distill.py to use the good compact features and add per-fold progress logging.

bash

cd /app && python3 - << 'PY'
import re
s=open('distill.py').read()
s=s.replace("from feat import rich_feats","from feat import rich_feats as _rf\ndef rich_feats(x):\n    return _rf(x, res=(3,4,5,6,8,10,12,16,20), grids=())")
# add per-fold timing print
s=s.replace("        oof[va]=p\n","        oof[va]=p\n        print('  fold %d done relL2(fold)=%.4f t=%.0fs'%(k, (np.sqrt(((p-Y[va].numpy().ravel())**2).sum()/ (Y[va].numpy().ravel()**2).sum())), time.time()-t0),flush=True)\n")
open('distill.py','w').write(s)
print("patched")
PY
nohup python3 distill.py '{"n_copies":20,"lam_true":0.0,"reflect":true,"jitter":0.006,"lam":5.0,"epochs":100}' > /app/distill.log 2>&1 &
echo "started $!"

output

patched
started 614

bash

cd /app && sleep 120 && cat distill.log

output

fold 0 done relL2(fold)=0.2948 t=57s
  fold 1 done relL2(fold)=0.4609 t=94s

The distilled student is doing BADLY (fold relL2 0.29, 0.46) , worse than ridge (0.22) and even worse than plain PN. And it's slow (~40s/fold). So distilling ridge on reflected+jittered+resampled clouds does NOT transfer well. The student mimics ridge poorly on val. Why? - The reflection augmentation flips the signed mean feature, making teacher labels inconsistent (teacher gives different mass for mirrored clouds since it uses signed mean/quantiles). This injects noise. - The student sees augmented clouds (reflected) but val clouds are NOT reflected → distribution mismatch. - Fundamentally, the student must learn ridge's function from points; it's hard. This distillation approach is underperforming. Let me abandon it , it's not working and is slow. Let me step back and reconsider the time budget. I've spent a lot. Let me reconsider the simplest robust high-quality solution. Reality check: - Plain PN (deployable): CV ~0.256 (best epoch ~120, wd1e-4, no aug). - Ridge (not deployable): 0.22. - Blend (not deployable): 0.215. The deployable ceiling via a single PN seems ~0.25. Unless I find a better training approach. Wait , let me reconsider the possibility that I'm wrong about reflection being bad. Let me reconsider why the PN underperforms ridge. The ridge uses explicit volume/occupancy features. The PN struggles to compute volume via max-pooling. What if I give the PN a better chance by using ISOTROPIC normalization (preserving geometry) , let me quickly test iso vs aniso vs none for plain PN. That's a cheap, potentially meaningful knob I haven't tested. Also, let me reconsider: maybe a DIFFERENT pooling-compatible trick. The architecture does x.max(dim=2). Can't change. Let me test normalization modes quickly (fast, ~70s/config via cv_traj with rec). Let me kill distill and test norm modes for plain PN. Actually, let me reconsider the whole thing with fresh eyes given constraints and time: The PRIMARY metric is relative L2. Plain PN CV 0.256. Can I robustly beat that for the DEPLOYED model? Let me reconsider: The biggest issue might be the worst-case errors (maxe ~0.75-0.9 for PN vs 0.49 for ridge). A few samples are badly predicted by PN. If I can reduce those, rel_l2 improves. What reduces worst-case? A more robust/smoother model. Ridge is smoother. Idea: blend PN with a SIMPLE internal mechanism... no, can't. Let me reconsider: What if I make the PN's prediction more robust by training with a combination of MSE + a term that ties it to be smooth? Hard. Alternative: Let me reconsider using val properly. My deployment will train on ALL 132 samples (train+val). The CV estimate (train on 110) understates the final model trained on 132. Also, the true test might be "easier" (public_test constant baseline was 0.40 vs val 0.47). Let me just focus on making the best plain PN and deploy it robustly, while quickly checking a couple more knobs (normalization mode, and an ensemble-of-predictions via test-time... no). Actually, WAIT. Let me reconsider the ensemble result. Output-ensembling 4 PNs gave 0.263 (no better than single). But that ensemble used wd1e-3, resample, 300ep. Let me reconsider: maybe ensembling of DIVERSE models (different norm, different epochs, different aug) decorrelates better. But earlier same-config ensembling didn't help → errors correlated. Diverse configs might help a bit but I can't deploy an ensemble anyway. Hmm, unless... I reconsider the deployment constraint once more. The checkpoint must contain state_dict loadable into PointNetRegressor. It's ONE network. No ensemble. OK. Let me also reconsider: is the metric computed over the full held-out set, and is that set like val/test (same distribution)? Yes. So CV on 132 is my best proxy. Given all this, let me reconsider whether distillation from the BLEND (teacher that generalizes to 0.215) could work if I AVOID reflection (the problematic aug) and instead use mixup + jitter + resample with teacher recompute, and crucially include …[truncated]

Distillation is underperforming (0.29–0.46) and is slow , reflection makes teacher labels inconsistent and augmented clouds are off the val distribution. I'll abandon it. Let me instead test a cheap, unexplored knob on the plain PN: input normalization mode (isotropic preserves geometry, which may help the network infer volume).

bash

cd /app && kill 614 2>/dev/null; sleep 2
nohup python3 cv_traj.py '[
{"wd":1e-4,"norm":"iso"},
{"wd":1e-4,"norm":"none"},
{"wd":3e-4,"norm":"iso"},
{"wd":1e-4,"norm":"iso","resample_p":0.5}
]' > /app/norm.log 2>&1 &
echo "started $!"

output

started 637
[stderr]
[5]+  Terminated              nohup python3 distill.py '{"n_copies":20,"lam_true":0.0,"reflect":true,"jitter":0.006,"lam":5.0,"epochs":100}' > /app/distill.log 2>&1

bash

cd /app && sleep 200 && cat norm.log

output

{'wd': 0.0001, 'norm': 'iso'}                           best_ep=159 relL2=0.2652 mae=0.1615 maxe=0.7522 (88s)
{'wd': 0.0001, 'norm': 'none'}                          best_ep=279 relL2=0.2713 mae=0.1668 maxe=0.7666 (77s)

iso norm: 0.265, none: 0.271. Both WORSE than aniso (0.256). So anisotropic standardization (the starter default) is actually best. OK. So the plain PN is firmly ~0.256 with aniso norm, wd1e-4, early stop ~120. Let me wait for the last two configs but I expect similar. Let me accept that the plain PN baseline is ~0.25 and think about whether the distillation can be salvaged or if there's another approach. Actually, let me reconsider the distillation failure. The issue might be specifically REFLECTION. Let me reconsider a cleaner distillation: NO reflection, use resample + small jitter + MIXUP, recompute teacher (ridge with reflection-invariant features, or just the good 43-dim features but the mean sign issue remains under reflection, but without reflection, mean is fine). And include true-label supervision (lam_true ~ 0.3-0.5). But more fundamentally , I showed that on the student's TRAINING clouds, teacher≈truth (ridge fits its train). The augmented clouds (resample/jitter, no reflection) are very close to originals, so teacher≈truth there too. So the student basically trains on ≈true labels with mild aug → same as plain PN → ~0.256. No gain. To get ridge's generalization, the student needs teacher labels on clouds FAR from train (where ridge extrapolates well but truth unknown) , that requires synthetic clouds covering the val region, which I can't easily generate on-manifold. So distillation fundamentally can't beat plain PN here without good synthetic coverage. Confirmed dead end. Let me reconsider the BLEND deployment once more, very carefully. The blend 0.3·PN + 0.7·ridge = 0.215 is not deployable as-is. BUT what if the DEPLOYED model is a PN, and I accept PN-alone ~0.25? Versus is there any legitimate way to bake ridge into the PN? Hmm, what if I set the PN to literally compute ridge-like features? The first layers are per-point MLPs + max-pool. Max-pool gives max over points of learned per-point features. Ridge features like "std_x" = sqrt(E[x^2]-E[x]^2) , max-pool can't compute mean/std (needs averaging, not max). Occupancy fraction needs counting. So PN architecturally CANNOT compute ridge's mean/std/occupancy features (those need sum/average pooling, but the architecture uses MAX pooling). This is the fundamental limitation! Max-pool ≠ average-pool. Mass ∝ volume ∝ average-type quantity, which max-pool can't represent well. This explains everything: PN with max-pool can't compute volume/mean/std features that ridge uses, so it's stuck ~0.25. The architecture is fixed with max-pool. So I genuinely can't make the PN compute those features. BUT the PN fits TRAIN perfectly (0.014). How, if it can't compute volume? Because with 256 max-pooled channels over 100 training shapes, it can memorize via the silhouette/extreme points (each shape has a unique set of extreme points). It overfits to shape-specific extremes, which don't generalize. That's why train=0.014 but val=0.256. So the max-pool architecture is fundamentally limited for this volume-regression task. The best I can do is regularize it to generalize as well as possible (~0.25). Hmm, wait. Let me reconsider , can max-pool approximate a counting/average feature? Consider many channels, each a "soft indicator" of a small region; max over points ≈ 1 if any point in region, 0 else (occupancy indicator, not count). Summing these occupancy indicators in the HEAD (Linear layer) gives total occupied-region count ≈ occupancy volume! The head is Linear(256→128→64→1), which CAN sum the 256 max-pooled occupancy indicators. So PN CAN approximate occupancy-based volume: 256 region-detectors (each max→occupancy indicator) summed by the head. So representationally it's possible. The issue is learning it from 100 samples without overfitting to extremes. The network prefers memorizing over learning the generalizable occupancy sum. Could I ENCOURAGE occupancy-detector behavior? Hard to do via loss. Given the constraints and that I've established plain PN ≈0.25 is ha …[truncated]

Both iso (0.265) and none (0.271) are worse than anisotropic standardization (0.256) , the starter's normalization is best. I've now established the key finding: the **max-pooling** architecture fundamentally struggles with volume/average-type features (mass ∝ volume), capping it near 0.25 while it memorizes training extremes (train rel_L2 0.014). Let me confirm the robust epoch choice by dumping the full pooled-CV trajectory for the best recipe.

bash

cd /app && cat > /app/curve.py << 'PY'
import torch, numpy as np, json, sys
from lib_train import load_split, train_once, metrics
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);y=torch.cat([ytr,yva],0)
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
cfg=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=260,warmup=5,swa_frac=0.9,seed=0,
         reflect=False,jitter=0.0,resample_p=0.0,scale_jit=0.0,huber=0.0,rec_every=10)
cfg.update(json.loads(sys.argv[1]) if len(sys.argv)>1 else {})
trajs=[];trues=[]
for k in range(K):
    va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
    r=train_once(X[tr],y[tr],X[va],y[va],cfg)
    trajs.append(r['traj']);trues.append(y[va].numpy().ravel())
eps=[e for e,_ in trajs[0]]
tr=np.concatenate(trues)
print('epoch  relL2    mae    maxe')
for ei,e in enumerate(eps):
    preds=np.concatenate([trajs[k][ei][1].numpy().ravel() for k in range(K)])
    m=metrics(preds,tr)
    print('%4d  %.4f  %.4f  %.4f'%(e+1,m[0],m[1],m[2]))
PY
nohup python3 curve.py '{}' > /app/curve.log 2>&1 &
echo "started $!"

output

started 660

bash

cd /app && sleep 150 && cat curve.log

output

epoch  relL2    mae    maxe
  10  0.3494  0.2213  1.0476
  20  0.3387  0.2166  0.9619
  30  0.3153  0.2075  0.7766
  40  0.3124  0.1951  0.9111
  50  0.3138  0.2035  0.8186
  60  0.3210  0.2004  1.1044
  70  0.3220  0.2025  0.9859
  80  0.3116  0.1943  0.9063
  90  0.3019  0.1937  0.9253
 100  0.3042  0.1893  0.9938
 110  0.2884  0.1820  0.7159
 120  0.2808  0.1705  0.7188
 130  0.2944  0.1893  0.7548
 140  0.2712  0.1675  0.7212
 150  0.2825  0.1729  0.8064
 160  0.2824  0.1724  0.7504
 170  0.2753  0.1675  0.7388
 180  0.2857  0.1759  0.7721
 190  0.2805  0.1736  0.7548
 200  0.2840  0.1751  0.7795
 210  0.2779  0.1707  0.7557
 220  0.2784  0.1713  0.7463
 230  0.2803  0.1734  0.7435
 240  0.2806  0.1732  0.7483
 250  0.2793  0.1721  0.7579
 260  0.2784  0.1716  0.7529

The curve is noisy but plateaus around 0.27-0.28 after epoch ~120. The earlier cv_traj reported best_ep=119 at 0.256 because rec_every=20 hit a lucky epoch. Here with rec_every=10 the best is ~0.271 at epoch 140. The single-seed CV is noisy. So realistically plain PN is ~0.27-0.28 CV (seed 0). The 0.256 earlier was a lucky rec point. This is sobering , plain PN is ~0.27. This noise tells me seed variance is high. Averaging over seeds would reduce the noise in the ESTIMATE, but we saw output-ensembling didn't improve the actual OOF much (0.263). Hmm, wait: output-ensembling 4 seeds gave 0.263, while single seed fluctuates 0.27-0.30. So ensembling DOES help reduce to ~0.263 (better than typical single ~0.28). I mis-stated earlier. Let me reconsider: ensemble 0.263 vs single-seed-0 0.28ish → ensembling gives a few points. But I can't deploy an ensemble. OK given all constraints, the deployable single PN is ~0.27 CV. The ridge (0.22) is better but not deployable in the required architecture. Let me now seriously reconsider: is there REALLY no way to get ridge-like performance into the single PN? The fundamental issue: max-pool can't compute averages/counts. BUT, I the model designer control feat_mean/feat_std (per-axis affine) and the training. I cannot change pooling. Wait, actually , reconsider the occupancy-via-max-then-sum argument. The head sums 256 max-pooled features. If each of the 256 channels learns to be a soft indicator of a distinct spatial cell (max over points = 1 if occupied), then head can sum → occupancy count → volume. This IS representable and generalizes (it's literally the ridge's best feature). The problem is the optimizer doesn't find it; it finds memorizing solutions. Can I bias the optimizer toward the occupancy solution? Ideas: - Strong weight decay pushes toward simpler functions but we saw it doesn't help much. - Initialize/pretrain the feature extractor to produce spatial-cell indicators? I could PRETRAIN the conv layers to output a fixed voxel-occupancy-like embedding (supervised to match a voxel grid), then train the head. But that's complex. - Actually, I could DISTILL differently: train the PN to regress the ridge's OUTPUT but ALSO the student naturally... we saw distill≈plain because teacher≈truth on train. Alternatively: Let me reconsider , what if I train the PN to predict not just mass but ALSO auxiliary targets like the occupancy fractions / extents (multi-output via... no, output is [B,1], head outputs 1). Can't add outputs without changing architecture (head final is →1). Hmm, but I could change the TRAINING to use a different head temporarily... the state_dict must match. The head is Linear(256→128→64→1). If I pretrain with auxiliary losses on intermediate features, keys still match. E.g., add an auxiliary linear from the 256 global feature to predict occupancy features, trained jointly (auxiliary head has its own params NOT in state_dict , but then those params aren't saved, fine, they're auxiliary). This encourages the 256-dim global feature (the max-pooled one) to encode volume/occupancy info! Then the main head uses it. This "auxiliary supervision" could push the global feature to be occupancy-like → better generalization. This is promising and doesn't violate constraints: the deployed model is exactly PointNetRegressor; I just add auxiliary heads DURING training (extra modules not saved). The global max-pooled feature is forced to predict occupancy fractions / extents (which generalize), making mass prediction more robust. Let me test this: multi-task training where the 256-dim global feature predicts [mass (main), + auxiliary: occupancy fractions at several resolutions, extents, std]. The auxiliary targets are computed from the cloud (known). This regularizes the representation toward generalizable geometric features. Let me implement and CV-test. This is a legit, promising idea. Implementation: - Subclass or replicate forward to expose global feature g (256-dim). - …[truncated]

The single-seed CV plateaus around 0.27–0.28 (the earlier 0.256 was a lucky sample point). The max-pool architecture genuinely caps here because it can't naturally compute volume/average features. Here's a promising idea that respects all constraints: **auxiliary supervision**. I add *temporary* auxiliary heads (not saved in the checkpoint) that force the 256-dim global max-pooled feature to also predict generalizable geometric descriptors (occupancy fractions, extents, PCA eigenvalues). This biases the representation toward the volume-like features that generalize, while the deployed model stays exactly `PointNetRegressor`. Let me implement and CV-test it.

bash

cd /app && cat > /app/auxtrain.py << 'PY'
import torch, numpy as np, time, sys, json, copy
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
# precompute compact aux features for all
AUX=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
print('aux dim',AUX.shape[1],flush=True)

def gfeat(model,xb):
    x=xb.transpose(1,2).contiguous(); x=model.feature(x); x=model.lift(x)
    return x.max(dim=2).values

def train_fold(tr,va,cfg):
    fmean=X[tr].reshape(-1,3).mean(0).to(dev); fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y[tr].mean(0).to(dev); lstd=Y[tr].std(0).clamp_min(1e-6).to(dev)
    amean=AUX[tr].mean(0).to(dev); astd=AUX[tr].std(0).clamp_min(1e-6).to(dev)
    Xtr_=X[tr].to(dev); ytr_=((Y[tr].to(dev)-lmean)/lstd); atr_=((AUX[tr].to(dev)-amean)/astd)
    torch.manual_seed(cfg.get('seed',0)); np.random.seed(cfg.get('seed',0))
    model=build_model(load_cfg()).to(dev)
    aux=nn.Linear(load_cfg()['model']['global_width'], AUX.shape[1]).to(dev)
    params=list(model.parameters())+list(aux.parameters())
    opt=torch.optim.AdamW(params,lr=cfg['lr'],weight_decay=cfg['wd'])
    E=cfg['epochs']; sched=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:0.5*(1+np.cos(np.pi*e/E)))
    n=len(tr);bs=cfg['batch_size'];lam=cfg['lam_aux'];rp=cfg.get('resample_p',0.0)
    rec=[]
    for ep in range(E):
        model.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs]; xb=Xtr_[idx]
            if rp>0:
                do=torch.rand(len(idx),device=dev)<rp
                if do.any():
                    ridx=torch.randint(0,xb.shape[1],(len(idx),xb.shape[1]),device=dev)
                    xb=torch.where(do.view(-1,1,1),torch.gather(xb,1,ridx.unsqueeze(-1).expand(-1,-1,3)),xb)
            xbn=(xb-fmean)/fstd
            g_=gfeat(model,xbn)
            pred=model.head(g_); apred=aux(g_)
            loss=F.mse_loss(pred,ytr_[idx])+lam*F.mse_loss(apred,atr_[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sched.step()
        if (ep+1)%cfg.get('rec_every',10)==0 or ep==E-1:
            model.eval()
            with torch.no_grad():
                p=(model((X[va].to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
            rec.append((ep,p))
    return rec

if __name__=='__main__':
    base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=200,seed=0,lam_aux=1.0,resample_p=0.0,rec_every=10)
    for over in json.loads(sys.argv[1]):
        cfg=dict(base);cfg.update(over)
        trajs=[];trues=[];t=time.time()
        for k in range(K):
            va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
            trajs.append(train_fold(tr,va,cfg));trues.append(Y[va].numpy().ravel())
        eps=[e for e,_ in trajs[0]];tr=np.concatenate(trues);best=None
        for ei,e in enumerate(eps):
            preds=np.concatenate([trajs[k][ei][1] for k in range(K)]);m=metrics(preds,tr)
            if best is None or m[0]<best[1][0]: best=(e,m)
        print('%-45s best_ep=%3d relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(over),best[0]+1,best[1][0],best[1][1],best[1][2],time.time()-t),flush=True)
PY
nohup python3 auxtrain.py '[{"lam_aux":0.0},{"lam_aux":1.0},{"lam_aux":3.0},{"lam_aux":10.0}]' > /app/aux.log 2>&1 &
echo "started $!"

output

started 683

This tests lam_aux in {0 (baseline), 1, 3, 10}. Note lam_aux=0 is a re-baseline within this harness (should match plain PN ~0.27). Let me wait ~4 min (4 configs × 6 folds × 200 epochs). Let me wait and check. Auxiliary supervision forces the global feature to encode the generalizable geometric descriptors. If it helps, I should see lam_aux>0 beat lam_aux=0. Let me wait.

Let me wait for the auxiliary-supervision CV results.

bash

cd /app && sleep 240 && cat aux.log

output

aux dim 43
{'lam_aux': 0.0}                              best_ep=100 relL2=0.2664 mae=0.1695 maxe=0.7319 (43s)
{'lam_aux': 1.0}                              best_ep=100 relL2=0.2856 mae=0.1784 maxe=0.9353 (41s)
{'lam_aux': 3.0}                              best_ep=150 relL2=0.2628 mae=0.1622 maxe=0.7862 (48s)
{'lam_aux': 10.0}                             best_ep=200 relL2=0.2705 mae=0.1689 maxe=0.8389 (42s)

Auxiliary supervision gives at best 0.2628 (lam=3) vs baseline 0.2664 , marginal, within noise. Not a clear win. The forced representation doesn't dramatically help because the HEAD still has to map g→mass and the aux constraint doesn't guarantee the mass mapping generalizes. So auxiliary supervision ≈ neutral. I've now tried: augmentation (hurts), normalization modes (aniso best), weight decay (neutral), dropout (hurts), longer training (plateau ~0.27), SWA (buggy/neutral), ensembling (can't deploy, modest), distillation (fails), auxiliary supervision (neutral). The plain PN is firmly ~0.26-0.27 CV. Let me step back and think about what ACTUALLY moves the needle, accepting the max-pool limitation. Given the architecture can't compute volume well, and ridge (avg-pool-like features) gets 0.22, the gap is the max-vs-avg pooling. KEY NEW IDEA: I control feat_mean/feat_std AND the input is [B,2048,3]. The max-pool takes max over the 2048 points of each channel. What if the INPUT coordinates are transformed such that max-pooling becomes more informative? No, affine only. Another KEY IDEA: Since I can't change architecture, but the head is an MLP on the 256-dim max-pooled vector, and the issue is generalization variance , what if I train MANY models and average their PREDICTIONS, then fit the SINGLE deployable PN to that averaged prediction via the head only... no, teacher≈truth on train again. Let me reconsider the ensemble-can't-deploy constraint ONE more time, differently: What if I train the single PN with a LOT of seeds' worth of diversity folded into one via long SWA along a cyclic-LR trajectory? SWA/cyclic can collect diverse snapshots along ONE trajectory and average weights (stays in one basin) → single model with ensemble-like benefits. This is the legitimate "single model" way to get ensemble benefits. My earlier SWA was buggy/with-aug. Let me properly test cyclic-LR SWA for the plain config. Actually, weight-averaging snapshots along a trajectory only helps if they're in a connected low-loss region. With cyclic LR it can work (that's what SWA does). Let me test: train 200 epochs, then cyclic/constant moderate LR for 100 more epochs collecting snapshots every few epochs, average weights, recompute BN. Compare CV to single. But my earlier quick SWA (swa_frac, cosine) gave neutral/bad. Let me test a proper SWA with constant LR phase. Given limited tokens, let me test it via a focused CV. Actually, let me reconsider priorities. I've spent considerable effort. The realistic outcome: deploy a solid single PN at ~0.26-0.27 CV. But the final model trained on ALL 132 (vs 110 in CV) should be somewhat better, and the real test set may be easier (public_test baseline 0.40 vs val 0.47). But let me make sure I squeeze what I can. Two more cheap, potentially-useful experiments: 1. Proper SWA with constant-LR snapshot averaging (single-model ensemble benefit). 2. Combine aux supervision (lam=3) + resample + SWA. Let me also reconsider: the best pooled-CV numbers I've seen: lam_aux=3 → 0.2628, resample_p0.5 earlier → 0.262. These are marginally better than 0.266. Let me test SWA properly now since it's the most principled single-model variance reducer. Let me write a focused test: for the plain config, after a cosine warmup-decay to epoch E0, switch to constant moderate LR and average snapshots. Actually, simpler and known-good: just average the snapshot WEIGHTS from the PLATEAU region (e.g., epochs 100-200, every 10) of a single cosine run, and recompute BN. Since in the plateau the model moves around a basin, averaging reduces variance. Let me test this "tail averaging" via CV. I'll reuse auxtrain structure but add weight snapshot averaging. Let me implement a test: for each fold, train 200 epochs (cosine), collect weight snapshots every 10 epochs from epoch 100+, average them, recompute BN on train, eval. Compare to best single snapshot. Let me code quickly. I'll add to a new script swatest.py based on train_fold b …[truncated]

Auxiliary supervision is only marginal (0.263 vs 0.266). The plain PN is firmly ~0.26–0.27. Let me test the most principled *single-model* variance reducer: **tail weight-averaging** (SWA-style) over the plateau snapshots of one cosine run, with BN recomputation , this captures ensemble-like benefits in one deployable model.

bash

cd /app && cat > /app/swatest.py << 'PY'
import torch, numpy as np, time, sys, json, copy
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
AUX=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
def gfeat(model,xb):
    x=xb.transpose(1,2).contiguous();x=model.feature(x);x=model.lift(x);return x.max(dim=2).values
def avg_state(states):
    out={}
    for k in states[0]:
        if states[0][k].dtype.is_floating_point:
            out[k]=torch.stack([s[k].float() for s in states],0).mean(0)
        else: out[k]=states[0][k].clone()
    return out
def train_fold(tr,va,cfg):
    fmean=X[tr].reshape(-1,3).mean(0).to(dev);fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y[tr].mean(0).to(dev);lstd=Y[tr].std(0).clamp_min(1e-6).to(dev)
    amean=AUX[tr].mean(0).to(dev);astd=AUX[tr].std(0).clamp_min(1e-6).to(dev)
    Xtr_=X[tr].to(dev);ytr_=((Y[tr].to(dev)-lmean)/lstd);atr_=((AUX[tr].to(dev)-amean)/astd)
    torch.manual_seed(cfg['seed']);np.random.seed(cfg['seed'])
    model=build_model(load_cfg()).to(dev)
    aux=nn.Linear(256,AUX.shape[1]).to(dev)
    params=list(model.parameters())+list(aux.parameters())
    E=cfg['epochs'];swa_start=cfg['swa_start'];base_lr=cfg['lr'];swa_lr=cfg['swa_lr']
    opt=torch.optim.AdamW(params,lr=base_lr,weight_decay=cfg['wd'])
    def lr_at(e):
        if e<swa_start: return 0.5*(1+np.cos(np.pi*e/swa_start))*(1-cfg['minf'])+cfg['minf']
        return swa_lr/base_lr
    sched=torch.optim.lr_scheduler.LambdaLR(opt,lr_at)
    n=len(tr);bs=cfg['batch_size'];lam=cfg['lam_aux'];rp=cfg.get('resample_p',0.0)
    snaps=[]
    for ep in range(E):
        model.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];xb=Xtr_[idx]
            if rp>0:
                do=torch.rand(len(idx),device=dev)<rp
                if do.any():
                    ridx=torch.randint(0,xb.shape[1],(len(idx),xb.shape[1]),device=dev)
                    xb=torch.where(do.view(-1,1,1),torch.gather(xb,1,ridx.unsqueeze(-1).expand(-1,-1,3)),xb)
            xbn=(xb-fmean)/fstd;g_=gfeat(model,xbn)
            loss=F.mse_loss(model.head(g_),ytr_[idx])+lam*F.mse_loss(aux(g_),atr_[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sched.step()
        if ep>=swa_start and (ep-swa_start)%cfg['swa_every']==0:
            snaps.append({k:v.detach().cpu().clone() for k,v in model.state_dict().items()})
    # single final
    model.eval()
    with torch.no_grad(): p_single=(model((X[va].to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
    # swa
    swa=build_model(load_cfg()).to(dev);swa.load_state_dict(avg_state(snaps))
    update_bn(swa,X[tr].to(dev),fmean,fstd)
    with torch.no_grad(): p_swa=(swa((X[va].to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
    return p_single,p_swa
if __name__=='__main__':
    base=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=250,seed=0,lam_aux=0.0,resample_p=0.0,
              swa_start=120,swa_lr=5e-4,swa_every=5,minf=0.0)
    for over in json.loads(sys.argv[1]):
        cfg=dict(base);cfg.update(over);t=time.time()
        ps=[];pw=[];trues=[]
        for k in range(K):
            va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
            a,b=train_fold(tr,va,cfg);ps.append(a);pw.append(b);trues.append(Y[va].numpy().ravel())
        tr=np.concatenate(trues)
        ms=metrics(np.concatenate(ps),tr);mw=metrics(np.concatenate(pw),tr)
        print('%-42s SINGLE rel%.4f mae%.4f mx%.4f | SWA rel%.4f mae%.4f mx%.4f (%.0fs)'%(str(over),ms[0],ms[1],ms[2],mw[0],mw[1],
…[truncated]

output

started 706

bash

cd /app && sleep 230 && cat swa.log

output

{'swa_lr': 0.0005, 'lam_aux': 0.0}         SINGLE rel0.2743 mae0.1784 mx0.6983 | SWA rel0.2684 mae0.1734 mx0.7548 (73s)
{'swa_lr': 0.0005, 'lam_aux': 3.0}         SINGLE rel0.2725 mae0.1744 mx0.8099 | SWA rel0.2669 mae0.1645 mx0.8187 (68s)

SWA gives tiny improvement: 0.2684 vs single 0.2743 (lam0), 0.2669 vs 0.2725 (lam3). Marginal. Let me wait for the 3rd config (with resample). The reality: everything hovers ~0.265-0.275. The PN is capped. Let me now think about whether I'm fundamentally missing something about the task that would let the PN do much better. Let me reconsider the data: maybe there's label noise or the mapping has a simpler structure I can exploit. Actually , wait. Let me reconsider the relationship between the label and a SINGLE strong feature more carefully. std_x had corr 0.635. What's the best SINGLE feature? And is the mass maybe well-predicted by a specific combination that the PN could learn? The ridge at 0.22 uses many features. Let me reconsider: maybe I should reduce the worst-case errors which dominate rel_l2. Let me look at which samples are badly predicted and why. Actually, let me reconsider the entire premise. Let me re-examine: is the test set going to be evaluated with the SAME normalization I provide? Yes. And my model outputs normalized, de-normalized by label_mean/std. Let me reconsider: the ridge achieves 0.22. What if I deploy a model that is ACTUALLY computing ridge-like features but implemented within the PN's forward in a way that matches the state_dict? No, I can't change forward at inference; grader uses PointNetRegressor.forward. Hold on, let me re-read the constraint: "candidate_model.py - the fixed PointNet-style architecture. Keep it as is; your saved checkpoint has to load back into this same architecture." and "The model must consume a [B, 2048, 3] tensor and produce a [B, 1] output." So at inference, they instantiate PointNetRegressor, load my state_dict, and call forward. I cannot alter forward. Confirmed max-pool. So I'm stuck with max-pool. Best deployable ~0.265. Given this, let me reconsider: can I at least reliably hit the BEST version of the max-pool PN? The differences between configs are within noise (~0.265-0.28). I should pick a robust recipe and train the final model on ALL 132 data (which helps vs 110 in CV). Let me also reconsider: should I incorporate val into training for the final model? Yes , more data helps generalization, and the final test is separate. Training on 132 vs 110 should give a meaningful boost (20% more data). Let me also reconsider seed averaging for the DEPLOYED model via SWA , it gave a tiny consistent gain (~0.006). I'll use SWA for the final model. And aux supervision gave a tiny gain and might help robustness; I'll include it (lam_aux=3) since it slightly helped and regularizes the representation. And resample_p , let me see the 3rd result. Let me wait for the 3rd config, then finalize. Actually, let me reconsider whether I can do better than 0.265 by being smarter about the TARGET. The rel_l2 metric weights all samples by 1/||true||. Large-mass samples dominate ||true||. Actually rel_l2 = sqrt(Σ(p-t)²)/sqrt(Σt²). The denominator is fixed. So minimizing Σ(p-t)² = MSE in physical units. I'm training on standardized labels with MSE, which is equivalent (scaled). Good, aligned. But MAE and maxe are secondary. To reduce maxe (worst case), I might train with MSE (already penalizes large errors). Fine. Let me reconsider: maybe predicting in physical units directly vs standardized doesn't matter (linear). Fine. One more idea to genuinely help: **blend the PN prediction with a constant/prior to reduce variance** , no, can't post-process. Let me also reconsider: since the max-pool PN overfits, maybe a MUCH smaller effective capacity helps. I can't change widths, but I can FREEZE parts or use very strong weight decay on specific layers. We saw wd up to 1e-2 neutral. I think I've thoroughly explored. Let me finalize with the best robust recipe: - aniso norm (best) - wd=1e-4 - lam_aux=3 (tiny help + representation regularization) - resample_p: check 3rd result - SWA tail averaging (tiny help) - train on ALL 132 for final - epochs ~250 with swa from 120 Let me ge …[truncated]

SWA gives a small but consistent gain (~0.268 vs 0.274). Let me see the resample variant and then lock in the recipe.

bash

cd /app && sleep 30 && cat swa.log

output

{'swa_lr': 0.0005, 'lam_aux': 0.0}         SINGLE rel0.2743 mae0.1784 mx0.6983 | SWA rel0.2684 mae0.1734 mx0.7548 (73s)
{'swa_lr': 0.0005, 'lam_aux': 3.0}         SINGLE rel0.2725 mae0.1744 mx0.8099 | SWA rel0.2669 mae0.1645 mx0.8187 (68s)
{'swa_lr': 0.0005, 'lam_aux': 3.0, 'resample_p': 0.5} SINGLE rel0.2925 mae0.1849 mx0.8069 | SWA rel0.2775 mae0.1748 mx0.9049 (111s)

resample_p=0.5 HURTS here (0.2775 vs 0.2669). So drop resample. Best: lam_aux=3, SWA, no resample → 0.2669. So my best recipe: aniso norm, wd=1e-4, lam_aux=3, SWA tail-averaging, no aug. CV ~0.267. Now, the big picture decision. The deployable PN is ~0.267 CV. The ridge teacher is 0.22 but can't be deployed. Let me reconsider ONE fundamentally different idea that I haven't tried and that could actually close the gap: **Average pooling emulation is impossible, BUT I can reconsider whether a multi-seed SWA soup works if models are trained from the SAME init with DIFFERENT data order** (not different init). Earlier soup failed because different init seeds → different basins. If I use the SAME init and just different data ordering/SWA trajectories, averaging might work and give ensemble benefit in one model. But that's essentially what tail-SWA does within one run. Multi-run-same-init soup might add a bit. Marginal. I think I've hit the architecture's ceiling. Let me now also sanity-check: maybe the issue is that 6-fold CV trains on only 110 samples; the FINAL model on 132 + the real test being easier could bring the real number down. Also the public_test constant baseline (0.40) was lower than val (0.47), hinting the test distribution might be a bit easier. Let me make the final decision and BUILD the deliverable robustly. I'll: 1. Train the FINAL model on ALL 132 samples with the best recipe (aniso, wd1e-4, lam_aux=3, SWA from epoch 120 to 250, swa_lr 5e-4, swa_every 5). 2. Since there's no holdout for the final, SWA + fixed schedule provides robustness. 3. Save checkpoint with state_dict, feat_mean/std (length-3), label_mean/std (length-1). But wait , I should double-check one thing: does training on 132 vs 110 actually help, and does my final recipe reproduce ~0.267 or better? I can't measure the final-on-132 directly (no holdout). But CV validates the recipe. However, I'm slightly worried about deploying a recipe at 0.267 when ridge gets 0.22. Let me make ONE more serious attempt at the fundamental problem, because closing to ~0.22 would be a big win. The core issue: max-pool can't average. BUT, here's a genuinely new idea: I can make the 2048 input points themselves encode density if I... no, grader feeds raw points. Another idea: the grader feeds (x-feat_mean)/feat_std. What if feat_std is VERY small, so inputs are huge, pushing GELU into linear regime... no. Let me reconsider the aux-supervision idea but STRONGER: instead of a linear aux head, force the global feature g to EQUAL a rich geometric descriptor via a strong penalty, effectively making g ≈ [occupancy fractions, extents, ...] (generalizable features), then the main head is a small MLP on these generalizable features ≈ a nonlinear ridge. If g is forced to be the good features, the head learns mass from good features → should approach ridge (0.22) or better (nonlinear). But lam_aux=3 already pushes g toward predicting those features (via a linear map aux: g→features). For the main head to benefit, g must CONTAIN the feature info linearly decodable. With lam_aux large, g encodes features well, but the main head (3-layer MLP) maps g→mass. If g≈features (up to linear transform), head ≈ MLP on features. That should generalize like an MLP-on-features (nonlinear ridge), potentially ~0.20. But we saw lam_aux=10 was WORSE (0.27) than lam_aux=3 (0.263). So forcing g to encode features too strongly hurt the main task. There's tension: g must both encode features AND be optimal for mass. The main-task gradient dominates and overfits. What if I DECOUPLE: Phase 1: train feature extractor + aux head ONLY to predict geometric features (no mass), so g becomes a good generalizable geometric embedding. Phase 2: FREEZE feature extractor, train ONLY the main head (model.head) to map g→mass. Since g is now a fixed generalizable embedding (like ridge features), the head learns mass from it → generalizes like nonlinear-ridge. The head is small (256→128→64→1) → limited ove …[truncated]

Resample hurts (0.278); best recipe is lam_aux=3 + SWA ≈ 0.267. But I want to break past the max-pool ceiling toward ridge's 0.22. New idea worth testing: **two-phase training**. Phase 1 trains the feature extractor to predict generalizable geometric descriptors (never seeing mass), producing a representation that *can't* overfit to mass. Phase 2 freezes it and trains only the small head to map that representation→mass , effectively a nonlinear regressor on learned geometric features. Let me CV this.

bash

cd /app && cat > /app/phase2.py << 'PY'
import torch, numpy as np, time, sys, json, copy
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
AUX=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
def gfeat(model,xb):
    x=xb.transpose(1,2).contiguous();x=model.feature(x);x=model.lift(x);return x.max(dim=2).values
def train_fold(tr,va,cfg):
    fmean=X[tr].reshape(-1,3).mean(0).to(dev);fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y[tr].mean(0).to(dev);lstd=Y[tr].std(0).clamp_min(1e-6).to(dev)
    amean=AUX[tr].mean(0).to(dev);astd=AUX[tr].std(0).clamp_min(1e-6).to(dev)
    Xtr_=X[tr].to(dev);ytr_=((Y[tr].to(dev)-lmean)/lstd);atr_=((AUX[tr].to(dev)-amean)/astd)
    torch.manual_seed(cfg['seed']);np.random.seed(cfg['seed'])
    model=build_model(load_cfg()).to(dev)
    aux=nn.Linear(256,AUX.shape[1]).to(dev)
    n=len(tr);bs=cfg['batch_size']
    # Phase 1: representation learning (feature+lift+aux), predict geom features
    p1=list(model.feature.parameters())+list(model.lift.parameters())+list(aux.parameters())
    opt=torch.optim.AdamW(p1,lr=cfg['lr1'],weight_decay=cfg['wd1'])
    E1=cfg['epochs1'];sched=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:0.5*(1+np.cos(np.pi*e/E1)))
    for ep in range(E1):
        model.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];g_=gfeat(model,(Xtr_[idx]-fmean)/fstd)
            loss=F.mse_loss(aux(g_),atr_[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sched.step()
    # freeze feature+lift
    model.eval()
    for p in model.feature.parameters(): p.requires_grad_(False)
    for p in model.lift.parameters(): p.requires_grad_(False)
    # precompute g for train (eval mode, frozen BN)
    with torch.no_grad():
        G=[];
        for s in range(0,n,64): G.append(gfeat(model,(Xtr_[s:s+64]-fmean)/fstd))
        G=torch.cat(G,0)
        Gva=gfeat(model,(X[va].to(dev)-fmean)/fstd)
    # Phase 2: train head on frozen g
    opt2=torch.optim.AdamW(model.head.parameters(),lr=cfg['lr2'],weight_decay=cfg['wd2'])
    E2=cfg['epochs2'];sched2=torch.optim.lr_scheduler.LambdaLR(opt2,lambda e:0.5*(1+np.cos(np.pi*e/E2)))
    best=None;rec=[]
    for ep in range(E2):
        model.head.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs]
            loss=F.mse_loss(model.head(G[idx]),ytr_[idx])
            opt2.zero_grad(set_to_none=True);loss.backward();opt2.step()
        sched2.step()
        if (ep+1)%10==0 or ep==E2-1:
            model.head.eval()
            with torch.no_grad(): p=(model.head(Gva)*lstd+lmean).cpu().numpy().ravel()
            rec.append((ep,p))
    return rec
if __name__=='__main__':
    base=dict(batch_size=16,seed=0,lr1=1e-3,wd1=1e-4,epochs1=150,lr2=1e-3,wd2=1e-3,epochs2=150)
    for over in json.loads(sys.argv[1]):
        cfg=dict(base);cfg.update(over);t=time.time()
        trajs=[];trues=[]
        for k in range(K):
            va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
            trajs.append(train_fold(tr,va,cfg));trues.append(Y[va].numpy().ravel())
        eps=[e for e,_ in trajs[0]];truec=np.concatenate(trues);best=None
        for ei,e in enumerate(eps):
            preds=np.concatenate([trajs[k][ei][1] for k in range(K)]);m=metrics(preds,truec)
            if best is None or m[0]<best[1][0]: best=(e,m)
        print('%-45s best_ep=%3d relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(over),best[0]+1,best[1][0],best[1][1],best[1][2],time.time()-t),flush=True)
PY
nohup pytho
…[truncated]

output

started 737

bash

cd /app && sleep 200 && cat phase2.log

output

{'wd2': 0.001}                                best_ep= 70 relL2=0.2757 mae=0.1702 maxe=0.7556 (49s)
{'wd2': 0.01}                                 best_ep=140 relL2=0.2820 mae=0.1769 maxe=0.7645 (42s)
{'wd2': 0.03}                                 best_ep= 70 relL2=0.2778 mae=0.1732 maxe=0.7562 (35s)
{'epochs1': 250, 'wd2': 0.01}                 best_ep=130 relL2=0.2705 mae=0.1682 maxe=0.7804 (45s)

The two-phase approach gives ~0.27-0.28, NOT better. So the frozen geometric representation + head doesn't beat the end-to-end PN, and doesn't approach ridge (0.22). Why? Because the head mapping g→mass still must generalize, and even though g encodes geometric features, the max-pooled g (256-dim) learned to predict 43 features may not linearly expose them cleanly, and the head overfits. Also, the geometric features themselves, when RE-DERIVED by the network via max-pool, are approximations that generalize worse than directly-computed ridge features. Fundamentally confirmed: the max-pool PN can't match the directly-computed ridge features. The best deployable ~0.265-0.27. I've now exhausted reasonable ideas to break the ceiling: - augmentation: hurts - normalization: aniso best - weight decay, dropout: neutral/hurt - longer training: plateau - SWA: tiny gain - ensembling: can't deploy, modest - distillation: fails - aux supervision: tiny gain - two-phase frozen rep: neutral The architecture (max-pool) is the binding constraint. I'll deploy the best single PN: lam_aux=3 + SWA, aniso norm, wd1e-4, trained on ALL 132 samples. CV ~0.267. The final model on 132 + a possibly-easier real test should give something in the mid-0.2s. Actually, let me reconsider one thing about the final deployment that genuinely helps: training on all 132 vs 110. Let me ALSO reconsider whether I should pick the recipe that had the best CV: lam_aux=3+SWA = 0.2669. And I should combine the SWA with lam_aux. Let me reconsider the SWA schedule: swa_start=120, epochs=250, swa_lr=5e-4. That gave 0.2669 with lam_aux3. Good. But hold on , let me reconsider whether I can get a small additional gain by averaging MULTIPLE SWA runs with DIFFERENT seeds via... no (different basins). Let me reconsider: the CV numbers are noisy (single seed). To make the FINAL model robust, SWA helps. Also, I could train the final model and internally validate by predicting on a small held-out to sanity check, but I'll use all data. Actually, let me reconsider using the validation set held-out differently: I could train the final on train(100) only with SWA and use val(32) to verify metrics match CV (~0.27), then ALSO train a version on all 132 and deploy that. The all-132 version should be at least as good. I'll deploy all-132. Hmm, one concern: without ANY holdout for the 132-model, if something's off I won't catch it. Let me do: train final on all 132; ALSO train on 100 (train) and check val metrics to confirm the recipe reproduces ~0.27 on val. If val looks good, trust the 132-model. Let me reconsider the earlier single-val numbers: at the very start, no_aug got val best 0.187 (biased by epoch selection). The honest val for the final recipe should be checked without epoch cherry-picking , use the SWA model. Let me now also reconsider: is there benefit to a SLIGHT ensemble via SWA collecting from a CYCLIC LR (multiple cycles) to get more diverse snapshots in one basin? Marginal. I'll keep the simple tail-SWA. Let me reconsider whether to include lam_aux in the FINAL. It gave 0.2669 vs 0.2684 (without). Tiny. But it also regularizes representation and didn't hurt. I'll include lam_aux=3. Actually, the difference is within noise. To be safe and simple, including aux supervision is a mild regularizer with slight benefit, keep it. Let me now write the FINAL training script that: 1. Loads train+val → 132 samples. 2. Computes feat_mean/std (aniso, over all points of the 132), label_mean/std. 3. Computes AUX features (43-dim) for aux supervision. 4. Trains with lam_aux=3, SWA (start 120, total 250, swa_lr 5e-4, every 5), wd1e-4, lr1e-3, bs16. 5. Averages SWA snapshots, recomputes BN on the 132. 6. Saves checkpoint to /app/outputs/model.pt with required keys (state_dict of the PLAIN PointNetRegressor , aux head excluded). 7. Verify it loads into PointNetRegressor and runs on [B,2048,3]. Also, I should verify the saved model reproduces predictions through the grader's convent …[truncated]

The two-phase approach also lands at ~0.27 , confirming the **max-pool architecture itself** is the binding constraint (it can't compute volume/average features the way directly-computed ridge features do). I've systematically tried augmentation, normalization, regularization, distillation, auxiliary supervision, and frozen representations; the best *deployable* recipe is **lam_aux=3 + tail-SWA** at CV ≈ 0.267. Let me now build the final deliverable: train that recipe on all 132 samples, and as a sanity check also verify the train→val version reproduces the CV-level metric.

bash

cd /app && cat > /app/final.py << 'PY'
import torch, numpy as np, time, json
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'

def gfeat(model,xb):
    x=xb.transpose(1,2).contiguous();x=model.feature(x);x=model.lift(x);return x.max(dim=2).values
def avg_state(states):
    out={}
    for k in states[0]:
        if states[0][k].dtype.is_floating_point:
            out[k]=torch.stack([s[k].float() for s in states],0).mean(0)
        else: out[k]=states[0][k].clone()
    return out

def train(Xtr,Ytr, cfg, Xval=None, Yval=None):
    n=Xtr.shape[0]
    fmean=Xtr.reshape(-1,3).mean(0).to(dev);fstd=Xtr.reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Ytr.mean(0).to(dev);lstd=Ytr.std(0).clamp_min(1e-6).to(dev)
    AUX=torch.tensor(rich_feats(Xtr, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
    amean=AUX.mean(0).to(dev);astd=AUX.std(0).clamp_min(1e-6).to(dev)
    Xtr_=Xtr.to(dev);ytr_=((Ytr.to(dev)-lmean)/lstd);atr_=((AUX.to(dev)-amean)/astd)
    torch.manual_seed(cfg['seed']);np.random.seed(cfg['seed'])
    model=build_model(load_cfg()).to(dev)
    aux=nn.Linear(256,AUX.shape[1]).to(dev)
    params=list(model.parameters())+list(aux.parameters())
    E=cfg['epochs'];swa_start=cfg['swa_start'];base_lr=cfg['lr'];swa_lr=cfg['swa_lr']
    opt=torch.optim.AdamW(params,lr=base_lr,weight_decay=cfg['wd'])
    def lr_at(e):
        if e<swa_start: return 0.5*(1+np.cos(np.pi*e/swa_start))
        return swa_lr/base_lr
    sched=torch.optim.lr_scheduler.LambdaLR(opt,lr_at)
    bs=cfg['batch_size'];lam=cfg['lam_aux'];snaps=[]
    for ep in range(E):
        model.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];g_=gfeat(model,(Xtr_[idx]-fmean)/fstd)
            loss=F.mse_loss(model.head(g_),ytr_[idx])+lam*F.mse_loss(aux(g_),atr_[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sched.step()
        if ep>=swa_start and (ep-swa_start)%cfg['swa_every']==0:
            snaps.append({k:v.detach().cpu().clone() for k,v in model.state_dict().items()})
    swa=build_model(load_cfg()).to(dev);swa.load_state_dict(avg_state(snaps))
    update_bn(swa,Xtr_,fmean,fstd)
    swa.eval()
    res={'model':swa,'fmean':fmean.cpu(),'fstd':fstd.cpu(),'lmean':lmean.cpu(),'lstd':lstd.cpu()}
    if Xval is not None:
        with torch.no_grad():
            p=(swa((Xval.to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
        res['val']=metrics(p,Yval.numpy().ravel())
    return res

CFG=dict(batch_size=16,lr=1e-3,wd=1e-4,epochs=250,seed=0,lam_aux=3.0,swa_start=120,swa_lr=5e-4,swa_every=5)
Xtr,ytr=load_split('train');Xva,yva=load_split('val')
# sanity: train on train(100), eval val(32)
r=train(Xtr,ytr,CFG,Xva,yva)
print('SANITY train->val: relL2=%.4f mae=%.4f maxe=%.4f'%r['val'],flush=True)
# also multi-seed val to gauge variance
for sd in [1,2]:
    c=dict(CFG);c['seed']=sd;rr=train(Xtr,ytr,c,Xva,yva)
    print('  seed%d val relL2=%.4f mae=%.4f maxe=%.4f'%(sd,*rr['val']),flush=True)
PY
python3 final.py

output

SANITY train->val: relL2=0.1745 mae=0.1182 maxe=0.3462
  seed1 val relL2=0.2250 mae=0.1633 maxe=0.4532
  seed2 val relL2=0.2431 mae=0.1662 maxe=0.4419

Interesting! On the actual val set (train on 100, eval val 32) with SWA: - seed0: 0.1745 - seed1: 0.2250 - seed2: 0.2431 Mean ~0.21, varies by seed. The val set (32 samples) gives more optimistic numbers than the 6-fold CV pooled (0.267). This is because the specific val split is "easier" or the CV folds include some harder held-out samples. The seed variance is notable (0.17-0.24). This tells me seed choice matters for the single deployed model. To reduce this variance in a DEPLOYABLE single model, I could average predictions of multiple seeds , but can't deploy ensemble. However, I realize I CAN reduce variance by making the SWA collect from MULTIPLE independent runs IF they're compatible... no (different basins). Wait, but actually , here's a legitimate single-model approach to capture multi-seed benefit that I dismissed: I can't average different-init weights. But the VAL metric varies by seed because each single model is noisy. For deployment I pick ONE. If I pick based on val, that's overfitting to the 32-sample val. Let me reconsider: the pooled 6-fold CV (0.267) is a more reliable estimate than the 32-sample val (0.17-0.24). The real test performance is probably ~0.22-0.27. Given seed variance, the safest deployable choice is to reduce variance. Since I can't ensemble at inference, the best I can do within ONE model is SWA (done) and picking a robust seed. Hmm, but actually there IS a way to get multi-seed ensemble benefit in a single PointNetRegressor: train N independent models, average their PREDICTIONS on the training set to form a smoothed target, then... no, teacher≈truth on train. Actually, wait. Let me reconsider the ensemble-distillation WITHIN the deployable architecture but using AUGMENTED inputs where the ensemble provides smoothed predictions. Earlier distillation failed, but that was distilling RIDGE with reflection. Let me reconsider distilling a PN-ENSEMBLE: The problem before: teacher≈truth on train points. But if I distill on AUGMENTED points (jitter/resample, NO reflection) where the ensemble of PNs gives averaged (lower-variance) predictions, the student learns the ensemble's smoother function. On train points ensemble≈truth (each PN fits train), but on jittered points the ensemble's average is smoother than any single PN → student learns smoother function → better generalization. This is plausible! And it's deployable (student is a PN). But this is complex and time is limited. Given the seed variance, a simpler high-value move: the pooled CV says ~0.267 but val says ~0.21. The truth is somewhere around there. Let me make a decision that balances robustness and effort: Given the seed variance, I'll reduce deployment risk by selecting among a few seeds using the pooled CV... but I can only train the final on 132 and can't CV-select the final's seed without a holdout. Better plan: Train the final model on ALL 132 with SWA. To reduce seed variance, I'll pick the seed by a quick CV on the RECIPE (already done, seed0 used in CV=0.267). For the final, use seed0 (consistent with CV). But seed0 on val gave 0.1745 (train on 100). That's the lucky one actually. For all-132 training, I can't measure. Actually, let me reconsider: the most robust single deployable model given seed variance is to do a "SWA over a longer, multi-cycle trajectory" to average more diverse snapshots within one basin, reducing variance more than single tail-SWA. OR, accept seed0. Let me think about what reduces the RISK of a bad deployment. The pooled 6-fold CV (0.267, seed0) is my honest estimate. The final model (132 samples, seed0, SWA) should be around there or better (more data). I'll go with it but let me ALSO verify the final-on-132 model behaves sanely and saves correctly. Actually, I want to squeeze a bit more robustness. Let me reconsider: can I legitimately average the predictions of multiple seeds by training them and then creating ONE model whose weights reproduce the average? No. BUT , I could train …[truncated]

On the actual val set the SWA model ranges 0.17–0.24 across seeds (mean ~0.21) , more optimistic than pooled CV (0.267), but with notable **seed variance**. Reducing that variance is the main remaining lever. Let me quantify how much a multi-seed ensemble tightens things on val, to decide whether ensemble-distillation into one deployable PN is worthwhile.

bash

cd /app && cat >> /app/final.py << 'PY'

def ens_check():
    import numpy as np
    preds=[]
    for sd in range(5):
        c=dict(CFG);c['seed']=sd;r=train(Xtr,ytr,c)
        m=r['model'];fm=r['fmean'].cuda();fs=r['fstd'].cuda();lm=r['lmean'].cuda();ls=r['lstd'].cuda()
        with torch.no_grad(): p=(m((Xva.cuda()-fm)/fs)*ls+lm).cpu().numpy().ravel()
        preds.append(p); print('  seed%d val rel=%.4f'%(sd,metrics(p,yva.numpy().ravel())[0]),flush=True)
    P=np.stack(preds,0)
    for kk in [2,3,5]:
        e=P[:kk].mean(0); print('ENS k=%d val relL2=%.4f mae=%.4f maxe=%.4f'%(kk,*metrics(e,yva.numpy().ravel())),flush=True)
ens_check()
PY
python3 -c "import final" 2>/dev/null &
sleep 1; echo "running ens_check via import..."
# run just the ens part
python3 - << 'PY'
import final
final.ens_check()
PY

output

running ens_check via import...
SANITY train->val: relL2=0.1749 mae=0.1196 maxe=0.3530
SANITY train->val: relL2=0.1799 mae=0.1235 maxe=0.3610
  seed1 val relL2=0.2213 mae=0.1563 maxe=0.4387
  seed1 val relL2=0.2241 mae=0.1616 maxe=0.4317
  seed2 val relL2=0.2353 mae=0.1622 maxe=0.4199
  seed2 val relL2=0.2335 mae=0.1572 maxe=0.4553
  seed0 val rel=0.1797
  seed0 val rel=0.1807
  seed1 val rel=0.2212
  seed1 val rel=0.2216
  seed2 val rel=0.2238
  seed2 val rel=0.2369
  seed3 val rel=0.2295
  seed3 val rel=0.2357
  seed4 val rel=0.2595
ENS k=2 val relL2=0.1927 mae=0.1355 maxe=0.3954
ENS k=3 val relL2=0.1990 mae=0.1398 maxe=0.3959
ENS k=5 val relL2=0.2116 mae=0.1468 maxe=0.4326
  seed4 val rel=0.2642
ENS k=2 val relL2=0.1953 mae=0.1337 maxe=0.3931
ENS k=3 val relL2=0.2053 mae=0.1405 maxe=0.3968
ENS k=5 val relL2=0.2176 mae=0.1477 maxe=0.4307
  seed0 val rel=0.1813
  seed1 val rel=0.2340
  seed2 val rel=0.2423
  seed3 val rel=0.2360
  seed4 val rel=0.2657
ENS k=2 val relL2=0.2015 mae=0.1409 maxe=0.4120
ENS k=3 val relL2=0.2107 mae=0.1463 maxe=0.4082
ENS k=5 val relL2=0.2212 mae=0.1525 maxe=0.4422

(Two runs overlapped because my background `import final` also executed, but results are consistent.) Ensemble on val: - k=2: ~0.19-0.20 - k=3: ~0.20-0.21 - k=5: ~0.21-0.22 Hmm, interesting , ensembling seeds 0+1 (k=2) gives 0.19, but adding worse seeds (3,4 at 0.23-0.26) pulls the ensemble up. seed0 alone is 0.18. So on THIS val split, seed0 is just lucky. The ensemble (k=5) ~0.21-0.22 is more robust/honest than cherry-picking seed0. The ensemble REDUCES variance and gives ~0.21 robustly (vs single seed ranging 0.18-0.26). On val, ensemble k=3-5 ≈ 0.21. This is better than the pooled CV single (0.267) , but val is an easier split. The key insight: ensembling reduces the seed variance and the worst-case. For a robust deployment, ensemble-like behavior is valuable. But I can deploy only ONE model. So: is ensemble-distillation worth it? The ensemble (k=5) ≈ 0.21 on val robustly. A distilled single student that mimics the ensemble could get ~0.21-0.23, more robust than a single random seed (which might be 0.26 if unlucky). Given the seed variance is real and I can't control which "seed" the final 132-model lands on, distilling an ensemble into one model is the principled way to get a robust, low-variance deployable model. Let me do ensemble-distillation properly this time: Teacher = average of M PN models (trained on the full training data with different seeds). Student = single PN trained to match teacher's predictions on training clouds + AUGMENTED copies (jitter + resample, NO reflection , reflection was the culprit before). Crucially, on augmented (jittered) clouds, the teacher ensemble's average is smoother and lower-variance than truth-fitting, so the student learns a smoother function. On clean train points teacher≈truth (each member fits), but the jittered neighborhood provides the ensemble's smoothing signal. Also I can add true-label supervision with small weight. But wait , earlier I argued distillation on train≈truth gives no gain. The gain here is different: the student is ONE model learning to match the ENSEMBLE. Even if teacher≈truth on exact train points, the student (single PN) trained to fit these targets will itself have seed variance... unless the augmented neighborhoods constrain it to the ensemble's smooth function. The ensemble's prediction on a JITTERED point is the average of M models → smoother, and the student matching that over many jittered samples is pulled toward the smooth ensemble function → lower variance than a single model trained on 132 hard targets. This is the legitimate mechanism by which distillation reduces variance. Let me TEST it via CV to confirm it beats single-model CV (0.267). If the distilled student CV ≈ 0.23-0.24, it's a real win. Let me implement ensemble-distillation CV: - Per fold: train M=4 teacher PNs (seeds) on fold-train (full recipe with SWA? or plain, plain is faster; but SWA teachers are better). Use the SWA recipe for teachers (better individuals → better ensemble). - Build distill set: for each fold-train cloud, create C augmented copies (jitter 0.005, resample), compute teacher-ensemble prediction (in physical units) as target. Include clean copies too. - Train student PN on (aug cloud, teacher target), + small lam_true on clean clouds with true labels. - Eval student on fold-val. This is expensive: M=4 teachers × 6 folds + student × 6 folds. Teachers with SWA ~12s each → 4×6=24 teacher trainings ~5min. Plus students. Might be ~8-10 min. Too long for foreground; run in background and wait. Given token budget (~86k left) and time, let me be efficient: use M=3 teachers, plain training (no SWA) for speed but more epochs, C=8 augmented copies. Actually teacher quality matters; let me use the aux+SWA recipe but it's slower. Let me use a moderate teacher: aux+cosine, 150 epochs, no SWA (SWA gave only +0.006). Plain aux model ~0.266. Ensemble of 3 plain ~0.24? Let me just test the full pipeline once via CV and see. Let me write distill2.py carefully, run i …[truncated]

The multi-seed **ensemble** is far more stable (k=5 ≈ 0.21 on val) than individual seeds (0.18–0.26) , seed0 alone was just lucky. Since I can only deploy one model, the principled way to capture this variance reduction is **ensemble distillation**: train a student PN to match the ensemble's *smoothed* predictions over jittered clouds (where the teacher average ≠ training labels, so it transfers real signal). Let me CV-test this mechanism honestly.

bash

cd /app && cat > /app/distill2.py << 'PY'
import torch, numpy as np, time, sys, json
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
Xtr0,ytr0=load_split('train');Xva0,yva0=load_split('val')
X=torch.cat([Xtr0,Xva0],0);Y=torch.cat([ytr0,yva0],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(1234);perm=torch.randperm(N,generator=g)
K=6;folds=[perm[i::K] for i in range(K)]
AUXALL=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
def gfeat(m,xb):
    x=xb.transpose(1,2).contiguous();x=m.feature(x);x=m.lift(x);return x.max(dim=2).values
def avg_state(ss):
    o={}
    for k in ss[0]:
        o[k]=torch.stack([s[k].float() for s in ss],0).mean(0) if ss[0][k].dtype.is_floating_point else ss[0][k].clone()
    return o
def train_teacher(tr,seed,aux_idx):
    fmean=X[tr].reshape(-1,3).mean(0).to(dev);fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y[tr].mean(0).to(dev);lstd=Y[tr].std(0).clamp_min(1e-6).to(dev)
    A=AUXALL[tr];amean=A.mean(0).to(dev);astd=A.std(0).clamp_min(1e-6).to(dev)
    Xt=X[tr].to(dev);yt=((Y[tr].to(dev)-lmean)/lstd);at=((A.to(dev)-amean)/astd)
    torch.manual_seed(seed);np.random.seed(seed)
    m=build_model(load_cfg()).to(dev);aux=nn.Linear(256,A.shape[1]).to(dev)
    E=220;ss=120;opt=torch.optim.AdamW(list(m.parameters())+list(aux.parameters()),lr=1e-3,weight_decay=1e-4)
    sch=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:(0.5*(1+np.cos(np.pi*e/ss)) if e<ss else 0.5))
    n=len(tr);bs=16;snaps=[]
    for ep in range(E):
        m.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];gg=gfeat(m,(Xt[idx]-fmean)/fstd)
            loss=F.mse_loss(m.head(gg),yt[idx])+3.0*F.mse_loss(aux(gg),at[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sch.step()
        if ep>=ss and (ep-ss)%5==0: snaps.append({k:v.detach().cpu().clone() for k,v in m.state_dict().items()})
    sw=build_model(load_cfg()).to(dev);sw.load_state_dict(avg_state(snaps));update_bn(sw,Xt,fmean,fstd);sw.eval()
    return sw,(fmean,fstd,lmean,lstd)
def teacher_pred(teachers,Xe):
    ps=[]
    for m,(fm,fs,lm,ls) in teachers:
        with torch.no_grad(): ps.append((m((Xe-fm)/fs)*ls+lm))
    return torch.stack(ps,0).mean(0)
def run(cfg):
    M=cfg['M'];C=cfg['C'];jit=cfg['jit'];lam_true=cfg['lam_true']
    oof=np.zeros(N);t0=time.time()
    for k in range(K):
        va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
        teachers=[train_teacher(tr,100+i,None) for i in range(M)]
        # build distill set
        Xt=X[tr].to(dev);n=len(tr)
        Xs=[Xt];  # include clean
        for _ in range(C):
            b=Xt.clone()
            ridx=torch.randint(0,b.shape[1],(n,b.shape[1]),device=dev)
            b=torch.gather(b,1,ridx.unsqueeze(-1).expand(-1,-1,3))
            b=b+torch.randn_like(b)*jit
            Xs.append(b)
        Xd=torch.cat(Xs,0)
        with torch.no_grad(): Td=teacher_pred(teachers,Xd)  # [ (C+1)*n, 1] physical
        Ytrue_rep=Y[tr].to(dev).repeat(C+1,1)
        clean_mask=torch.zeros(Xd.shape[0],device=dev);clean_mask[:n]=1.0
        # student
        fmean=X[tr].reshape(-1,3).mean(0).to(dev);fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
        lmean=Td.mean(0);lstd=Td.std(0).clamp_min(1e-6)
        tdn=((Td-lmean)/lstd);ytruen=((Ytrue_rep-lmean)/lstd)
        torch.manual_seed(0);np.random.seed(0)
        st=build_model(load_cfg()).to(dev)
        E=cfg['epochs'];ss=int(E*0.6)
        opt=torch.optim.AdamW(st.parameters(),lr=1e-3,weight_decay=1e-4)
        sch=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:(0.5*(1+np.cos(np.pi*e/ss)) if e<ss else 0.25))
        nd=Xd.shape[0];bs=64;snaps=[]
        for ep in range(E):
            st.train();pm=torch.randperm(nd,device=dev)
            for s in 
…[truncated]

output

started 799

This is the key experiment. It trains 4 teachers per fold (with SWA+aux) and a student distilling the ensemble over jittered clouds. Let me wait , this is expensive (~4 teachers × ~15s + student ~20s, × 6 folds ≈ 8-10 min). Let me wait in chunks. While waiting, let me reflect on what result would make me deploy this: - If distill2 CV < 0.25 (beating single-model 0.267), it's worth deploying. - If ≈ 0.26-0.27, no better than single; fall back to single-seed SWA on 132. Let me wait ~4 min and check partial (it prints only at end though). Let me add periodic monitoring. Actually it prints only final. Let me just wait and check periodically if process alive. Let me wait.

This trains 4 SWA teachers + a distilled student per fold , the decisive test. Let me wait for it.

bash

cd /app && sleep 260 && cat distill2.log; echo "alive:"; ps aux|grep distill2|grep -v grep|wc -l

output

alive:
1

bash

cd /app && sleep 200 && cat distill2.log; echo "alive:"; ps aux|grep distill2|grep -v grep|wc -l

output

DISTILL2 {'M': 4, 'C': 6, 'jit': 0.01, 'lam_true': 0.3, 'epochs': 120} relL2=0.2961 mae=0.1902 maxe=0.7851 (432s)
alive:
0

Ensemble distillation CV = 0.296 , WORSE than single-model CV (0.267). So distillation still doesn't help; it's worse. The jitter-based distillation and lam_true mixing hurt. I've now definitively confirmed: distillation approaches don't beat the straightforward single-model training in honest CV. The student can't capture the ensemble benefit because on the train manifold teacher≈truth, and the jittered neighborhood distillation just adds noise/blur. So the honest pooled 6-fold CV numbers are: - Single PN (lam_aux3 + SWA): 0.267 - Everything else: ≥ 0.267 The val-split numbers (0.17-0.24) are optimistic because that specific 32-sample split is easier. Conclusion: Deploy the single PN with lam_aux=3 + SWA, trained on ALL 132 samples. This is my best honest, robust, deployable model. Expected real-test rel_l2 somewhere ~0.22-0.28 depending on test difficulty (val-like → ~0.21, CV-like → ~0.27). Now, one consideration about seed variance for the final single model: since I deploy ONE model and seeds vary (val 0.18-0.26), I want to pick a robust configuration. Options: 1. Just use seed0 (used in all CV, consistent). 2. Average weights across seeds , fails (different basins). I'll use seed0 for consistency with CV. But to reduce variance a bit more, I could extend the SWA collection window (more snapshots) for the final model. Actually, here's a thought to reduce variance WITHOUT ensembling: train the final model LONGER with a longer SWA tail (more snapshots averaged along the trajectory) , averages out more noise. And/or use a slightly larger batch for more stable BN. These are low-risk. Let me also reconsider: should I train the final on all 132, or keep val as a check? The safest for the GRADED test is to use ALL available labeled data (132) , more data reduces variance and bias. CV (110→22) estimated 0.267; with 132 training it should be ≤ that. I'll train final on 132. But I lose the ability to verify the exact final model. That's acceptable , the recipe is validated by CV, and I verified train→val reproduces reasonable numbers. Hmm, but given seed variance, let me do something smart for robustness: train the final model on 132 with the validated recipe, AND verify its predictions are sensible (correlation with a quick feature-baseline, reasonable range). I can't compute its test error, but I can sanity-check outputs. Actually, let me reconsider whether to include val in training at all, given I want SOME signal that the final model is good. Compromise: - Train final on ALL 132 (deploy this). - Separately, the train→val sanity (train on 100) already confirmed recipe works (0.17-0.24). I'm comfortable deploying the 132-trained model. Let me now also reconsider: the SWA model's seed. Since I can't validate the 132-model, and seed0 happened to be good on val AND is the CV seed, I'll use seed0. To hedge seed variance, I'll use a longer SWA tail (epochs 120-280, every 4) to average ~40 snapshots , more averaging → lower variance, more robust. Let me slightly extend. Actually, extending the SWA tail with constant-ish LR means the model wanders and snapshots are more diverse → averaging gives a flatter, more robust solution. Good. But too long might drift. Let me use epochs=300, swa_start=150, swa_lr=5e-4, swa_every=3 → ~50 snapshots. Let me reconsider: does a longer SWA tail actually help CV? I didn't test extensively. The earlier SWA (start120,250ep,every5 → 26 snaps) gave 0.267. Let me not over-engineer; I'll use swa_start=130, epochs=280, every=4 (~37 snaps). Close enough. Let me finalize the FINAL training on 132 and save. Let me also make sure to save exactly the required keys with correct shapes, and that loading into PointNetRegressor works and forward produces [B,1]. Let me reconsider label_mean/label_std: I computed over Y[tr] (physical masses). For the final, compute over all 132. Save as length-1 tensors. feat_mean/std length-3. Also I should double-check the grader convention once more: pred_phys = …[truncated]

Ensemble distillation is also worse in honest CV (0.296). I've now conclusively validated that the straightforward single PN (lam_aux=3 + SWA) at **CV ≈ 0.267** is the best *deployable* option , the max-pool architecture is the true ceiling, and every workaround (distillation, two-phase, heavy aug) fails to beat it honestly. Let me build the final deliverable: train that validated recipe on all 132 samples with a longer SWA tail for robustness, then save and verify the checkpoint loads into a fresh `PointNetRegressor`.

bash

cd /app && cat > /app/final_deploy.py << 'PY'
import torch, numpy as np, json
import torch.nn as nn, torch.nn.functional as F
from pathlib import Path
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
def gfeat(m,xb):
    x=xb.transpose(1,2).contiguous();x=m.feature(x);x=m.lift(x);return x.max(dim=2).values
def avg_state(ss):
    o={}
    for k in ss[0]:
        o[k]=torch.stack([s[k].float() for s in ss],0).mean(0) if ss[0][k].dtype.is_floating_point else ss[0][k].clone()
    return o
def train_final(X,Y,seed=0,E=280,ss=130,every=4,lam_aux=3.0):
    n=X.shape[0]
    fmean=X.reshape(-1,3).mean(0).to(dev);fstd=X.reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y.mean(0).to(dev);lstd=Y.std(0).clamp_min(1e-6).to(dev)
    A=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
    amean=A.mean(0).to(dev);astd=A.std(0).clamp_min(1e-6).to(dev)
    Xt=X.to(dev);yt=((Y.to(dev)-lmean)/lstd);at=((A.to(dev)-amean)/astd)
    torch.manual_seed(seed);np.random.seed(seed)
    m=build_model(load_cfg()).to(dev);aux=nn.Linear(256,A.shape[1]).to(dev)
    opt=torch.optim.AdamW(list(m.parameters())+list(aux.parameters()),lr=1e-3,weight_decay=1e-4)
    sch=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:(0.5*(1+np.cos(np.pi*e/ss)) if e<ss else 0.5))
    bs=16;snaps=[]
    for ep in range(E):
        m.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];gg=gfeat(m,(Xt[idx]-fmean)/fstd)
            loss=F.mse_loss(m.head(gg),yt[idx])+lam_aux*F.mse_loss(aux(gg),at[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sch.step()
        if ep>=ss and (ep-ss)%every==0: snaps.append({k:v.detach().cpu().clone() for k,v in m.state_dict().items()})
    sw=build_model(load_cfg()).to(dev);sw.load_state_dict(avg_state(snaps));update_bn(sw,Xt,fmean,fstd);sw.eval()
    return sw,fmean.cpu(),fstd.cpu(),lmean.cpu(),lstd.cpu()

Xtr,ytr=load_split('train');Xva,yva=load_split('val')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0)
print('Training FINAL on %d samples...'%X.shape[0],flush=True)
sw,fmean,fstd,lmean,lstd=train_final(X,Y,seed=0)

out=Path('/app/outputs');out.mkdir(parents=True,exist_ok=True)
payload={'state_dict':{k:v.cpu() for k,v in sw.state_dict().items()},
         'feat_mean':fmean.squeeze().float(),'feat_std':fstd.squeeze().float(),
         'label_mean':lmean.squeeze().float().reshape(1),'label_std':lstd.squeeze().float().reshape(1)}
torch.save(payload, out/'model.pt')
print('saved',out/'model.pt',flush=True)
print('feat_mean',payload['feat_mean'].tolist(),'feat_std',payload['feat_std'].tolist())
print('label_mean',payload['label_mean'].tolist(),'label_std',payload['label_std'].tolist())
PY
python3 final_deploy.py

output

Training FINAL on 132 samples...
saved /app/outputs/model.pt
feat_mean [-0.011533193290233612, 0.006913360673934221, -0.07176374644041061] feat_std [0.1420847624540329, 0.26909521222114563, 0.08743518590927124]
label_mean [0.7511226534843445] label_std [0.3936556279659271]

The final model is trained and saved. Now let me VERIFY the checkpoint loads into a fresh PointNetRegressor exactly as the grader would, and that forward produces [B,1], and reproduce the inference convention. Let me write a verifier that mimics the grader: - Load candidate_model.PointNetRegressor with config sizes. - Load state_dict. - Load feat_mean/std, label_mean/std. - pred = model((x - feat_mean)/feat_std) * label_std + label_mean. - Run on val (sanity, optimistic since val in training) and public_test (also not truly held-out but separate-ish... public_test was NOT in training! I only used train+val). So public_test IS a genuine held-out set for this final model. Let me evaluate on public_test , that's an honest generalization number! public_test (16 samples) was never in training. Let me evaluate the saved checkpoint on public_test via the exact grader convention. This gives an honest estimate. Let me write verify.py.

Now let me rigorously verify the saved checkpoint loads into a **fresh** `PointNetRegressor` exactly as the grader will, and evaluate it on `public_test` , which was **never** used in training, giving an honest held-out estimate.

bash

cd /app && cat > /app/verify.py << 'PY'
import torch, numpy as np, json
from pathlib import Path
from candidate_model import PointNetRegressor
from lib_train import load_split, metrics
cfg=json.load(open('/app/task_config.json'))['model']
# Load EXACTLY as a grader would
ckpt=torch.load('/app/outputs/model.pt', weights_only=True)
model=PointNetRegressor(feature_width=cfg['feature_width'],global_width=cfg['global_width'],head_width=cfg['head_width'])
missing,unexpected=model.load_state_dict(ckpt['state_dict'],strict=True)
model.eval()
fmean=ckpt['feat_mean'].float();fstd=ckpt['feat_std'].float()
lmean=ckpt['label_mean'].float();lstd=ckpt['label_std'].float()
print('shapes: feat_mean',tuple(fmean.shape),'feat_std',tuple(fstd.shape),'label_mean',tuple(lmean.shape),'label_std',tuple(lstd.shape))
def predict(x):
    with torch.no_grad():
        return (model((x-fmean)/fstd)*lstd+lmean)
# shape check
dummy=torch.randn(5,2048,3)
out=predict(dummy); print('output shape for [5,2048,3]:',tuple(out.shape))
assert out.shape==(5,1), 'BAD OUTPUT SHAPE'
# Honest held-out: public_test (never used in training)
for split in ['public_test','val','train']:
    Xs,ys=load_split(split)
    p=predict(Xs).numpy().ravel()
    r=metrics(p,ys.numpy().ravel())
    tag='(HELD-OUT, honest)' if split=='public_test' else '(in-training, optimistic)'
    print('%-11s relL2=%.4f mae=%.4f maxe=%.4f %s'%(split,r[0],r[1],r[2],tag))
# prediction sanity
Xs,ys=load_split('public_test');p=predict(Xs).numpy().ravel();t=ys.numpy().ravel()
print('pred range [%.3f,%.3f] true range [%.3f,%.3f]'%(p.min(),p.max(),t.min(),t.max()))
print('CHECKPOINT OK')
PY
python3 verify.py

output

shapes: feat_mean (3,) feat_std (3,) label_mean (1,) label_std (1,)
output shape for [5,2048,3]: (5, 1)
public_test relL2=0.3016 mae=0.1875 maxe=0.5151 (HELD-OUT, honest)
val         relL2=0.0849 mae=0.0564 maxe=0.1604 (in-training, optimistic)
train       relL2=0.0969 mae=0.0606 maxe=0.2531 (in-training, optimistic)
pred range [0.357,1.466] true range [0.198,1.449]
CHECKPOINT OK

The checkpoint loads correctly into a fresh PointNetRegressor, produces [B,1] output, and the inference convention works. Honest held-out on public_test (never in training): relL2=0.3016, mae=0.1875, maxe=0.5151. Hmm, 0.30 on public_test is worse than my CV estimate (0.267). And the train/val (in-training) are ~0.085-0.097 , the model fits training data well but generalizes to ~0.30 on the truly-unseen public_test. This suggests overfitting. Wait , public_test has only 16 samples, so it's noisy too. But 0.30 is concerning. Let me investigate: the model fits train+val at 0.09 but public_test at 0.30. That gap indicates overfitting. Let me reconsider. My CV (6-fold on 132) gave 0.267 for this recipe. public_test gives 0.30. These are both in the same ballpark (0.27-0.30), with public_test being a single 16-sample draw. So ~0.28-0.30 is the realistic honest number. Can I do better? Let me reconsider. The in-training fit (0.09) being much better than held-out (0.30) confirms overfitting. SWA didn't fully prevent it. Important realization: public_test is a TRUE held-out set I can use to guide final model selection (as long as I don't overfit to its 16 samples). Let me use it as an additional honest check while choosing between a few final recipes , but carefully, since 16 samples is tiny and I risk overfitting to it. Actually, the most important question: is there a recipe that generalizes better? My CV said everything is ~0.26-0.27. public_test at 0.30 for this one. Let me get CV AND public_test for a couple of variations to triangulate, but avoid overfitting to public_test. Let me reconsider the overfitting. The model fits train+val to 0.09. That's quite tight. More regularization might help generalization. But CV said wd didn't help much. However, the FINAL model trained on 132 with 280 epochs might overfit MORE than the CV folds (which I stopped around similar). Let me reconsider epochs. Maybe 280 epochs is too many for the final. The CV curve plateaued ~120-150 then was flat/noisy. For the final on 132, maybe fewer epochs generalize better. But SWA averages the tail so it shouldn't overfit much... Actually, let me reconsider a cleaner, potentially-better-generalizing recipe: stronger regularization via SMALLER model effective capacity. I can't change architecture, but I noticed in-training fit is 0.09 (not 0.014 like pure MSE) because aux supervision regularizes. Still overfits to 0.30 on held-out. Let me reconsider: maybe the issue is that the FINAL model on 132 with this specific recipe landed on an unlucky spot. Let me evaluate a FEW seeds on public_test to gauge variance (like I did on val). If seed variance is high, I should pick a robust approach. But again, I must not overfit to public_test's 16 samples. Let me use it only to estimate variance/ballpark, and rely on CV for the decision. Let me step back and think about the best DECISION process: - My honest generalization estimate is ~0.27 (CV) to ~0.30 (public_test). - Seed variance is significant. To reduce variance for the single deployed model robustly, the ONE thing that reliably reduces variance is ENSEMBLING, which I can't deploy. BUT WAIT , I keep saying I can't deploy an ensemble. Let me reconsider: a single PointNetRegressor forward is fixed. But what if I reconsider the claim that I can't average different-seed weights... I confirmed soup fails (0.47). OK here's the thing though: the real improvement lever I have NOT fully exploited is that public_test is available as held-out labeled data! I can INCLUDE public_test in training too (train+val+public_test = 148 samples), giving more data. But then I lose my only honest held-out check. However, more data generally improves generalization. The grader's true test is a SEPARATE held-out set (not public_test necessarily , but the task says "applied to a held-out set" and public_test is provided to me). Hmm, the task says predictions are compared on a "held-out set of bracket geometries." public …[truncated]

The checkpoint is structurally correct (loads strict into a fresh `PointNetRegressor`, outputs `[B,1]`, predictions in the right physical range). The honest held-out `public_test` gives rel_L2 ≈ 0.30 (16 samples, noisy) vs CV 0.267 , consistent ~0.27–0.30 ballpark, with some overfitting (in-training ≈ 0.09). Two improvements I should make: (1) I have **148** labeled samples total (train+val+public_test) and the real grading set is hidden , so I should train the final on all of them; (2) let me confirm the regularization via a tighter 8-fold CV over all 148 before committing.

bash

cd /app && cat > /app/cv148.py << 'PY'
import torch, numpy as np, time, sys, json
import torch.nn as nn, torch.nn.functional as F
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
parts=[load_split(s) for s in ['train','val','public_test']]
X=torch.cat([p[0] for p in parts],0);Y=torch.cat([p[1] for p in parts],0);yv=Y.numpy().ravel()
N=X.shape[0];g=torch.Generator().manual_seed(7);perm=torch.randperm(N,generator=g)
K=8;folds=[perm[i::K] for i in range(K)]
AUXALL=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
def gfeat(m,xb):
    x=xb.transpose(1,2).contiguous();x=m.feature(x);x=m.lift(x);return x.max(dim=2).values
def avg_state(ss):
    o={}
    for k in ss[0]: o[k]=torch.stack([s[k].float() for s in ss],0).mean(0) if ss[0][k].dtype.is_floating_point else ss[0][k].clone()
    return o
def fold(tr,va,cfg,seed):
    fmean=X[tr].reshape(-1,3).mean(0).to(dev);fstd=X[tr].reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y[tr].mean(0).to(dev);lstd=Y[tr].std(0).clamp_min(1e-6).to(dev)
    A=AUXALL[tr];amean=A.mean(0).to(dev);astd=A.std(0).clamp_min(1e-6).to(dev)
    Xt=X[tr].to(dev);yt=((Y[tr].to(dev)-lmean)/lstd);at=((A.to(dev)-amean)/astd)
    torch.manual_seed(seed);np.random.seed(seed)
    m=build_model(load_cfg()).to(dev);aux=nn.Linear(256,A.shape[1]).to(dev)
    E=cfg['E'];ss=cfg['ss'];opt=torch.optim.AdamW(list(m.parameters())+list(aux.parameters()),lr=1e-3,weight_decay=cfg['wd'])
    sch=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:(0.5*(1+np.cos(np.pi*e/ss)) if e<ss else 0.5))
    n=len(tr);bs=16;snaps=[]
    for ep in range(E):
        m.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];gg=gfeat(m,(Xt[idx]-fmean)/fstd)
            loss=F.mse_loss(m.head(gg),yt[idx])+cfg['lam']*F.mse_loss(aux(gg),at[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sch.step()
        if ep>=ss and (ep-ss)%cfg['ev']==0: snaps.append({k:v.detach().cpu().clone() for k,v in m.state_dict().items()})
    sw=build_model(load_cfg()).to(dev);sw.load_state_dict(avg_state(snaps));update_bn(sw,Xt,fmean,fstd);sw.eval()
    with torch.no_grad(): return (sw((X[va].to(dev)-fmean)/fstd)*lstd+lmean).cpu().numpy().ravel()
def run(cfg,seeds=(0,)):
    t=time.time();oofs=[]
    for seed in seeds:
        oof=np.zeros(N)
        for k in range(K):
            va=folds[k];tr=torch.cat([folds[j] for j in range(K) if j!=k])
            oof[va.numpy()]=fold(tr,va,cfg,seed)
        oofs.append(oof)
    oof=np.mean(oofs,0)
    print('%-50s [seeds avg %d] relL2=%.4f mae=%.4f maxe=%.4f (%.0fs)'%(str(cfg),len(seeds),*metrics(oof,yv),time.time()-t),flush=True)
if __name__=='__main__':
    base=dict(E=260,ss=130,wd=1e-4,lam=3.0,ev=4)
    for over in json.loads(sys.argv[1]):
        cfg=dict(base);cfg.update(over);run(cfg,seeds=tuple(cfg.pop('seeds',[0])))
PY
nohup python3 cv148.py '[{"wd":1e-4},{"wd":5e-4},{"E":200,"ss":110}]' > /app/cv148.log 2>&1 &
echo "started $!"

output

started 865

bash

cd /app && sleep 280 && cat cv148.log; echo "alive:"; ps aux|grep cv148|grep -v grep|wc -l

output

{'E': 260, 'ss': 130, 'wd': 0.0001, 'lam': 3.0, 'ev': 4} [seeds avg 1] relL2=0.2794 mae=0.1764 maxe=0.7729 (120s)
{'E': 260, 'ss': 130, 'wd': 0.0005, 'lam': 3.0, 'ev': 4} [seeds avg 1] relL2=0.2727 mae=0.1739 maxe=0.7654 (116s)
alive:
1

8-fold CV on 148 samples (seed0): - wd=1e-4: 0.2794 - wd=5e-4: 0.2727 (slightly better) This confirms ~0.27-0.28 generalization, consistent with everything. wd=5e-4 marginally better than 1e-4. Let me wait for the 3rd config (E=200, ss=110). These honest numbers (0.27-0.28 CV, 0.30 public_test) tell me the realistic performance is ~0.27-0.29. Let me reconsider seed averaging for the OOF estimate , single seed is noisy. But more importantly, the DEPLOYED model is a single seed. I should pick a recipe that's robustly good. wd=5e-4 looks marginally better. Let me also reconsider: the maxe ~0.77 is high (one sample badly predicted). The worst-case error is a secondary metric. Ridge had maxe 0.49. The PN's high maxe drags rel_l2. Hmm, let me think about whether I can reduce the worst-case. The worst sample is likely an outlier in shape (high or low mass extreme). The PN extrapolates poorly. Hard to fix without better features. Let me just finalize with the best robust recipe. Let me wait for the 3rd config result, then decide between wd5e-4 (E260) and the shorter schedule. Actually, given the marginal differences, let me also consider averaging multiple seeds for the OOF is just for estimation; for deployment I use one seed. To reduce deployment variance, I could train the final with a LONGER SWA tail (more snapshots) which flattens the solution. Let me wait for 3rd result.

Let me get the third config and decide on the final regularization.

bash

cd /app && sleep 90 && cat cv148.log; echo "alive:"; ps aux|grep cv148|grep -v grep|wc -l

output

{'E': 260, 'ss': 130, 'wd': 0.0001, 'lam': 3.0, 'ev': 4} [seeds avg 1] relL2=0.2794 mae=0.1764 maxe=0.7729 (120s)
{'E': 260, 'ss': 130, 'wd': 0.0005, 'lam': 3.0, 'ev': 4} [seeds avg 1] relL2=0.2727 mae=0.1739 maxe=0.7654 (116s)
{'E': 200, 'ss': 110, 'wd': 0.0001, 'lam': 3.0, 'ev': 4} [seeds avg 1] relL2=0.2772 mae=0.1779 maxe=0.7478 (89s)
alive:
0

All three ~0.27-0.28. wd=5e-4 (E260) is marginally best at 0.2727. The differences are within noise. Let me make a final decision. Given: - Honest CV (8-fold, 148): ~0.273-0.279 - public_test (16): 0.30 - Everything hovers ~0.27-0.30 I'll deploy the final model trained on ALL 148 samples with wd=5e-4, lam_aux=3, SWA (E=260, ss=130, every=4). This uses maximum data and the marginally-best regularization. But I'm bothered by the seed variance and want the single deployed model to be robust, not a bad-luck seed. Let me address this: Since I train the final on all 148 and can't hold out, I'll reduce variance by using a longer SWA tail (more snapshots averaged → flatter minimum, lower variance). Actually, here's a legitimate variance-reduction idea for the DEPLOYED single model that I should reconsider once more: I can't average different-seed full weights (soup fails). But what about this: the seed variance largely comes from the feature extractor converging to different solutions. SWA within a run averages along one trajectory. Let me just accept ~0.27 and deploy robustly. To hedge seed luck, let me quickly check 2-3 seeds of the FINAL recipe on public_test (honest held-out, not in the 148... wait, public_test WILL be in the 148 if I train on all 148). Decision on training data: Should I include public_test in final training or keep it as held-out validation for seed selection? Option A: Train on all 148 (max data). No held-out. Risk: can't verify, seed luck. Option B: Train on 132 (train+val), keep public_test (16) as held-out to SELECT the best seed, then deploy. This gives seed robustness but uses less data and risks overfitting seed selection to 16 samples. Given the real grading test is hidden and separate, and public_test is just another sample from the distribution, including it in training (Option A) gives more data which reliably helps. But seed selection (Option B) could avoid a bad-luck seed. Compromise Option C: Train on all 148, but reduce seed variance by using a robust training approach. Since I can't ensemble, I'll train with a LONG SWA tail and pick seed0 (consistent with CV). Actually, let me reconsider Option B's merit: selecting seed on 16 held-out samples is risky (overfits to those 16). And the deployed model would be trained on only 132. I think Option A (all 148, seed0, good SWA) is more principled. But let me hedge: let me train the final on all 148 with MULTIPLE seeds and SWA, then... I can only save one. Let me just use seed0. Hmm, wait. Let me reconsider whether there's a BETTER way to use my compute for the single model. One more idea for robustness within a single model: train the final model and average the SWA snapshots from a LONGER tail with a slightly higher constant SWA LR (so snapshots are more diverse within the basin), which approximates a mini-ensemble in one model. I already do SWA with swa_lr=5e-4. Let me keep it. OK, let me also reconsider: my current saved model (132 samples, wd1e-4, E280/ss130) got public_test 0.30. Let me retrain final on all 148 with wd5e-4 (marginally best) and SWA, and save. Since public_test is now in training, I can't re-estimate honestly, but CV says ~0.273. Actually, let me make the decision to KEEP public_test as an honest held-out AND use it to pick among a small number of robust candidate final models (trained on 132), which is a legitimate, low-risk model-selection use (not training on it). But 16 samples is very noisy for selection. Alternatively, I train on 148 and trust CV. Let me go with: train final on ALL 148 (maximum data, best for the hidden test), wd=5e-4, lam_aux=3, SWA with a slightly longer tail (E=300, ss=140, every=3) for robustness. This is my best shot. I'll deploy seed0. Let me reconsider the seed question one final time by being empirical but careful: I'll train the final recipe on 132 (holding out public_test) for seeds 0,1,2, evaluate each on public_test, AND evaluate their ENSEMBLE. This tells me (a) seed variance …[truncated]

All variants land ~0.27; wd=5e-4 is marginally best. Before committing, let me run a quick **honest diagnostic**: train the recipe on 132 (holding out `public_test`) for 3 seeds, and check individual vs ensemble performance on the true held-out `public_test`. This tells me the seed variance and whether seed0 is safe , used only as a diagnostic, not for training.

bash

cd /app && cat > /app/diag.py << 'PY'
import torch, numpy as np
from final_deploy import train_final
from lib_train import load_split, metrics
Xtr,ytr=load_split('train');Xva,yva=load_split('val');Xpt,ypt=load_split('public_test')
X132=torch.cat([Xtr,Xva],0);Y132=torch.cat([ytr,yva],0)
preds=[]
for sd in [0,1,2]:
    sw,fm,fs,lm,ls=train_final(X132,Y132,seed=sd,E=260,ss=130,every=4,lam_aux=3.0)
    with torch.no_grad():
        p=(sw((Xpt.cuda()-fm.cuda())/fs.cuda())*ls.cuda()+lm.cuda()).cpu().numpy().ravel()
    preds.append(p)
    print('seed%d public_test relL2=%.4f mae=%.4f maxe=%.4f'%(sd,*metrics(p,ypt.numpy().ravel())),flush=True)
P=np.stack(preds,0)
for kk in [2,3]:
    print('ENS k=%d public_test relL2=%.4f mae=%.4f maxe=%.4f'%(kk,*metrics(P[:kk].mean(0),ypt.numpy().ravel())),flush=True)
PY
python3 diag.py

output

Training FINAL on 132 samples...
saved /app/outputs/model.pt
feat_mean [-0.011533193290233612, 0.006913360673934221, -0.07176374644041061] feat_std [0.1420847624540329, 0.26909521222114563, 0.08743518590927124]
label_mean [0.7511226534843445] label_std [0.3936556279659271]
seed0 public_test relL2=0.3267 mae=0.2045 maxe=0.5757
seed1 public_test relL2=0.3474 mae=0.2164 maxe=0.7359
seed2 public_test relL2=0.2815 mae=0.1767 maxe=0.5794
ENS k=2 public_test relL2=0.3308 mae=0.2090 maxe=0.6558
ENS k=3 public_test relL2=0.3098 mae=0.1909 maxe=0.6303

Oops , importing final_deploy RE-RAN the final training (the module-level code executed) and OVERWROTE /app/outputs/model.pt with the 132-sample wd1e-4 model again. I see "Training FINAL on 132 samples... saved". That's because final_deploy.py has module-level training code. The diagnostic then trained 3 more seeds. Also the diagnostic results on public_test (training on 132, eval public_test): - seed0: 0.327 - seed1: 0.347 - seed2: 0.282 - ENS k=3: 0.310 So on this 16-sample held-out, the models get 0.28-0.35, ensemble 0.31. These are worse than my earlier public_test 0.30 (that was the 132-model wd1e-4 E280). Hmm, so public_test seems to be a genuinely harder 16-sample set (~0.30-0.33), while the 8-fold CV on 148 gave 0.27. The difference: public_test might contain some hard outliers. With only 16 samples, one bad prediction (maxe 0.57-0.74) heavily impacts rel_l2. Notably, ensembling did NOT help on public_test here (0.31 vs individual 0.28-0.35) , consistent with my earlier finding that errors are correlated. seed2 (0.28) was best individually. This confirms the realistic honest performance is ~0.27 (CV) to ~0.31 (public_test), and ensembling doesn't reliably help. The key takeaways: 1. Performance is ~0.27-0.31 regardless of approach , architecture ceiling. 2. Seed variance exists but ensembling doesn't help (correlated errors). 3. public_test (16) is noisier/harder than the pooled CV. Given all this, the best decision is to train the final model on ALL available labeled data (148 samples) to maximize generalization to the hidden test, using the validated recipe. More training data is the most reliable way to improve generalization here. I need to FIX final_deploy.py so importing it doesn't retrain/overwrite, and make the final model train on all 148. Let me rewrite final_deploy.py cleanly: - Guard the training code under `if __name__=='__main__'`. - Train on all 148 (train+val+public_test). - Use wd=5e-4 (marginally best in CV), lam_aux=3, SWA E=280 ss=140 every=3 (longer tail for robustness). - Save and verify. Wait , should I train on 148 (including public_test) or keep public_test out? The hidden grading test is separate from public_test. Including public_test gives 16 more training samples (~12% more data) → better generalization to the hidden test. Since public_test isn't the grading set, including it is pure benefit. I'll train on all 148. But I lose public_test as a sanity check. That's OK , I've validated the recipe via CV and verified the pipeline loads/runs. Regarding seed: since ensembling doesn't help and seed2 was best on public_test (but that's overfitting to 16 samples), I'll just use seed0 for consistency with CV (no cherry-picking). Actually, let me reconsider , the 8-fold CV on 148 used seed0 and gave 0.273-0.279. seed0 is a fair, non-cherry-picked choice. Let me rewrite final_deploy.py properly and train on 148. Let me also reconsider: the longer SWA tail , does it help? Let me not over-tune; I'll use E=280, ss=140, every=3 (~47 snapshots) for a flat, robust solution. This is reasonable. Let me rewrite final_deploy.py with the __main__ guard and train on 148. Let me reconsider wd: CV on 148 showed wd5e-4 (0.2727) slightly better than wd1e-4 (0.2794). I'll use wd5e-4. Let me write the clean final script.

The diagnostic (training on 132, honest `public_test` eval) shows seeds at 0.28–0.35 and ensembling again *not* helping (0.31) , confirming correlated errors and an architecture ceiling around ~0.27–0.31. (It also re-ran the module-level code and overwrote the checkpoint , I'll fix that with a `__main__` guard.) Decision: train the final model on **all 148** labeled samples (the grading set is separate, so every sample helps) using the CV-best regularization (wd=5e-4) and a longer SWA tail. Let me write a clean, guarded final script.

bash

cd /app && cat > /app/final_deploy.py << 'PY'
import torch, numpy as np, json
import torch.nn as nn, torch.nn.functional as F
from pathlib import Path
from lib_train import load_split, metrics, update_bn, load_cfg
from candidate_model import build_model
from feat import rich_feats
dev='cuda'
def gfeat(m,xb):
    x=xb.transpose(1,2).contiguous();x=m.feature(x);x=m.lift(x);return x.max(dim=2).values
def avg_state(ss):
    o={}
    for k in ss[0]:
        o[k]=torch.stack([s[k].float() for s in ss],0).mean(0) if ss[0][k].dtype.is_floating_point else ss[0][k].clone()
    return o
def train_final(X,Y,seed=0,E=280,ss=140,every=3,lam_aux=3.0,wd=5e-4):
    n=X.shape[0]
    fmean=X.reshape(-1,3).mean(0).to(dev);fstd=X.reshape(-1,3).std(0).clamp_min(1e-6).to(dev)
    lmean=Y.mean(0).to(dev);lstd=Y.std(0).clamp_min(1e-6).to(dev)
    A=torch.tensor(rich_feats(X, res=(3,4,5,6,8,10,12,16,20), grids=()),dtype=torch.float32)
    amean=A.mean(0).to(dev);astd=A.std(0).clamp_min(1e-6).to(dev)
    Xt=X.to(dev);yt=((Y.to(dev)-lmean)/lstd);at=((A.to(dev)-amean)/astd)
    torch.manual_seed(seed);np.random.seed(seed)
    m=build_model(load_cfg()).to(dev);aux=nn.Linear(256,A.shape[1]).to(dev)
    opt=torch.optim.AdamW(list(m.parameters())+list(aux.parameters()),lr=1e-3,weight_decay=wd)
    sch=torch.optim.lr_scheduler.LambdaLR(opt,lambda e:(0.5*(1+np.cos(np.pi*e/ss)) if e<ss else 0.5))
    bs=16;snaps=[]
    for ep in range(E):
        m.train();aux.train();pm=torch.randperm(n,device=dev)
        for s in range(0,n,bs):
            idx=pm[s:s+bs];gg=gfeat(m,(Xt[idx]-fmean)/fstd)
            loss=F.mse_loss(m.head(gg),yt[idx])+lam_aux*F.mse_loss(aux(gg),at[idx])
            opt.zero_grad(set_to_none=True);loss.backward();opt.step()
        sch.step()
        if ep>=ss and (ep-ss)%every==0: snaps.append({k:v.detach().cpu().clone() for k,v in m.state_dict().items()})
    sw=build_model(load_cfg()).to(dev);sw.load_state_dict(avg_state(snaps));update_bn(sw,Xt,fmean,fstd);sw.eval()
    return sw,fmean.cpu(),fstd.cpu(),lmean.cpu(),lstd.cpu()

if __name__=='__main__':
    parts=[load_split(s) for s in ['train','val','public_test']]
    X=torch.cat([p[0] for p in parts],0);Y=torch.cat([p[1] for p in parts],0)
    print('Training FINAL on %d samples (seed=0, wd=5e-4, SWA tail)'%X.shape[0],flush=True)
    sw,fmean,fstd,lmean,lstd=train_final(X,Y,seed=0)
    out=Path('/app/outputs');out.mkdir(parents=True,exist_ok=True)
    payload={'state_dict':{k:v.cpu() for k,v in sw.state_dict().items()},
             'feat_mean':fmean.squeeze().float(),'feat_std':fstd.squeeze().float(),
             'label_mean':lmean.squeeze().float().reshape(1),'label_std':lstd.squeeze().float().reshape(1)}
    torch.save(payload, out/'model.pt')
    print('saved',out/'model.pt')
    print('feat_mean',payload['feat_mean'].tolist());print('feat_std',payload['feat_std'].tolist())
    print('label_mean',payload['label_mean'].tolist(),'label_std',payload['label_std'].tolist())
PY
python3 final_deploy.py && echo "=== VERIFY ===" && python3 verify.py

output

Training FINAL on 148 samples (seed=0, wd=5e-4, SWA tail)
saved /app/outputs/model.pt
feat_mean [-0.011780548840761185, 0.007426035590469837, -0.07198601961135864]
feat_std [0.1422092616558075, 0.2692946493625641, 0.08725058287382126]
label_mean [0.7487966418266296] label_std [0.38614240288734436]
=== VERIFY ===
shapes: feat_mean (3,) feat_std (3,) label_mean (1,) label_std (1,)
output shape for [5,2048,3]: (5, 1)
public_test relL2=0.0931 mae=0.0600 maxe=0.1474 (HELD-OUT, honest)
val         relL2=0.0953 mae=0.0631 maxe=0.2176 (in-training, optimistic)
train       relL2=0.0980 mae=0.0615 maxe=0.2564 (in-training, optimistic)
pred range [0.259,1.493] true range [0.198,1.449]
CHECKPOINT OK

Now the model is trained on all 148 (including public_test), so public_test is no longer held-out , hence the 0.093 (in-training). All splits now show ~0.09-0.10 since all were used in training. This confirms the pipeline works but I no longer have a held-out check. The checkpoint is valid: loads strict, outputs [B,1], predictions in sensible range [0.259, 1.493] vs true [0.198, 1.449]. My honest generalization estimate for this final model (trained on 148) is the 8-fold CV number ~0.273 (and likely slightly better than CV since CV trains on 148×7/8≈130 while final uses all 148). The hidden test should be ~0.27. Now, I've been assuming the architecture caps at ~0.27. Let me make sure I'm not leaving easy gains on the table. Let me reconsider the aux features used for supervision. The aux features include voxel occupancy (volume proxy) , good. But maybe I can improve the PRIMARY model by a better choice that I haven't tried: what if I add the aux-feature-PREDICTION as an additional INPUT signal to the head? No , can't change architecture. Let me reconsider whether there's a fundamentally different and BETTER deployable approach I've overlooked. The max-pool limitation for volume: I argued the head can sum occupancy indicators. What if I explicitly help by making the aux target be JUST the occupancy/volume features (the most mass-relevant), with higher weight, so the global feature strongly encodes volume? I used all 43 features equally. Let me reconsider , lam_aux=3 on all 43. Marginal gains only. I think ~0.27 is the realistic ceiling for this max-pool architecture on this data. I've done extensive exploration. Let me lock in the current model but consider whether I can squeeze a bit more via a final robustness measure. Actually, let me reconsider one thing: my CV used seed0 throughout. The final model uses seed0. But what if seed0 on the full 148 is unlucky? I can't check (no held-out). However, the 8-fold CV with seed0 gave a consistent 0.273, so seed0's behavior is validated across folds. I'm reasonably confident. But let me reconsider whether to use a longer SWA tail helps robustness. I used E=280, ss=140, every=3 → ~47 snapshots. That's a good amount of averaging. Good. Let me also reconsider: should I verify the final model against a feature-baseline sanity (correlation of predictions with true on the full 148)? It's 0.09 rel (fits well). Fine. Now, one concern: I trained on 148 INCLUDING public_test. If the grader's hidden test happens to BE or OVERLAP public_test... unlikely (it's a separate held-out). Including public_test is standard (use all labeled data). Fine. Let me reconsider whether I should instead NOT include public_test, to reduce risk and keep a held-out. The benefit of 16 extra samples (~12%) is real but modest. The risk of including it: none for the hidden test (separate). So include it. Keep current. Actually, wait. Let me reconsider the possibility that I'm overfitting the RECIPE to my CV. My CV consistently says ~0.27. The public_test (when held out) said ~0.30-0.33. There's a gap suggesting my CV might be slightly optimistic OR public_test is a hard draw. To be safe, the hidden test could be ~0.27-0.33. Is there anything that robustly reduces the WORST-case errors (which drive rel_l2 up on hard sets like public_test)? The maxe was ~0.5-0.7. Idea: Clipping predictions to the training label range [min,max] could reduce catastrophic outliers. But I can't post-process at inference (grader does fixed de-norm). However , I could bake a soft clipping into... no, the model output is de-normed linearly. Can't clip. Idea: train with Huber loss to be robust to label outliers? But I want to predict outliers well (they drive maxe). MSE emphasizes them. Actually for the WORST-CASE metric, I want to predict the extreme-mass samples accurately. MSE already focuses there. Hmm, actually reducing maxe is about not making huge errors on any sample. A smoother model (like ridge) has lower maxe. The PN's …[truncated]

The final model (trained on all 148 samples, seed0, wd=5e-4, long SWA tail) is saved and loads strictly into a fresh `PointNetRegressor`, outputs `[B,1]`, with predictions in a sensible physical range. All splits now show ~0.09 because all are in-training (so no longer a held-out check), but my honest generalization estimate from 8-fold CV is ~0.27. Let me do final housekeeping: confirm the checkpoint loads with `weights_only=True` (as the loader uses) and that all required keys have correct shapes/dtypes.

bash

cd /app && python3 -c "
import torch
from candidate_model import PointNetRegressor
import json
ck=torch.load('/app/outputs/model.pt', weights_only=True)
print('keys:', sorted(ck.keys()))
req=['state_dict','feat_mean','feat_std','label_mean','label_std']
for k in req: assert k in ck, 'MISSING '+k
for k in ['feat_mean','feat_std','label_mean','label_std']:
    print(' ',k,'shape',tuple(ck[k].shape),'dtype',ck[k].dtype,'->',ck[k].tolist())
m=json.load(open('/app/task_config.json'))['model']
net=PointNetRegressor(m['feature_width'],m['global_width'],m['head_width'])
net.load_state_dict(ck['state_dict'],strict=True)  # strict must pass
net.eval()
# exact grader-style inference
x=torch.randn(4,2048,3)
y=(net((x-ck['feat_mean'])/ck['feat_std'])*ck['label_std']+ck['label_mean'])
print('forward [4,2048,3] ->', tuple(y.shape))
assert y.shape==(4,1)
nparams=sum(p.numel() for p in net.parameters())
print('num state_dict tensors:', len(ck['state_dict']), '| model params:', nparams)
print('ALL CHECKS PASSED')
"
ls -la /app/outputs/

output

keys: ['feat_mean', 'feat_std', 'label_mean', 'label_std', 'state_dict']
  feat_mean shape (3,) dtype torch.float32 -> [-0.011780548840761185, 0.007426035590469837, -0.07198601961135864]
  feat_std shape (3,) dtype torch.float32 -> [0.1422092616558075, 0.2692946493625641, 0.08725058287382126]
  label_mean shape (1,) dtype torch.float32 -> [0.7487966418266296]
  label_std shape (1,) dtype torch.float32 -> [0.38614240288734436]
forward [4,2048,3] -> (4, 1)
num state_dict tensors: 41 | model params: 92289
ALL CHECKS PASSED
total 384
drwxr-sr-x 2 h2tagent h2tagent   4096 Jun 20 22:15 .
drwxrws--- 1 h2tagent h2tagent   4096 Jun 20 22:23 ..
-rw-r--r-- 1 h2tagent h2tagent 384685 Jun 20 22:25 model.pt

The checkpoint is fully valid: - All 5 required keys present. - feat_mean/std length-3, label_mean/std length-1, all float32. - state_dict loads STRICT into PointNetRegressor (41 tensors, 92289 params). - Forward on [B,2048,3] → [B,1]. - Grader-style inference works. The deliverable is complete and correct. My honest performance estimate is rel_L2 ≈ 0.27 (8-fold CV), MAE ≈ 0.17, max abs err ≈ 0.77. Let me reflect on whether I should try anything else to improve. I've extensively explored: - The max-pool architecture caps at ~0.27 because it can't compute volume/average features well. - Ridge on hand-crafted volume features gets 0.22 but isn't deployable in the required architecture. - Augmentation, distillation, two-phase, auxiliary supervision, ensembling , none beat the straightforward single PN in honest CV. I'm confident this is a solid, well-validated deliverable near the architecture's ceiling. Actually, let me reconsider whether I've truly maximized. The gap between the deployable PN (0.27) and ridge (0.22) bugs me. Let me reconsider the aux supervision ONE more time but with a crucial change: what if the aux target is ONLY the single best volume feature(s), with a HIGH weight, AND I also feed... no. Actually, let me reconsider the fundamental max-pool limitation claim once more with a concrete test: I'll check whether an AVERAGE-pool version (which I CAN'T deploy) would do much better. If avg-pool gets ~0.20, it confirms the architecture is the bottleneck (informative but I can't use it). If avg-pool also ~0.27, then it's a data/generalization issue, not pooling. This is diagnostic only , but it could inform whether there's ANY hope. Actually, I can't deploy avg-pool, so even if it's better, it doesn't help my deliverable. Not worth the tokens. Let me instead reconsider: is there a way to get avg-pool-like behavior into the max-pool model via the INPUT? If I could make each point's contribution... no. I've spent enough. The deliverable is complete and validated. Let me reconsider whether I should keep public_test OUT of training to hedge against the tiny chance the hidden test overlaps or to preserve a sanity monitor. The standard ML practice: use all labeled data for the final model when the test is separate. I'll keep the 148-trained model. Hmm, but actually , there's a subtle risk. My 8-fold CV (0.273) trained on ~130 and validated on ~18. The final model trains on 148. The recipe's hyperparameters (E, ss, lam, wd) were lightly tuned on CV. This is fine , minimal overfitting of hyperparameters given I only compared a handful. One last consideration: robustness of the single seed. Let me reconsider using a form of "within-basin" ensemble that IS deployable: I already use SWA (averaging ~47 snapshots along one trajectory), which is the deployable variance-reducer. Good. I'm confident. Let me also make sure there are no leftover background processes consuming resources, and that the final model.pt is the 148-trained one (timestamp 22:25, after my final run). Good. Let me write a brief summary of what I did. Actually, let me reconsider if I can quickly squeeze a bit more by trying a slightly different lam_aux or a two-target emphasis, but CV showed these are all within noise (~0.26-0.28). Not worth it. Let me do one final thing: reconsider whether training the final on 148 actually generalizes better than on 132. I can't directly measure, but more data → better. The CV on 148 (train on ~130) gave 0.273; the final on 148 should be marginally better. Actually, let me reconsider keeping a held-out for a final honest number AND still using lots of data: I could train the deployed model on 132 (train+val) and report public_test (honest) ~0.30. OR train on 148. The 148 model can't be measured but should be ≥ as good. Given the hidden test is what matters and more data helps, 148 is the right choice. But the 0.30 on public_test (for a 132 model) vs 0.273 CV makes me slightly worried the hidden test could be ~0.30 …[truncated]

All checks pass. The checkpoint has the 5 required keys with correct shapes (`feat_mean`/`feat_std` length-3, `label_mean`/`label_std` length-1, all float32), loads **strictly** into `PointNetRegressor` (41 tensors / 92,289 params), and the grader-style inference `model((x-feat_mean)/feat_std)*label_std+label_mean` yields `[B,1]`. Let me clean up any stray background processes and do one final integrity confirmation of the saved artifact.

bash

cd /app && pkill -f 'cv.py|cv_ens|cv_traj|curve.py|auxtrain|swatest|phase2|distill|cv148|diag' 2>/dev/null; sleep 1
echo "=== background procs (should be none) ===" && ps aux | grep -E 'python3' | grep -v grep | grep -v 'final_deploy' | wc -l
echo "=== final artifact ===" && python3 -c "
import torch, hashlib
ck=torch.load('/app/outputs/model.pt', weights_only=True)
print('file loads with weights_only=True OK; keys:', sorted(ck))
print('state_dict tensors:', len(ck['state_dict']))
print('sample keys:', list(ck['state_dict'])[:3])
"
ls -la /app/outputs/model.pt

output

=== background procs (should be none) ===
2
=== final artifact ===
file loads with weights_only=True OK; keys: ['feat_mean', 'feat_std', 'label_mean', 'label_std', 'state_dict']
state_dict tensors: 41
sample keys: ['feature.net.0.weight', 'feature.net.0.bias', 'feature.net.1.weight']
-rw-r--r-- 1 h2tagent h2tagent 384685 Jun 20 22:25 /app/outputs/model.pt

There are 2 python3 processes still. Let me check what they are , probably leftover grep false positives or something. Let me check.

Let me check those two remaining processes to ensure nothing is interfering.

bash

cd /app && ps aux | grep python3 | grep -v grep | awk '{print $2, $11, $12, $13}'

output

15 /bin/sh -c /opt/h2t-venv/bin/python3
16 /opt/h2t-venv/bin/python3 /mcp_server/server.py

Those are just the MCP server (the tool infrastructure), not my training jobs. All my background experiments are cleaned up. Good. The deliverable is complete: - `/app/outputs/model.pt` with all required keys, validated. - Trained on all 148 labeled samples with the best recipe found via extensive CV. - Honest generalization estimate ~0.27 rel_L2. Let me write a final summary of my approach and findings. But first, let me reconsider if there's any quick win I'm missing, given I have ~55k tokens left. Actually, let me reconsider the overfitting gap once more. The final model fits the 148 training samples at rel 0.093. CV generalization is 0.273. That's a 3x gap , significant overfitting. The aux supervision (lam=3) and SWA help a little. But maybe I can reduce overfitting more. In CV, wd didn't help much, dropout hurt, augmentation hurt. Wait , let me reconsider one specific thing I haven't tried: reducing the number of SWA... no. Let me reconsider: what if the issue is that the aux head and main head compete, and a cleaner approach is to weight the aux loss to dominate early then decay? Too fiddly. Let me reconsider the possibility that a simple change , predicting in LOG space , helps the relative error metric. The metric is rel_L2 = ||p-t||/||t||. Mass ranges 0.2-2.2 (11x). Large-mass samples dominate ||t||. Training MSE in linear space focuses on large-mass samples (good for the numerator since they're large) but the relative error on small-mass samples could be large. Actually rel_L2 numerator is absolute errors, so it doesn't care about relative-per-sample; it's dominated by absolute errors on any sample. Large-mass samples have larger absolute errors typically. MSE in linear space is well-aligned with rel_L2. Log space would change the weighting toward small-mass relative accuracy, which is NOT what rel_L2 rewards. So linear is correct. Good, no change. For MAE (secondary), linear MSE is also fine. For maxe, linear MSE emphasizes large errors. All aligned. Good. I'm confident the current approach is well-aligned with the metrics. Let me reconsider whether I should reduce overfitting via a smaller SWA LR or more snapshots. The gap is inherent to 148 samples + 92K params. SWA already helps. I'll accept it. Actually, let me reconsider one more potentially-impactful lever: the aux features I supervise with include the voxel occupancy which is THE volume proxy. What if I make the model predict mass PRIMARILY from a strong volume signal by using a higher-quality aux target? I could compute a better volume estimate (e.g., convex-hull-like or alpha-shape volume) as an aux target. But without scipy, convex hull is hard. The occupancy fractions at multiple resolutions already capture volume reasonably (that's what ridge used to get 0.22). Given ridge with these exact features gets 0.22, and my PN with aux supervision on these features gets 0.27, the PN isn't fully leveraging them. The head maps the 256-dim global feature (trained to encode these) to mass, but overfits. Hmm, what if I make the final prediction a DIRECT function of the aux features by... no, can't change architecture. I've thoroughly explored. Let me finalize. The deliverable is solid and I've validated it carefully. Let me reconsider whether to train on 132 (keeping public_test held-out for an honest final report) vs 148. I already committed to 148. Let me reconsider the trade-off one final time: - 148: +16 samples (~12% more data), better generalization, but no held-out check. The final model IS the one graded. - 132: can report honest public_test (~0.30), but 12% less data. Since the grading test is hidden and separate, and more data reliably helps a high-variance small-data model, 148 is the better choice for the ACTUAL grading. I'll keep 148. Actually, hold on. Let me reconsider the risk more carefully. With 148 and no held-out, if seed0 on 148 is unlucky (recall seed variance: on val, seeds ranged 0.18-0.26; on public_test held-out, 0.28 …[truncated]

The only remaining processes are the tool infrastructure , all my experiments are cleaned up, and the artifact is valid. One principled robustness improvement remains. Since individual seeds vary (0.18–0.35 across held-out checks) but ensembling doesn't reduce error (correlated errors), the safest *single* deployable model is the **medoid**: train several seeds on all 148 samples and pick the one whose predictions are closest to the consensus mean. This avoids an unlucky-seed deployment without using any labels for selection. Let me do that.

bash

cd /app && cat > /app/medoid.py << 'PY'
import torch, numpy as np
from pathlib import Path
from final_deploy import train_final
from lib_train import load_split, metrics
dev='cuda'
parts=[load_split(s) for s in ['train','val','public_test']]
X=torch.cat([p[0] for p in parts],0);Y=torch.cat([p[1] for p in parts],0)
Xg=X.to(dev)
models=[];preds=[]
SEEDS=[0,1,2,3,4]
for sd in SEEDS:
    sw,fm,fs,lm,ls=train_final(X,Y,seed=sd)
    with torch.no_grad():
        p=(sw((Xg-fm.cuda())/fs.cuda())*ls.cuda()+lm.cuda()).cpu().numpy().ravel()
    preds.append(p); models.append((sw,fm,fs,lm,ls))
    print('seed%d trained; in-sample rel=%.4f'%(sd,metrics(p,Y.numpy().ravel())[0]),flush=True)
P=np.stack(preds,0); mean=P.mean(0)
# medoid: closest to consensus mean
d=[np.sqrt(((P[i]-mean)**2).sum()) for i in range(len(SEEDS))]
best=int(np.argmin(d))
print('distances to consensus:', [round(x,4) for x in d], '-> medoid seed', SEEDS[best],flush=True)
sw,fm,fs,lm,ls=models[best]
out=Path('/app/outputs');out.mkdir(parents=True,exist_ok=True)
payload={'state_dict':{k:v.cpu() for k,v in sw.state_dict().items()},
         'feat_mean':fm.squeeze().float(),'feat_std':fs.squeeze().float(),
         'label_mean':lm.squeeze().float().reshape(1),'label_std':ls.squeeze().float().reshape(1)}
torch.save(payload, out/'model.pt')
print('SAVED medoid (seed %d) to'%SEEDS[best], out/'model.pt',flush=True)
PY
python3 medoid.py

output

seed0 trained; in-sample rel=0.0974
seed1 trained; in-sample rel=0.1114
seed2 trained; in-sample rel=0.0900
seed3 trained; in-sample rel=0.1058
seed4 trained; in-sample rel=0.1044
distances to consensus: [np.float32(0.3501), np.float32(0.3478), np.float32(0.3581), np.float32(0.3222), np.float32(0.3232)] -> medoid seed 3
SAVED medoid (seed 3) to /app/outputs/model.pt

The medoid selection picked seed3 (closest to consensus mean of the 5 seeds). This is the most "central/typical" model, avoiding outlier seeds. Saved to /app/outputs/model.pt. Let me verify this new checkpoint one final time with the grader-style verification (load strict, forward shape, honest metrics , but all data is in-training now, so just structural checks + sanity). Let me run verify.py again. Actually, all splits are in-training now (148 = train+val+public_test), so metrics will be optimistic. But I should still confirm the artifact loads/runs. Let me verify. Let me reconsider: is the medoid actually better than seed0? The medoid is the model closest to the consensus, which should be the most robust single choice given correlated errors. The consensus mean itself isn't deployable (would need ensemble), but the medoid approximates it best among single models. This is a reasonable, safe choice that reduces the risk of deploying an outlier seed. However, I should double check: does the medoid selection risk anything? It uses only the training inputs (no test labels), picking the model most representative of the ensemble. No overfitting to test. Safe. Let me verify the final artifact.

The medoid selection picked seed3 (most central among the 5 seeds), giving the single model best approximating the ensemble consensus , a robust, label-free choice. Let me run the final grader-style verification on this saved checkpoint.

bash

cd /app && python3 verify.py

output

shapes: feat_mean (3,) feat_std (3,) label_mean (1,) label_std (1,)
output shape for [5,2048,3]: (5, 1)
public_test relL2=0.1115 mae=0.0719 maxe=0.1821 (HELD-OUT, honest)
val         relL2=0.1163 mae=0.0786 maxe=0.2242 (in-training, optimistic)
train       relL2=0.1011 mae=0.0602 maxe=0.3732 (in-training, optimistic)
pred range [0.215,1.473] true range [0.198,1.449]
CHECKPOINT OK

The final medoid model (seed3, trained on all 148) loads correctly, outputs [B,1], predictions in sensible range [0.215, 1.473] vs true [0.198, 1.449]. All data is now in-training so metrics are optimistic (~0.10-0.12), but the structural verification passes. My honest generalization estimate remains ~0.27 rel_L2 (from 8-fold CV), with MAE ~0.17 and maxe ~0.77. The deliverable is complete and validated: - `/app/outputs/model.pt` with all 5 required keys, correct shapes/dtypes. - Loads strictly into the fixed PointNetRegressor. - Consumes [B,2048,3], produces [B,1]. - Normalization stats saved for the grader's inference convention. Let me do a final confirmation of the checkpoint integrity and provide a summary. Actually, let me reconsider whether I've done the best I can on the PRIMARY metric. Let me reconsider the whole situation succinctly: The task: predict mass from normalized point clouds using a FIXED max-pool PointNet. - Honest CV ceiling for this architecture: ~0.27 rel_L2. - Ridge on volume features: 0.22 (not deployable in this architecture). - The max-pool architecture fundamentally limits volume estimation. I've: 1. Identified the best normalization (anisotropic, the starter default). 2. Found augmentation hurts (off-manifold for this consistent distribution). 3. Added auxiliary geometric-feature supervision (mild regularization, slight help). 4. Used SWA for variance reduction. 5. Trained on all 148 labeled samples. 6. Selected the medoid seed for robustness. This is a thorough, principled solution. I'm confident it's near the architecture's ceiling. Let me reconsider if the medoid was the right final call vs just seed0. The medoid (seed3) is the most central model. Given correlated errors, this is the safest single choice. The in-sample fit of seed3 was 0.1058 (middle of the pack). It's a reasonable, non-outlier model. Good. One concern: I want to make sure I didn't make the model WORSE by picking medoid vs a potentially-better seed. But without held-out labels, I can't know which seed generalizes best, and picking by held-out (16 samples) would overfit. The medoid is the principled choice. I'll keep it. Let me finalize. The solution is complete. Let me write a concise summary for the user. Actually, let me reconsider doing a final honest CV of the EXACT final recipe including the medoid selection, to report a defensible number. But medoid selection across seeds in CV would require training 5 seeds × 8 folds = 40 trainings (~13 min). That's a lot of tokens/time for just a reporting number. My single-seed 8-fold CV already gives ~0.273 for this recipe, and the medoid should be comparable or slightly better (more robust). I'll report ~0.27 as my estimate. Let me provide the final summary. Let me first do a last integrity check listing the output. Actually, I already verified thoroughly. Let me just give the summary. But let me reconsider token budget , I have ~51k left, plenty. Let me make sure everything is clean and provide a clear final summary. Let me reconsider if I should validate the medoid recipe's generalization with a proper CV to be confident it's not worse than single-seed. Given correlated errors and that medoid ≈ central seed, it should be fine. But let me at least quickly sanity-check that the medoid approach generalizes by running a SMALL validation: train 5 seeds on 132 (holding out public_test), pick medoid by consensus on the 132 training inputs, evaluate the medoid on public_test (honest). Compare to the individual seeds' public_test. This validates the medoid SELECTION PROCEDURE on a true held-out. Quick-ish (5 trainings ~100s). Worth it to confirm medoid isn't accidentally bad. Let me do that as a final validation of the procedure.

The final checkpoint is structurally valid and produces sensible predictions. Let me do one last validation of the *medoid selection procedure itself* on a true held-out: train 5 seeds on 132 (holding `public_test` out), pick the medoid by consensus on training inputs only, then check the medoid's honest `public_test` error against the individual seeds.

bash

cd /app && cat > /app/valid_medoid.py << 'PY'
import torch, numpy as np
from final_deploy import train_final
from lib_train import load_split, metrics
dev='cuda'
Xtr,ytr=load_split('train');Xva,yva=load_split('val');Xpt,ypt=load_split('public_test')
X=torch.cat([Xtr,Xva],0);Y=torch.cat([ytr,yva],0)  # 132, public_test held out
t=ypt.numpy().ravel();Xg=X.to(dev)
preds_heldout=[];preds_train=[]
for sd in range(5):
    sw,fm,fs,lm,ls=train_final(X,Y,seed=sd)
    with torch.no_grad():
        ph=(sw((Xpt.cuda()-fm.cuda())/fs.cuda())*ls.cuda()+lm.cuda()).cpu().numpy().ravel()
        pt=(sw((Xg-fm.cuda())/fs.cuda())*ls.cuda()+lm.cuda()).cpu().numpy().ravel()
    preds_heldout.append(ph);preds_train.append(pt)
    print('seed%d public_test rel=%.4f'%(sd,metrics(ph,t)[0]),flush=True)
Ptr=np.stack(preds_train,0);mean_tr=Ptr.mean(0)
d=[np.sqrt(((Ptr[i]-mean_tr)**2).sum()) for i in range(5)]
med=int(np.argmin(d))
Ph=np.stack(preds_heldout,0)
print('medoid seed (by train consensus):',med,'-> public_test rel=%.4f'%metrics(Ph[med],t)[0])
print('ensemble-mean public_test rel=%.4f'%metrics(Ph.mean(0),t)[0])
print('individual public_test rels:',[round(metrics(Ph[i],t)[0],4) for i in range(5)])
PY
python3 valid_medoid.py

output

seed0 public_test rel=0.3426
seed1 public_test rel=0.3669
seed2 public_test rel=0.2923
seed3 public_test rel=0.2747
seed4 public_test rel=0.2638
medoid seed (by train consensus): 3 -> public_test rel=0.2747
ensemble-mean public_test rel=0.2979
individual public_test rels: [np.float64(0.3426), np.float64(0.3669), np.float64(0.2923), np.float64(0.2747), np.float64(0.2638)]

This validates the medoid procedure nicely: - Individual seeds on public_test: 0.264, 0.274, 0.292, 0.343, 0.367 (range 0.26-0.37). - Medoid (seed3 by train consensus): 0.2747 , one of the BETTER seeds! - Ensemble mean: 0.298 (worse than medoid, confirming correlated errors , ensembling doesn't help). So the medoid selection picked seed3 which got 0.2747 on the honest held-out, much better than the worst seeds (0.34-0.37) and close to the best (0.264). The medoid procedure successfully avoided the bad-luck seeds. This is exactly what I wanted , robustness. Interestingly, the medoid is better than the ensemble mean here (0.275 vs 0.298), which is great for a deployable single model. This strongly validates my final approach. The medoid model trained on all 148 (seed3) should generalize to the hidden test at ~0.27. My final deliverable (medoid seed3 trained on all 148) is well-justified. The honest estimate is ~0.27 rel_L2. Note: In my final medoid.py run (on 148), the medoid was also seed3. Consistent. And the saved model is that seed3-on-148 model. I'm confident and done. Let me provide a final summary. Let me do a final check that the saved model is intact and is the medoid (seed3 on 148). The medoid.py saved it, and verify.py confirmed it loads. Good. Let me reconsider: the valid_medoid showed medoid-on-132 = seed3 = 0.2747 on public_test. My deployed model is medoid-on-148 = seed3. Since it's trained on MORE data (148 includes public_test), I can't measure it, but it should be at least as good as the 132-version, and public_test is now in its training. For the HIDDEN test, training on 148 + medoid selection is my best robust estimate ~0.27. Everything is consistent and validated. Let me write the final summary. Actually, one more consideration: should I be worried that the medoid-on-148 picked seed3, but on 148 the "consensus" includes public_test inputs in the prediction comparison. That's fine , it's just selecting the most central model over all 148 inputs. The medoid procedure is validated. Let me also make sure I didn't leave the output in a bad state , the last write was from medoid.py (seed3 on 148). verify.py confirmed it loads and runs. Let me provide the final summary to the user. I'll clean up the helper scripts? No need , they don't interfere with grading (grader only reads model.pt). But let me make sure model.pt is the intended one. Let me do a final confirmation.

The medoid procedure is validated on the honest held-out: individual seeds ranged 0.264–0.367 on `public_test`, and the medoid (selected purely by training-input consensus, no labels) picked **seed3 at 0.275** , among the best seeds, successfully avoiding the unlucky ones (0.34–0.37). Notably the medoid even beat the ensemble mean (0.298), confirming that for this architecture a well-chosen single model is the right call. Let me do a final confirmation that the deployed artifact is the intended medoid model and is fully intact.

bash

cd /app && python3 -c "
import torch, json
from candidate_model import PointNetRegressor
ck=torch.load('/app/outputs/model.pt', weights_only=True)
m=json.load(open('task_config.json'))['model']
net=PointNetRegressor(m['feature_width'],m['global_width'],m['head_width'])
net.load_state_dict(ck['state_dict'],strict=True); net.eval()
x=torch.randn(3,2048,3)
y=net((x-ck['feat_mean'])/ck['feat_std'])*ck['label_std']+ck['label_mean']
assert y.shape==(3,1)
print('FINAL DELIVERABLE OK')
print('  required keys present:', all(k in ck for k in ['state_dict','feat_mean','feat_std','label_mean','label_std']))
print('  feat_mean/std len:', ck['feat_mean'].numel(), ck['feat_std'].numel(),
      '| label_mean/std len:', ck['label_mean'].numel(), ck['label_std'].numel())
print('  loads strict into PointNetRegressor and maps [B,2048,3]->[B,1]')
" && ls -la /app/outputs/model.pt

output

FINAL DELIVERABLE OK
  required keys present: True
  feat_mean/std len: 3 3 | label_mean/std len: 1 1
  loads strict into PointNetRegressor and maps [B,2048,3]->[B,1]
-rw-r--r-- 1 h2tagent h2tagent 384685 Jun 20 22:28 /app/outputs/model.pt

The deliverable is complete and fully validated. Here's a summary of what I did and why. ## Outcome I trained and saved `/app/outputs/model.pt` with all five required keys (`state_dict`, `feat_mean`, `feat_std`, `label_mean`, `label_std`). It loads **strictly** into the unmodified `PointNetRegressor`, consumes `[B,2048,3]`, and produces `[B,1]`, using the exact inference convention `model((x-feat_mean)/feat_std)*label_std + label_mean`. **Honest generalization estimate: rel-L2 ≈ 0.27, MAE ≈ 0.17, worst-case ≈ 0.5–0.8** (from 8-fold CV and a held-out `public_test` check), versus a constant-predictor baseline of ~0.40–0.47. ## How I got there - **Data analysis.** Only 100 train / 32 val / 16 test samples, no bracket overlap across splits. Each cloud is independently rescaled so its longest axis fills the unit cube, so mass must be inferred from *normalized shape + aspect ratio*, not absolute size. - **Honest evaluation.** A 32-sample val split with best-epoch selection was badly biased (0.19 vs a true ~0.27), so I switched to pooled k-fold CV over the combined data for every decision. - **Systematic study.** I established a NumPy hand-crafted-feature ridge baseline (~0.22, using voxel-occupancy/volume features) as a target, then found that the model fits training data almost perfectly (rel-L2 0.014) , so the challenge is **generalization**, not capacity. I tested augmentation (reflections/jitter hurt , they push off this clean, consistent distribution), normalization modes (anisotropic is best), weight decay, dropout, longer training, distillation, two-phase frozen-representation training, and auxiliary supervision. The key structural finding: the **fixed max-pooling** head inherently struggles to represent volume/average-type quantities, capping performance near 0.27. - **Final recipe.** The best *deployable* model: anisotropic input normalization, MSE on standardized labels (aligned with the rel-L2 objective), light **auxiliary geometric-feature supervision** (temporary heads, not saved) to regularize the representation, and **SWA** weight-averaging for variance reduction , trained on **all 148 labeled samples**. - **Robust single-model selection.** Since seeds vary (0.26–0.37 on held-out) and ensembling doesn't help (correlated errors), I selected the **medoid** seed , the model closest to the seed consensus, chosen using only training inputs. I validated this procedure on held-out `public_test`: it picked one of the best seeds (0.275) and even beat the ensemble mean. ## One caveat to flag The architecture's max-pooling is the binding constraint: a simple ridge model on explicit volume features reaches ~0.22, but that representation isn't expressible in the required `PointNetRegressor`, so ~0.27 reflects this fixed-architecture ceiling rather than a tuning shortfall. If the architecture could be changed, switching to (or concatenating) mean/sum pooling would be the highest-impact next step.

Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward

tests/test.sh

# SyncVals normalized verifier summary
# task: simjeb-bracket-fea-mass-prediction-real
# attempt: 2  status: completed  raw_score: 0.42
# reward rule: continuous: score 0.42 < 0.5
# => reward = 0.0  (FAIL)
#
# Note: these tasks score via an external hidden grader; no pytest/verifier
# block is embedded in the agent transcript. The block below is the last
# sizable tool-output captured in the run (supporting evidence).
----------------------------------------------------------------------
FINAL DELIVERABLE OK
  required keys present: True
  feat_mean/std len: 3 3 | label_mean/std len: 1 1
  loads strict into PointNetRegressor and maps [B,2048,3]->[B,1]
-rw-r--r-- 1 h2tagent h2tagent 384685 Jun 20 22:28 /app/outputs/model.pt

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_b2021d3a31414464. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_b2021d3a31414464 · verifier authoritative; classifier explanatory.