Style transfer: honor trained crop_size (I2-A) + partition-of-unity tile blending (I2-B)

[sibocw]

This work was implemented by Claude Code (Opus 4.8) upon user instruction.

Addresses the style-transfer resolution/tiling findings from the audit in #48 (I2-A and I2-B).

I2-A — Honor the trained CUT resolution at inference (was: frozen at 256)

style_transfer/scripts/run_inference.py hard-coded image_side_length=256 (its CLI default) and a fixed resize_and_crop preprocess, so every frame was hard-resized to 256 and the generator emitted 256 — even for a model trained at a higher resolution. Unlike the tiled script, it ignored the crop_size/load_size/preprocess stored in the trained model's train_options.json.

Fix: a new resolve_trained_resolution() helper sources crop_size (→ image_side_length), load_size, and preprocess from the model's train_options.json via the existing parse_hyperparameters_from_checkpoint_path (mirroring run_tiled_inference.py and test_trained_models.py). It falls back to the requested image_side_length when train_options.json is absent, logs the resolution actually used, and warns if a user-supplied size conflicts with the trained crop_size. cut_inference.py already accepts a parsed preprocess_opt, so no change was needed there.

I2-B — Partition-of-unity tile blending (quality improvement; existing outputs were already acceptable)

style_transfer/tiled_inference.py blended overlapping tiles with weight_type="uniform" by default — an all-ones map (plain averaging, no feathering) that can leave hard seams where two independently InstanceNorm-normalized tiles meet. The non-default cosine (0.5-0.5cos) and pyramid (1-|2t-1|) windows are zero at the tile border (so on a 50%-overlap grid the seam pixel gets ~0 weight from one tile) and are not a partition of unity. randomize_seams=True by default moved seams every frame (temporal flicker).

Fixes (all safe and backward-compatible — every existing option stays selectable):

• (a) Added a "feather" window: a symmetric trapezoidal ramp sampled at pixel midpoints. Its 50%-overlap copies sum to exactly 1 (verified) and it is strictly nonzero at the seam, so two adjacent tiles always co-contribute there. It is now the recommended default for the tiled CLI (run_tiled_inference.py).
• (b) Mirror-padded out-of-frame tile regions are explicitly zero-weighted (_clamp_weights_to_frame) before accumulation, so hallucinated padding cannot leak into genuine edge pixels.
• (c) Default randomize_seams changed to False to avoid per-frame seam flicker.

Note: per #48, outputs were manually judged OK with the old uniform default, so I2-B is an improvement, not a critical bug fix.

Findings from #48 addressed

• I2-A — Resolution frozen before the pose model sees it: style-transfer (non-tiled) path resized every frame to 256 and ignored the trained crop_size/preprocess from train_options.json.
• I2-B — Tiled high-res inference can inject seam/blend artifacts: uniform default does no feathering; cosine/pyramid are zero at the seam and not a partition of unity; mirror-padded regions leak into edge pixels; randomize_seams=True causes flicker.

Refs #48

Tests

New tests/test_tiled_blend.py (pure-numpy; stubs the unavailable CUT GAN library so the real tiled_inference helpers are imported and exercised directly — the GAN path is not run):

• Feather is a partition of unity in 1D and 2D (overlap-add ≈ 1.0 across the interior, tol 1e-5) and nonzero at borders/seams; both overlapping tiles contribute meaningful weight at the seam.
• Contrast: cosine/pyramid collapse to ~0 at the border and cosine's overlap-add is visibly non-uniform.
• Out-of-frame / mirror-padded tile regions receive exactly zero weight (top-left, bottom-right, fully-off-frame, and a genuine edge tile from the real _compute_half_overlap_tile_starts grid).
• No regression in _normalize_weight_type (all five types, case/whitespace, guassian→gaussian, unknown raises) and all legacy windows still build valid maps.
• I2-A: parse_hyperparameters_from_checkpoint_path sources image_side_length from crop_size (and finetune_load_size when decaying), and raises when train_options.json is missing.

$ .venv/bin/python -m pytest tests/test_tiled_blend.py -q
25 passed in ~1.9s

Additionally validated end-to-end with an identity "GAN" on a constant frame: only feather yields an accumulation weight of exactly 1.0 everywhere (including edges) and reproduces the input exactly, whereas cosine produces a non-constant weight map.

Caveats

• The GAN inference path itself was not executed (requires the cut package + a CUDA model, unavailable here); I2-A is verified by reading the code and unit-testing the parsing helper it relies on.
• I2-B reduces seam artifacts but does not eliminate the intrinsic per-tile InstanceNorm statistics difference (each tile is normalized independently); the feather minimizes its visibility. Mirror-padded pixels still influence in-frame outputs through the generator's receptive field — only their direct weight contribution is removed.
• Defaults changed only on run_tiled_inference.py:run_tiled_inference_for_checkpoint. The separate cluster wrapper scripts_on_cluster/.../run_tiled_inference_all_simulations.py (outside this PR's scope) sets its own explicit weight_type/randomize_seams, so its behavior is unchanged; it can now also select feather.
• When load_size > crop_size, the inherited resize_and_crop preprocess uses a RandomCrop at inference (non-deterministic) — this is the pre-existing behavior of the parsed-options path (test_trained_models.py), not introduced here.

🤖 Generated with Claude Code