Finish the image-tuning work for the open-source T31 ISP driver so that live output is acceptably close to the OEM driver in color, demosaic quality, denoise behavior, and mode switching, while preserving compatibility with libimp.so.
The project is no longer blocked on basic stream bring-up. Video streams, but image quality is still not production-ready. The current failure mode is severe false-color / green-magenta blobbing, which indicates the raw-to-color pipeline is still not behaviorally equivalent to the OEM driver.
This PRD turns the current reverse-engineering findings into a finish plan rather than continuing ad-hoc debugging.
- Achieve stable, repeatable, visually correct color output in normal streaming
- Match OEM behavior closely enough that
libimp.sotuning and runtime controls behave as expected - Replace major synthetic/default tuning content with OEM-derived or OEM-shaped data
- Establish a repeatable validation workflow for day, night, and WDR modes
- Perfect mathematical equivalence for every internal register before visible parity is reached
- New user-facing features unrelated to OEM parity
- Retuning the pipeline from scratch without using OEM evidence
- Driver/module loads
- Core MMIO mapping and large ISP window mapping are in place
- Dual IRQ model (
isp-m0/isp-w02) is understood and implemented - Subdevice graph and stream control are operational enough for live video
- Tuning node exists and major blocks initialize
- Current live image shows strong false-color blob artifacts
- Some ISP blocks still depend on synthetic/default parameter banks
- Several modules are structurally present but not fully OEM-calibrated
f37c28868810f760002b00e0a00707569b64bbdbis the last known “crisp image” checkpoint- The next relevant regression point was expansion of the enabled processing-block whitelist
- Under the current masked logic, the effective crisp-image block set is
0xDD04
That means the project now has a concrete historical anchor for “good enough to compare against,” even if full parity was not yet achieved.
Resolved or understood areas include:
- ISP core mapping must cover the large OEM register space (
0x90000mapping) - IRQ ownership must not be double-registered or double-disabled
- platform/subdevice bring-up order matters
The following were identified as real color-path concerns:
- live CFA pattern programming at register
0x8 - DMSC output option ownership around register
0x4800 - DMSC gain refresh ordering during total-gain updates
The gamma path needed OEM-style register programming:
0x40000(R)0x48000(G)0x50000(B)
This is now understood and implemented, but gamma alone did not resolve the blobs.
The OEM tuning manifest shows that several subsystems still rely on synthetic, zeroed, or placeholder data. This means some remaining image-quality failures are likely caused by missing OEM-calibrated tables, not just bad control flow.
Based on OEM_TUNING_BLOB_MANIFEST.md, the biggest remaining data-quality gaps are:
AE tables— RESOLVED: ae0_ev_list and _lum_list now loaded from tuning binary; ae0_tune2 reimplemented with OEM-exact EV-domain convergence, EV FIFO, and histogram-based brightness feedback (data_9a2ec). Seedocs/AE_CONVERGENCE_ARCHITECTURE.md.- ADR/WDR tone-mapping tables
- BCSH / CCM / WB tables
- MDNS / RDNS / SDNS banks
Important interpretation: even if register sequencing is correct, visibly wrong output can persist if the active tables are still synthetic.
The AE convergence controller has been reimplemented to match the OEM ae0_tune2 (0x500b8). Key findings and status:
- Domain mismatch: _lum_list values are EV-domain targets (100-140K), not 0-255 brightness. The OEM compares target/FIFO_average (both EV domain), not target/wmean.
- Brightness feedback:
data_9a2ec=1enables histogram-based scaling of ev_list/lum_list tables by(256 - wmean/2) / 256. This is the mechanism that connects scene brightness to EV-domain convergence. - EV FIFO: 15-entry temporal smoothing using previous-frame tisp_ae_target outputs (stable lum_list values), not volatile ITAGDG products.
- Cold-start ramp: Emergency brightness-based ratio (target=128/wmean, capped 2x) active when wmean < 10 to bootstrap from black image.
- Zone data: Separate R/G/B per-zone arrays extracted from 4-word AE DMA entries.
- ae0_weight_mean2: Reimplemented with OEM algorithm (separate R/G/B, 64-bit weighted accumulation, channel mixing).
- Full OEM gain distribution (Phase H): threshold-based mode using data_c46a8 reference
- tisp_ae_tune function: scene-adaptive EV table adjustment (arg28 callback in OEM)
- Anti-flicker correction factor in ae0_weight_mean2
- OEM arg4/arg5 dark/bright pixel correction in ae0_weight_mean2
- Complete OEM Tiziano_ae0_fpga histogram processing (mode 1 highlight weighting)
The most likely live-path suspects remain:
- GIB (highest-confidence single block suspect)
- LSC
- YDNS
Reason: after restoring the effective crisp whitelist, the strongest residual delta from the OEM-style broad set is in these blocks, while ADR and MDNS are still guarded or parked in practice.
The project has already fixed several obvious sequencing and register-path mismatches. The next large quality gains are likely to require replacing placeholder content with OEM-backed parameter data, especially for AE, CCM/BCSH, gamma-adjacent tables, and denoise banks.
- Normal streaming must produce stable color without large green/magenta blobs
- Day/night switching must not corrupt color or wedge the pipeline
- WDR enable/disable must execute without hangs and with sane output
- Core
libimp.socontrols for exposure, white balance, and tuning must remain functional
- Output must be visually comparable to OEM in daylight and indoor lighting
- No frame-stable color blobs in neutral scenes
- No persistent CFA-phase corruption after sensor flips or mode changes
- No catastrophic regressions when enabling candidate ISP blocks
- Every tuning fix must be tied to either OEM HLIL evidence or a documented empirical comparison
- New data/table imports must have a provenance note (blob range, OEM array, or inferred mapping)
- Validation must produce reproducible captures/logs for before/after comparison
- record the exact “crisp-ish” and “broken” checkpoints
- capture the effective top-bypass set, mode, sensor mbus code, and flip state
- save reference screenshots for daylight, indoor warm light, and low light
- standardize capture scenes and camera placement
- standardize logs to collect around
tisp_init,tiziano_dmsc_init,tiziano_gamma_init, and top-bypass lines - record per-run block-enable state and current mode (day/night, WDR, shvflip)
Focus on blocks that can directly create visible early-pipeline corruption:
- GIB
- LSC
- YDNS
- residual DMSC/CFA edge cases
For each block:
- confirm OEM init sequence
- confirm enable/bypass semantics
- confirm register windows and trigger/commit writes
- test block enabled vs bypassed with fixed scene captures
Recommended reconstruction order:
- AE tables
- Gamma / degamma adjacent tuning content
- BCSH / CCM / WB tables
- ADR/WDR tone-mapping tables
- RDNS / MDNS / SDNS banks
This order follows both current manifest confidence and likely image-quality payoff.
- day mode
- night mode
- WDR on/off
- sensor flip / mirror combinations
- stream stop/start and second-stream stability
- labeled screenshots/video frames
- matching logs
- commit hashes / module parameters used
Per major block, document:
- OEM function(s)
- open-source function(s)
- register ranges
- current status: OEM-parity / partial / synthetic / parked
For each imported or reconstructed table, document:
- blob source range
- destination arrays/functions
- confidence level
- visual impact observed
- comparison against OEM captures
- unresolved differences
- recommended defaults for production use
- False positives from changing multiple blocks at once
- Good control flow still producing bad images because of placeholder tables
- WDR / MDNS / ADR changes causing hangs even when they look architecturally correct
- Overfitting to a single sensor/mode/scene
- OEM Binary Ninja access for function-level parity
- consistent hardware test setup
- known-good reference captures from OEM driver where possible
- stable way to extract or map OEM tuning blob content into open-source arrays
- Keep the new documentation current
- Build the baseline capture pack
- Do a controlled block matrix around GIB / LSC / YDNS from the
0xDD04baseline - Start importing real AE tables from the identified OEM blob ranges
- Move next to BCSH/CCM/WB and denoise-bank content
This effort is complete when:
- normal streaming output is visually sane across representative scenes
- no persistent false-color blob artifacts remain
- day/night and WDR transitions work without hangs or gross corruption
- the main remaining differences vs OEM are minor, documented, and acceptable
Completed (Phase 3 data extraction):
-
Gamma LUT — Replaced 256-entry identity linear ramp with real OEM 129-entry gamma curves extracted from tx-isp-t31.ko tuning blob. Old data was
{0x000, 0x008, 0x010, ...}(no gamma correction). OEM curve has proper shadow lift. However: tuning bin IS loaded at runtime and overrides these defaults, so the change is a fallback improvement. -
AE tables — Replaced ~30 all-zero AE parameter structs with OEM data: center-weighted 15x15 zone map, exposure thresholds, EV lists, log2/weight LUTs, flicker detection, scene parameters, WDR and AE1 variants. Same caveat: overridden by tuning bin at runtime.
-
BCSH/CCM — Replaced zero-initialized BCSH arrays with OEM data: CCM matrices (d/t/a), hue-dependent parameters, saturation lists, clip parameters, and MMatrix/MinvMatrix color space transforms.
-
SDNS — OEM data extracted but array sizes mismatch (OEM uses 4/9-element arrays, our driver has 16). Structural fix deferred.
Key finding: The tuning bin IS loaded by libimp at runtime and overrides all initial values. The gamma, AE, and BCSH data we embedded serve as proper fallback defaults when the tuning bin is unavailable.
Tested (Phase 2 block enable):
- GIB + LSC enabled together (0xDD34): SEVERE REGRESSION — large color blobs spanning entire frame. Immediately reverted to 0xDD04.
Root cause analysis of GIB+LSC regression:
- LSC writes registers 0x28000-0x4A8A0 (651 iterations x 3 channels x 16-byte stride), a huge range
- Runtime tuning data shows
mesh_size=48x36,mesh_scale=0,lut_count=1953— these are real GC2053 data from libimp, NOT from .ko defaults - With
mesh_scale=0,base_strength=0x800andlsc_curr_str=0at init, all LSC values collapse to 0x800 (flat) — but even flat writes to that register range may conflict with other ISP state - GIB alone has NOT been tested separately — registers are limited to 0x1030-0x1070 range, much smaller footprint
Next steps:
- Test GIB alone (isp_block_enable=0xDD24) to isolate which block causes regression
- Investigate LSC mesh_scale=0 behavior — may need special handling
- Investigate whether the 0x28000+ register range overlaps with active ISP blocks
- Consider whether LSC needs lsc_curr_str > 0 before enabling (currently 0 = no correction)