Add sdata.pl.annotate() — interactive region selection via anywidget#684
Draft
timtreis wants to merge 6 commits into
Draft
Add sdata.pl.annotate() — interactive region selection via anywidget#684timtreis wants to merge 6 commits into
timtreis wants to merge 6 commits into
Conversation
Add plans/interactive-selection.md documenting the v0 design for sdata.pl.interactive(...): in-notebook selector widget that draws a region on a spatialdata-plot canvas and persists it back into the SpatialData object as a ShapesModel. Includes resolved Q1-Q4, coordinate- system rules, downsampling strategy, persistence policy, and a 12-task implementation queue. Add a pixi `interactive` dep-group (ipympl, ipywidgets, squidpy) and a new `dev-interactive-py313` environment for prototyping. Register a dedicated `sdata-plot-interactive` kernel-install task to avoid the existing `pixi-dev` kernel name collision. Rewrite the broken [tool.pixi] inline-dotted block to explicit table headers ([tool.pixi.workspace], etc.) so pixi 0.54.2 actually loads the manifest. This commit records the ipympl-based prototype iteration. The notebook prototype (Sandbox.ipynb in lustre, not tracked here) revealed that websocket-streamed PNG frames are too laggy over SSH for full-slide interactive drawing; the next iteration switches to Plotly's client-side draw tools while keeping the same spec and task queue. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add anywidget and plotly>=5.20,<6 to the pixi interactive dep-group so the prototype notebook can render a custom HTML5/SVG drawing widget. anywidget is the canonical path for traitlet-based widget sync in VSCode-Remote; plotly is pinned to 5.x because its 6.0 anywidget-backed FigureWidget does not relay client-side relayout events back to Python (so layout.shapes never syncs there). The Sandbox.ipynb prototype itself lives outside this repo (/home/.../lustre/projects/spatialdata-plot/), but its current state implements a working anywidget-based draw canvas: pure client-side SVG drawing (rectangle drag, polygon click-then-Close-polygon, lasso freehand drag), shapes pushed back via the `shapes` traitlet, pixel→CS coordinate mapping that respects matplotlib's origin='upper' image axis, multi-shape commit per Save, and an explicit "Write last to disk" button for persistence. Sandbox.anywidget-v0.ipynb is preserved alongside as a reference snapshot before optimization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Productionises the Sandbox.ipynb prototype as a user-facing method on
PlotAccessor. Public surface is a single function:
sdata.pl.annotate(coordinate_system, element, *, persist=True) -> None
Both args are required positional. The function validates that the image
element is registered in the given coordinate system, renders it to a PNG,
constructs an internal _InteractiveSession with anywidget-driven drawing
tools (rectangle / polygon / lasso), and displays the widget. Drawn shapes
are written into sdata.shapes[name] on click of the Save button; the
optional "Write to disk" button persists via sdata.write_element.
Module layout (src/spatialdata_plot/pl/interactive/):
- _canvas.py DrawCanvas anywidget class
- static/draw_canvas.js ESM module read from disk by anywidget (HMR-friendly)
- _render.py render_to_png: sdata.pl → PNG + ax extent
- _commit.py pixel-coord shape → CS-coord shapely Polygon → ShapesModel
- _persist.py commit_to_memory + persist_to_disk (collision policy)
- _session.py _InteractiveSession orchestrating the widget
The new optional extra `interactive` (anywidget, ipykernel, ipywidgets)
gates this feature behind a clear ImportError when missing:
pip install 'spatialdata-plot[interactive]'
The prototype iteration explored ipympl (rejected: PNG-over-websocket
latency unusable over SSH) and plotly's FigureWidget (rejected: client-
side relayout events don't sync back to Python in VSCode-Remote, plus
plotly 6's anywidget-backed FigureWidget broke the comm path entirely).
The custom anywidget approach was the only architecture that worked
reliably over SSH while staying responsive.
Drawing UX:
- Tools: rect (drag), polygon (click + snap-close), lasso (drag freehand)
- Wheel zoom, shift-drag pan, alt-click shape to delete
- Ctrl+Z undo, R/P/L tool shortcuts, F fit view, Enter close polygon
- Multi-shape bundling: each Save commits all canvas shapes as one
ShapesModel with multiple rows under a single name
Tests cover the unit surface (pixel→CS conversion, ShapesModel transform
registration, render-to-PNG correctness, commit/persist policy, widget
smoke).
Spec at plans/interactive-selection.md updated to document the
architectural pivot from the original ipympl approach.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Tests: import `DrawCanvas` from `._canvas` (internal class is not re-exported), and gate `test_canvas` with `pytest.importorskip` so CI envs without the `interactive` extra skip rather than fail. - `_persist.py`: replace deprecated `datetime.utcnow()` with timezone-aware `datetime.now(timezone.utc)`. Document on-disk overwrite behaviour (asymmetric with in-memory rename-on-collision). - `_render.py`: wrap render in `try/finally` so figures don't leak if `render_images().show()` or `savefig` raises. - `draw_canvas.js`: Delete/Backspace now removes the most recent shape (matches Ctrl+Z) instead of wiping the whole canvas — the toolbar Clear button covers the wipe case. - `basic.py` docstring: note that the canvas clears on every Save and that the Write-to-disk button overwrites same-named on-disk elements. - Add `tests/test_interactive/test_annotate.py` covering the three validation paths (`unknown CS`, `unknown element`, `element not in CS`) by stubbing `_InteractiveSession.show`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #684 +/- ##
==========================================
- Coverage 77.79% 74.79% -3.00%
==========================================
Files 11 17 +6
Lines 3693 3944 +251
Branches 877 901 +24
==========================================
+ Hits 2873 2950 +77
- Misses 490 662 +172
- Partials 330 332 +2
🚀 New features to boost your workflow:
|
Reuse / convention fixes: - Delete `_persist.persist_to_disk` — `SpatialData.write_element` already raises `ValueError` when `path is None`. Inline `write_element(name, overwrite=True)` in `_on_persist` and fix the docstring claim about overwrite semantics. - Match project import convention: `from spatialdata.transformations. operations import get_transformation` and `...transformations import Identity`, replacing `sd.transformations.*` references in `_session`, `_commit`, and tests. - `_validate` uses `sdata[element]` indexing + `get_transformation` to match the established pattern in `pl/utils.py` / `pl/render.py`. Quality: - Introduce frozen `RenderExtent` dataclass returned by `render_to_png`; collapses the 5-tuple return + 4 cached attrs on `_InteractiveSession` into one object. `pixel_shape_to_polygon(shape, extent)` drops 4 args. - `traitlets.Enum(TOOLS, ...)` for `DrawCanvas.tool` so a typo raises. - `BannerKind = Literal["info","success","error","hint"]`; drop the silent `.get(..., default)` fallback so a banner-kind typo raises. - Factor `_trigger_btn(description, icon, trait_name, after=...)` — replaces 4 near-identical `_on_close_polygon` / `_on_undo` / `_on_clear` / `_on_fit` methods. - Split `_on_save` into `_collect_polygons` / `_commit_polygons` / `_reset_canvas_state`; orchestrator stays ~10 lines. - Drop redundant `spine.set_visible(False)` loop after `set_axis_off()`. - Guard `persist_btn` construction behind `persist=True`; `_on_persist` early-returns if disabled. - Strip restate-the-code comments (`_canvas` module docstring, `_render` v0/v1 narration, `_commit` lasso-restating comment). - Underscore truly-private attrs (`_sdata`, `_commits`); keep `canvas` un-underscored since `_validate` callers in tests still need a way in. Efficiency (JS): - Incremental in-progress shape update during rect/lasso drag — keep a stable reference to the in-progress SVG node; mutate its attributes in `onMouseMove` instead of full `redraw()` (60 Hz × O(N) DOM ops → O(1)). - Lasso vert-push gated on viewbox-px ≥ 1 from the last vert; cuts vertex count ~5-10× for typical drags and the kernel-side traitlet payload on commit. - `setShapes` early-returns on `next === shapes` or both-empty; the `clear_trigger` handler routes through `setShapes([])` and skips when there is nothing to clear or cancel. - `zoomAt` / `panBy` / `fitView` snapshot the vbox pre-clamp and skip `applyViewbox` + `redraw` if the clamped vbox is unchanged. - `change:tool` only redraws if there was an in-progress shape to clear. - Dedup: `popLastShape` helper used by Ctrl+Z, Delete/Backspace, and `change:undo_trigger`. `shapeNode` extracts the common stroke/fill attrs and uses a single `pointsAttr` formatter for polygon/polyline. Tests: - Pytest `no_display` fixture replaces three duplicate `monkeypatch. setattr(...)` calls in `test_annotate.py`. - `pixel_shape_to_polygon` tests updated to the `RenderExtent` signature via a small `_extent(...)` helper. - `test_render` reads `extent.image_w` / `extent.xlim` from the dataclass instead of unpacking a 5-tuple. - Drop the two `test_persist` tests for the deleted `persist_to_disk` wrapper; `commit_to_memory` policy tests remain. All 18 interactive tests pass in dev-interactive-py313. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UX: - Responsive canvas via `width: 100%` + `max-width: Npx` + CSS `aspect-ratio` on the wrap/container divs. Removes the fixed `DISP_MAX = 760` pixel box. Below `max_width` the canvas scales with the surrounding column; above it the canvas caps and centers. - New `max_width: int = 880` kwarg on `sdata.pl.annotate(...)` plumbed through `_InteractiveSession.__init__` and a new `max_display_width` `Int` traitlet on `DrawCanvas`. Pure display hint; underlying PNG render is unchanged (840 × 840). - Toolbar reflow: replace `HBox` with `Box(layout=Layout(flex_flow= "row wrap", ...))` so the tool toggle + icon buttons wrap onto a second row under narrow notebook widths instead of overflowing. - Icon-only auxiliary buttons (Close polygon / Undo / Clear / Fit / Write to disk) — `description=""`, 36px square, tooltip carries the affordance. Save button keeps its text label. - Drop the standalone "0 shape(s) on canvas" row and the `description= "Tool:"` / `description="Name:"` widget-side labels — none of them added information, all cost vertical space. Behaviour: - Remove the UTC-timestamp collision rename in `commit_to_memory`. Same name now overwrites in-memory (and on-disk via `write_element`, which we already pass `overwrite=True`). Drops the `datetime` import and the rename-on-collision banner branch. Reviewer flagged the prior rename as off-convention vs upstream spatialdata. - Update `test_commit_to_memory_renames_on_collision` → `test_commit_to_memory_overwrites_on_collision`. Other tests unchanged. Lint: - Pre-commit pass: `_dt.timezone.utc` → `_dt.UTC` (UP017) before the whole datetime block was dropped; ruff PT018 split assertion into two lines in `_collect_polygons`; biome + ruff-format normalised the changed Python and JS. All 18 interactive tests pass in dev-interactive-py313. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
sdata.pl.annotate(coordinate_system, element, *, persist=True)— an in-notebook widget for drawing regions on a spatialdata-plot canvas and persisting them back into the SpatialData object as aShapesModel. Works over SSH to a remote/SLURM node.src/spatialdata_plot/pl/interactive/(canvas, render, commit, persist, session) backed by a custom anywidget with HTML5/SVG drawing tools (rectangle, polygon, lasso).pip install 'spatialdata-plot[interactive]'(anywidget, ipywidgets, ipykernel) — feature is gated behind a clearImportErrorwhen missing.plans/interactive-selection.md(v0 scope, Q1–Q4 locked).Why anywidget (and not ipympl / plotly)
Both were prototyped and rejected:
The custom anywidget was the only architecture that worked reliably over SSH while staying responsive — image renders once via the existing
render_images().show()pipeline, then all drawing happens client-side and only the final shape geometry round-trips to Python.Drawing UX
ShapesModel(multiple rows) under a single nameCoordinate-system safety
Session is bound to one CS; render is 1:1 in that CS; committed
ShapesModelregisters with{cs_name: Identity()}to avoid the classic double-applied-transform bug.