Skip to content

fix(migrations): keep still-live dev intervals in v0071#5899

Open
anxkhn wants to merge 1 commit into
SQLMesh:mainfrom
anxkhn:patch-7
Open

fix(migrations): keep still-live dev intervals in v0071#5899
anxkhn wants to merge 1 commit into
SQLMesh:mainfrom
anxkhn:patch-7

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 14, 2026

Copy link
Copy Markdown
Contributor
The v0071 intervals migration drops every still-live dev interval on upgrade
because of a type mismatch in a set-membership check.

In `_migrate_intervals`, `used_dev_versions` is a `Set[Tuple[str, str]]` and its
only insertion adds a `(name, dev_version)` 2-tuple:

```python
used_dev_versions.add((name, dev_version))

But the guard that decides whether to drop a dev interval compares a bare
dev_version string against that set:

dev_version = snapshot_ids_to_dev_versions.get((name, identifier))
if dev_version not in used_dev_versions and is_dev:
    # If the interval's dev version is no longer used and this is a dev interval, we can safely delete it
    continue

A str is never an element of a set of 2-tuples, so dev_version not in used_dev_versions is always True and the condition collapses to
if is_dev: continue. Every dev interval that had already passed the preceding
used_versions check (i.e. is still live) is discarded, forcing a re-backfill of
dev environments after the upgrade.

This changes the check to compare the correctly-typed (name, dev_version) key,
matching the tuples stored in used_dev_versions, so only genuinely unused dev
intervals are removed.

Test Plan

  • Added test_migrate_v0071_keeps_still_live_dev_intervals in
    tests/core/state_sync/test_state_sync.py. It builds an in-memory DuckDB state
    at schema v70 (intervals table without the dev_version column) with a live
    prod snapshot/interval and a live dev snapshot/interval, runs the v0071
    migrate_schemas + migrate_rows, and asserts both intervals survive.
  • Confirmed the test fails without the fix (the dev interval is dropped:
    {'prod_int'} != {'dev_int', 'prod_int'}) and passes with it.
  • Also confirmed a genuinely-stale dev interval (its dev version no longer mapped
    to a live snapshot) is still correctly removed, so the fix is not over-broad.
  • make style (ruff + ruff-format + mypy) passes on the changed files; the
    tests/core/state_sync/test_state_sync.py migration/interval subset passes.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

The v0071 intervals migration compared a bare dev version string against
`used_dev_versions`, which only ever holds `(name, dev_version)` tuples. A
string is never an element of a set of 2-tuples, so
`dev_version not in used_dev_versions` was always true and the guard collapsed
to `if is_dev: continue`. Every dev interval whose version was still live (it
had already passed the `used_versions` check) was dropped on upgrade, forcing a
re-backfill of dev environments.

Compare the correctly-typed `(name, dev_version)` key so only genuinely unused
dev intervals are removed. Add a regression test that runs the migration against
an in-memory DuckDB state with a still-live dev interval and asserts it survives.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant