Skip to content

Persist trigger context requirements#69843

Draft
idan192 wants to merge 5 commits into
apache:mainfrom
idan192:idan/20260713/persist-trigger-context
Draft

Persist trigger context requirements#69843
idan192 wants to merge 5 commits into
apache:mainfrom
idan192:idan/20260713/persist-trigger-context

Conversation

@idan192

@idan192 idan192 commented Jul 14, 2026

Copy link
Copy Markdown

Persist whether a task trigger needs Dag context so ordinary deferrals can skip serialized-Dag lookup and deserialization.

  • Add a nullable trigger.start_from_trigger column without a server default or backfill, preserving rolling-upgrade NULL rows.
  • Write explicit requirements from scheduler, Execution API, callback, and asset paths.
  • Self-classify legacy rows once, remove deferred task instances whose pinned Dag no longer contains the task, and retry when the pinned Dag itself is unavailable.

This draft is stacked on #69842 and will be rebased onto main after that backportable fix lands.

Validation

Workload tests cover the no-lookup ordinary path, context-rich direct-to-triggerer workloads, both legacy classifications and their database persistence, missing-task removal without blocking a healthy workload, missing-Dag retry behavior, and unchanged asset workloads. Writer-path tests verify scheduler True and ordinary-trigger False persistence, including callbacks and assets. The SQLite 128-revision migration round trip and ORM/schema consistency checks pass locally; PostgreSQL and MySQL migration coverage is delegated to CI because Docker is unavailable in the local environment.

closes: #69841


Was generative AI tooling used to co-author this PR?
  • Yes: Codex (GPT-5)

Generated-by: Codex (GPT-5) following the guidelines

idan192 added 3 commits July 13, 2026 17:55
A stale task reference in a pinned serialized Dag can otherwise abort workload construction for every trigger in the supervisor batch and cause repeated triggerer restarts.
Release notes should make the restored triggerer behavior visible to operators evaluating a 3.3 patch release.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent stale task triggers from blocking healthy workloads

1 participant