Skip to content

Keep triggerer alive when deferred tasks disappear#69842

Open
idan192 wants to merge 2 commits into
apache:mainfrom
idan192:idan/20260713/triggerer-tasknotfound-guard
Open

Keep triggerer alive when deferred tasks disappear#69842
idan192 wants to merge 2 commits into
apache:mainfrom
idan192:idan/20260713/triggerer-tasknotfound-guard

Conversation

@idan192

@idan192 idan192 commented Jul 14, 2026

Copy link
Copy Markdown

A stale deferred task can raise TaskNotFound while the triggerer loads its pinned serialized Dag, aborting workload construction for unrelated triggers in the same supervisor batch.

Catch only that task lookup failure, emit the trigger and task-instance identifiers as structured fields, and construct the ordinary context-free workload. Direct-to-triggerer tasks that still exist continue to receive Dag and DagRun context unchanged.

The durable context-requirement persistence change is tracked in #69841. This PR is deliberately small and backportable; please cherry-pick it to v3-3-test for the next 3.3.x release.

Validation

  • A missing task now produces a context-free workload and all structured identifiers are present.
  • A mixed stale/healthy batch constructs both workloads; the healthy direct-to-triggerer workload retains Dag and DagRun data.
  • Diff-scoped pre-commit and manual hooks pass, including mypy, ruff, security, and logging checks.
  • The full triggerer test module reached 42 passing tests locally before an existing macOS subprocess test exited with Bad file descriptor; the Linux Core/Other matrix remains delegated to CI.

related: #69841


Was generative AI tooling used to co-author this PR?
  • Yes — Codex (GPT-5)

Generated-by: Codex (GPT-5) following the guidelines

A stale task reference in a pinned serialized Dag can otherwise abort workload construction for every trigger in the supervisor batch and cause repeated triggerer restarts.
@boring-cyborg

boring-cyborg Bot commented Jul 14, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Release notes should make the restored triggerer behavior visible to operators evaluating a 3.3 patch release.
@idan192 idan192 marked this pull request as ready for review July 14, 2026 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant