Add reattach_on_existing option to TriggerDagRunOperator#69839
Open
crobertson-astro wants to merge 1 commit into
Open
Add reattach_on_existing option to TriggerDagRunOperator#69839crobertson-astro wants to merge 1 commit into
crobertson-astro wants to merge 1 commit into
Conversation
When a triggered Dag run fails on a single task, users often fix it by hand in the triggered Dag (marking the task success, clearing it, or re-running it) and then want the triggering task to resume waiting on that same run. Today the only ways to re-run the triggering task are reset_dag_run, which wipes the manual fix by clearing the whole triggered run, or the default, which raises DagRunAlreadyExists. Neither lets the triggering task simply re-attach. reattach_on_existing lets a cleared triggering task attach to the existing triggered Dag run and, with wait_for_completion, poll or defer on it until it reaches an allowed or failed state, preserving any manual intervention. reset_dag_run and skip_when_already_exists keep precedence.
crobertson-astro
requested review from
amoghrajesh,
ashb and
kaxil
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a
reattach_on_existingoption toTriggerDagRunOperator.When a triggered Dag run fails on a single task, users often fix it by hand in the triggered Dag (mark the task success, clear it, or re-run it) and then want the triggering task to resume waiting on that same run. Today, re-running the triggering task either wipes the manual fix (
reset_dag_run=Trueclears the whole triggered run) or raisesDagRunAlreadyExists(the default). Neither lets the triggering task simply re-attach.With
reattach_on_existing=True, a cleared triggering task attaches to the existing triggered Dag run and — withwait_for_completion=True— polls or defers on it until it reaches an allowed or failed state, preserving the manual intervention.reset_dag_runandskip_when_already_existskeep precedence.Implemented on both the Airflow 2.x path (operator
DagRunAlreadyExistshandling) and the Airflow 3.x path (DagRunTriggerException+ task-SDK execution), with tests covering reattach, wait-success, wait-failed, deferrable, and reset/skip precedence on both paths.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines