when_all waits for all tasks before surfacing the first failure#1166
when_all waits for all tasks before surfacing the first failure#1166acroca wants to merge 1 commit into
Conversation
Previously a WhenAllTask completed as soon as one child failed, while sibling tasks were still running and their later completions were dropped. Now the composite stages the first failure and keeps waiting; it completes only once every child has finished, then fails with the first recorded exception (or returns the ordered results if none failed). Signed-off-by: Albert Callarisa <albert@diagrid.io>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1166 +/- ##
==========================================
+ Coverage 82.78% 82.83% +0.05%
==========================================
Files 123 123
Lines 10127 10128 +1
==========================================
+ Hits 8384 8390 +6
+ Misses 1743 1738 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR changes the durabletask when_all composite task semantics so that it waits for all child tasks to finish before completing, and only then surfaces the first observed failure (if any). This aligns task completion accounting with real runtime state and ensures orchestrators that catch failures do so only after all fan-out work has concluded.
Changes:
- Update
WhenAllTaskto stage the first child failure internally and defer failing the composite until all children complete. - Expand/adjust unit tests and executor-level tests to assert the new “defer failure” behavior and first-failure selection.
- Update public-facing docstrings for
when_allto reflect the behavior change.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
dapr/ext/workflow/_durabletask/task.py |
Implements deferred-failure behavior in WhenAllTask by staging the first exception until all children complete. |
dapr/ext/workflow/dapr_workflow_context.py |
Updates the when_all wrapper docstring to document deferred failure surfacing. |
tests/ext/workflow/durabletask/test_task.py |
Adds/updates task-level tests for staged failures, multiple failures, and pre-failed children. |
tests/ext/workflow/durabletask/test_orchestration_executor.py |
Adjusts orchestration executor tests to validate no early completion/actions and correct failure surfacing timing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description
when_allnow waits for every task to finish and then surfaces the first error, instead of completing on the first failure while sibling tasks were still running.This fixes several rough edges: completions arriving after a failure were silently dropped, so
get_completed_tasks()andpending_tasksreported stale counts from the moment a child failed; and an orchestrator that caught the failure (e.g. to run compensation) would do so while fanned-out activities were still executing. Now the counts stay accurate for the composite's whole lifetime, and by the time the error is raised into the orchestrator, all fanned-out work is known to be finished. When multiple children fail, the first failure is the one reported. The failure is staged internally until the last child completes, so the task never reportsis_failedwhile it is still incomplete.Note this is a behavior change: a fan-out that hits a failure now runs its remaining activities to completion before the workflow fails, instead of failing immediately.