Depends on #66610 getting merged first.
Summary
PR #66610 added airflow-core/src/airflow/utils/log/callback_log_reader.py to read
deadline-callback execution logs for the UI. Its remote/local read helpers
re-implement logic that already exists in FileTaskHandler
(airflow-core/src/airflow/utils/log/file_task_handler.py) and is fronted by
TaskLogReader (airflow-core/src/airflow/utils/log/log_reader.py). This issue
tracks consolidating the two paths onto shared helpers as a follow-up.
This is tech debt, not a bug — the duplication is intentional in #66610 to keep
that PR focused.
Duplicated logic
callback_log_reader.py |
Existing equivalent in file_task_handler.py |
_read_callback_remote_logs |
FileTaskHandler._read_remote_logs (:967) |
_read_callback_local_logs (glob-by-prefix + os.path.commonpath containment) |
FileTaskHandler._read_from_local (:862, :880) |
remote-first-then-local fallback + _interleave_logs + "Log message source details" group header |
FileTaskHandler.read (:737) |
tuple[list[str], list[RawLogStream]] return annotation |
existing StreamingLogResponse alias |
Why a shared abstraction needs design (why it was deferred)
FileTaskHandler's read path is TaskInstance-centric — it takes (ti, try_number, metadata) and renders paths from the TI. Callback logs have no TaskInstance: the
path is a fixed prefix (executor_callbacks/{dag_id}/{run_id}/{callback_id} or
triggerer_callbacks/...). Extracting the common core means separating the
path-resolution step from the storage-read step so both callers can supply their own
relative path(s).
Follow-up work
Acceptance criteria
References
Depends on #66610 getting merged first.
Summary
PR #66610 added
airflow-core/src/airflow/utils/log/callback_log_reader.pyto readdeadline-callback execution logs for the UI. Its remote/local read helpers
re-implement logic that already exists in
FileTaskHandler(
airflow-core/src/airflow/utils/log/file_task_handler.py) and is fronted byTaskLogReader(airflow-core/src/airflow/utils/log/log_reader.py). This issuetracks consolidating the two paths onto shared helpers as a follow-up.
This is tech debt, not a bug — the duplication is intentional in #66610 to keep
that PR focused.
Duplicated logic
callback_log_reader.pyfile_task_handler.py_read_callback_remote_logsFileTaskHandler._read_remote_logs(:967)_read_callback_local_logs(glob-by-prefix +os.path.commonpathcontainment)FileTaskHandler._read_from_local(:862,:880)_interleave_logs+ "Log message source details" group headerFileTaskHandler.read(:737)tuple[list[str], list[RawLogStream]]return annotationStreamingLogResponsealiasWhy a shared abstraction needs design (why it was deferred)
FileTaskHandler's read path is TaskInstance-centric — it takes(ti, try_number, metadata)and renders paths from the TI. Callback logs have no TaskInstance: thepath is a fixed prefix (
executor_callbacks/{dag_id}/{run_id}/{callback_id}ortriggerer_callbacks/...). Extracting the common core means separating thepath-resolution step from the storage-read step so both callers can supply their own
relative path(s).
Follow-up work
_interleave_logs,source-header emission,
os.path.commonpathcontainment) into a shared helper thattakes already-resolved relative path(s) rather than a TI.
FileTaskHandler.readandcallback_log_reader.read_callback_logcall it.StreamingLogResponsealias.Acceptance criteria
callback_log_reader.pyno longer carries its own copies of the remote/local read andcontainment logic.
References