Skip to content

Centralise structlog initialisation in BaseDagBundle#69859

Open
dabla wants to merge 2 commits into
apache:mainfrom
dabla:feature/make-basedagbundle-logging-dry
Open

Centralise structlog initialisation in BaseDagBundle#69859
dabla wants to merge 2 commits into
apache:mainfrom
dabla:feature/make-basedagbundle-logging-dry

Conversation

@dabla

@dabla dabla commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

BaseDagBundle now owns the structlog logger via a lazy _log property, so subclasses no longer need to construct and bind it themselves in their constructors. I saw all Dag bundles extending this class re-implemented the same logic regarding the structlog initialization when reviewing PR #67016, hence why I created this PR to make the code more DRY.

What changed

  • BaseDagBundle gains a _log lazy property that creates a structlog
    logger on first access, bound with bundle_name, version, and
    whatever _log_context() returns.
  • A _log_context(self) -> dict template-method hook (returns {} by
    default) lets each subclass declare its own extra log fields without
    duplicating the boilerplate structlog.get_logger(...).bind(...) call.
  • GCSDagBundle, S3DagBundle, and GitDagBundle each implement
    _log_context() and drop their manual self._log = log.bind(...) init
    blocks and the now-unused module-level / local structlog.get_logger
    calls.
  • LocalDagBundle needs no change — the default _log_context() returns
    {}, which is the correct behaviour for a bundle with no extra context.

Why lazy?

BaseDagBundle.__init__ is called via super().__init__() before subclass-specific attributes (e.g. bucket_name, bare_repo_path) are set. Evaluating _log_context() at base-__init__ time would therefore fail or produce incomplete context. The lazy property defers evaluation until first use, by which point all __init__ work across the hierarchy is complete.


Was generative AI tooling used to co-author this PR?
  • [ x ] Yes (please specify the tool below)

Claude Sonnet 4.6


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:DAG-processing area:providers provider:amazon AWS/Amazon - related issues provider:git provider:google Google (including GCP) related issues labels Jul 14, 2026
@dabla dabla force-pushed the feature/make-basedagbundle-logging-dry branch from dbb2de2 to b6d4f1c Compare July 14, 2026 12:25
… versions can also benefit from the common log solution
@dabla dabla force-pushed the feature/make-basedagbundle-logging-dry branch from b6d4f1c to 1b7170f Compare July 14, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing area:providers provider:amazon AWS/Amazon - related issues provider:git provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant