Apache Airflow version:
3.x
Is your feature request related to a problem?
Setting AIRFLOW__LOGGING__JSON_LOGS=True has no effect on the Celery worker's stdout. The worker always emits plain-text logs. Other components (API server, core logging) already respect this setting, but celery_command.py calls configure_logging without passing json_output:
if AIRFLOW_V_3_0_PLUS:
from airflow.sdk.log import configure_logging
configure_logging(output=sys.stdout.buffer) # json_output defaults to False
Proposed solution
Add a [celery] json_logs option for a celery-specific override, falling back to the existing [logging] json_logs — consistent with how [logging] CELERY_LOGGING_LEVEL / [logging] LOGGING_LEVEL already work in the same file:
if AIRFLOW_V_3_0_PLUS:
from airflow.sdk.log import configure_logging
json_output = conf.getboolean("celery", "json_logs", fallback=None)
if json_output is None:
json_output = conf.getboolean("logging", "json_logs", fallback=False)
configure_logging(output=sys.stdout.buffer, json_output=json_output)
Changes required:
providers/celery/src/airflow/providers/celery/cli/celery_command.py — the lookup above
- Celery provider config schema — add
[celery] json_logs option
Who would benefit?
Operators running the Celery executor on Kubernetes or any platform that routes worker stdout to a structured log collector.
Are you willing to submit a PR?
Yes.
Apache Airflow version:
3.x
Is your feature request related to a problem?
Setting
AIRFLOW__LOGGING__JSON_LOGS=Truehas no effect on the Celery worker's stdout. The worker always emits plain-text logs. Other components (API server, core logging) already respect this setting, butcelery_command.pycallsconfigure_loggingwithout passingjson_output:Proposed solution
Add a
[celery] json_logsoption for a celery-specific override, falling back to the existing[logging] json_logs— consistent with how[logging] CELERY_LOGGING_LEVEL/[logging] LOGGING_LEVELalready work in the same file:Changes required:
providers/celery/src/airflow/providers/celery/cli/celery_command.py— the lookup above[celery] json_logsoptionWho would benefit?
Operators running the Celery executor on Kubernetes or any platform that routes worker stdout to a structured log collector.
Are you willing to submit a PR?
Yes.