Description
Airflow Params are based on JSON Schema, but there is no clear support for duration values. Since JSON Schema already defines a duration format, it could be useful to support something like Param("PT15M", type="string", format="duration").
Pendulum also seems to support parsing ISO 8601 durations, although that support is not properly documented yet. I just opened a related issue about that: python-pendulum/pendulum#961.
Open points:
- How this should be rendered in the Trigger DAG form.
- I would assume full ISO 8601 durations are supported.
- Whether Airflow should also support a fixed-duration subset, meaning durations limited to non-calendar units such as days, hours, minutes, and seconds, excluding months or years.
Use case/motivation
This could be useful for data pipelines that need to configure time windows, offsets, or latency thresholds through Params, for example to consume data for the last X hours, apply a 1 hour offset, etc.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Description
Airflow Params are based on JSON Schema, but there is no clear support for duration values. Since JSON Schema already defines a
durationformat, it could be useful to support something likeParam("PT15M", type="string", format="duration").Pendulum also seems to support parsing ISO 8601 durations, although that support is not properly documented yet. I just opened a related issue about that: python-pendulum/pendulum#961.
Open points:
Use case/motivation
This could be useful for data pipelines that need to configure time windows, offsets, or latency thresholds through Params, for example to consume data for the last X hours, apply a 1 hour offset, etc.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct