Allow AthenaOperator queries without a database argument#69846
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
Athena does not require the `Database` field in `QueryExecutionContext`. Queries with fully qualified table names can therefore run without a default database. https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html#API_StartQueryExecution_RequestSyntax Requiring the argument forces users to provide a default database even when the query does not need one.
When the `database` argument of `AthenaOperator` is `None`, the `Database` field in `QueryExecutionContext` can still provide the default database for unqualified table names in a query.
b49829e to
d7d2c25
Compare
|
The |
Suggested in apache#69846 (comment) It is better to explain to users who omit `database` or set it to `None` that `Database` in `query_execution_context` remains effective.
Thank you for your suggestion! |
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
Summary
This PR makes the
databaseargument optional inAthenaOperator, allowing queries that do not require a default database.This is useful when all referenced table names are fully qualified, such as in queries across multiple databases.
When
databaseis not provided,Databasefromquery_execution_contextis used as the fallback database for OpenLineage dataset extraction.Testing
breeze run pytest providers/amazon/tests/unit/amazon/aws/operators/test_athena.pypassed (16 test cases).prek run --from-ref upstream/mainpassed.databaseargument from theread_tabletask inexample_athena.pylike this:CataloginQueryExecutionContext:Was generative AI tooling used to co-author this PR?
Generated-by: OpenAI Codex (GPT-5.5) following the guidelines
Drafted-by: OpenAI Codex; reviewed by @te-horie before posting