Pin databricks-sql-connector>=4.0.0 in generated constraints#69863
Conversation
thrift 0.24.0 (apache/thrift#3584) is incompatible with the databricks-sql-connector thrift<=0.23.0 requirement, so during highest resolution the resolver preferred the newest thrift and downgraded the connector, which in turn dragged the databricks provider back to an old version. Pinning the connector to >=4.0.0 keeps PyPI constraints installable with a current databricks-sql-connector and provider. closes: apache#69603
potiuk
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496 and
jscheffl
as code owners
boring-cyborg
Bot
added
area:dev-tools
backport-to-v3-3-test
|
FYI: @vatsrahul1001 -> this one is important for 3.3.1 to merge as the databricks provider got downgraded because of interaction between thrift and databricks-mysql-connector dependencies. @eladkal - I will also separately add protection to flag when things like that happen in the future - I think downgrading any of our providers is a sign that something weird happens with dependency resolution. |
Flag how? The mechanism doesn't involve release manager. Are you thinking about Slack notification? Or a manual step during core release that will verify the constraints file against this issue? |
|
Before the release -> posting message to internal-airflow-ci-cd when provider gets downgraded . Those things do not happen at release time - this one happened literally few days before See: #69870 |
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
…nts (apache#69863) thrift 0.24.0 (apache/thrift#3584) is incompatible with the databricks-sql-connector thrift<=0.23.0 requirement, so during highest resolution the resolver preferred the newest thrift and downgraded the connector, which in turn dragged the databricks provider back to an old version. Pinning the connector to >=4.0.0 keeps PyPI constraints installable with a current databricks-sql-connector and provider. (cherry picked from commit 98b4b01) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> closes: apache#69603
…nts (apache#69863) thrift 0.24.0 (apache/thrift#3584) is incompatible with the databricks-sql-connector thrift<=0.23.0 requirement, so during highest resolution the resolver preferred the newest thrift and downgraded the connector, which in turn dragged the databricks provider back to an old version. Pinning the connector to >=4.0.0 keeps PyPI constraints installable with a current databricks-sql-connector and provider. (cherry picked from commit 98b4b01) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> closes: apache#69603
…nts (#69863) (#69871) thrift 0.24.0 (apache/thrift#3584) is incompatible with the databricks-sql-connector thrift<=0.23.0 requirement, so during highest resolution the resolver preferred the newest thrift and downgraded the connector, which in turn dragged the databricks provider back to an old version. Pinning the connector to >=4.0.0 keeps PyPI constraints installable with a current databricks-sql-connector and provider. (cherry picked from commit 98b4b01) closes: #69603 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
3 participants
During highest-resolution constraint generation, the resolver preferred the newest
thrift(0.24.0, from apache/thrift#3584) over the newestdatabricks-sql-connector. Because currentdatabricks-sql-connectorrequiresthrift<=0.23.0, the resolver downgraded the connector — which in turn dragged thedatabricksprovider back to an old version in the PyPI constraints.Pinning
databricks-sql-connector>=4.0.0inadditional_constraints_for_highest_resolutionkeeps the constraints installable with a current connector and provider.Tracked upstream at databricks/databricks-sql-python#859.
closes: #69603
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8, 1M context) following the guidelines