Skip to content

Add DuckDB FlightSQL sample connector (ADBC)#542

Open
ryan-syed wants to merge 7 commits into
microsoft:masterfrom
ryan-syed:dev/ryan-syed/publishFlightSQLDuckDbSampleConnector
Open

Add DuckDB FlightSQL sample connector (ADBC)#542
ryan-syed wants to merge 7 commits into
microsoft:masterfrom
ryan-syed:dev/ryan-syed/publishFlightSQLDuckDbSampleConnector

Conversation

@ryan-syed
Copy link
Copy Markdown
Contributor

Summary

Adds a new sample connector for DuckDB using the FlightSQL protocol via ADBC (Arrow Database Connectivity). This is the first ADBC-based connector sample in the repository, intended as a reference implementation for partners building FlightSQL connectors for Power BI.

What's included

Connector source:

  • Adbc.Connection-based connectivity with FlightSQL gRPC transport
  • Full query folding via SqlView.Generator with DuckDB SQL dialect overrides
  • Navigation tree (Database > Schema > Table/View) built from information_schema
  • Primary/foreign key detection for automatic Power BI relationship creation
  • Support for 24 DuckDB types including HUGEINT, UUID, JSON, ARRAY, STRUCT
  • Three authentication modes: Username/Password, Bearer Token, Anonymous
  • DirectQuery and Native Query support

SQL Generator architecture:

  • SqlGeneratorCommon.pqm: Shared SQL92 base infrastructure (reusable by other ADBC connectors)
  • SqlGenerator.pqm: DuckDB-specific overrides (LIMIT/OFFSET, function remapping, typed literals, type facets)

Test suite:

  • 200+ standard query folding and functional tests
  • Datasource-specific tests for extended types (HUGEINT, UUID, JSON, BLOB) and nested types (ARRAY, STRUCT)
  • Performance tests over 10,000-row NYC Taxi dataset
  • Docker-based test environment using SQLFlite
  • Setup scripts and guides for running tests locally

Notes

  • The FlightSQL ADBC driver ships with Power Query SDK Tools and Power BI Desktop; no separate driver installation is needed
  • DuckDB logo icons are not included
  • Test credentials in Tests/Credentials/ are dummy values for the local Docker environment

@ryan-syed ryan-syed marked this pull request as ready for review May 14, 2026 19:16
@@ -0,0 +1,104 @@
# DuckDb Connector (FlightSQL / ADBC)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the documentation in the Power Query Documentation samples-directory and a dedicated page like GitHub Connector Sample?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, once this is published, we can add documentation for it in the power-query learn docs.

@@ -0,0 +1,426 @@
// DuckDb.pq — Sample DuckDb connector using FlightSQL via Adbc.Connection.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a demo for the connector to the PR description?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who would be the target audience for the demo and what is that we want to demo here?

@@ -0,0 +1,495 @@
// SqlGenerator.pqm — DuckDb-specific SQL generator overrides.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like Alok's take on this. SQLGenerator is its own beast that we don't have public docs for.


Queries `information_schema.table_constraints` + `key_column_usage` for PKs, applies them via `Type.ReplaceTableKeys`. Exposes `GetForeignKeys()` for FK discovery. Power BI uses these to auto-create relationships.

## Supported Types
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might need to clarify that these are specific to DuckDb and not the only types that are supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants