Skip to content

perf: optimize to_local_time in datafusion-functions#23545

Draft
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/to_local_time-datafusion-20260713-202219
Draft

perf: optimize to_local_time in datafusion-functions#23545
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/to_local_time-datafusion-20260713-202219

Conversation

@andygrove

Copy link
Copy Markdown
Member

This PR was created by an LLM as a draft PR. I will mark it as ready for review after human review.

Which issue does this PR close?

N/A — autonomous exploratory PR.

Rationale for this change

Replaced chrono DateTime/TimeDelta calendar arithmetic in the per-row to_local_time hot path with an exact integer offset shift on the raw i64 timestamp, removing the TimeDelta construction, the DateTime addition, and the convert-back per element.

What changes are included in this PR?

Replaced chrono DateTime/TimeDelta calendar arithmetic in the per-row to_local_time hot path with an exact integer offset shift on the raw i64 timestamp, removing the TimeDelta construction, the DateTime addition, and the convert-back per element.

Are these changes tested?

Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs main).

Benchmark (criterion):

  • to_local_time_no_nulls_100k: 8.891% faster (base 4139190ns -> cand 3771157ns)
  • to_local_time_10pct_nulls_100k: 8.45% faster (base 3828852ns -> cand 3505310ns)

Full criterion output:

to_local_time_no_nulls_100k
                        time:   [3.7694 ms 3.7712 ms 3.7741 ms]
                        change: [−9.2033% −8.8914% −8.6695%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

to_local_time_10pct_nulls_100k
                        time:   [3.5037 ms 3.5053 ms 3.5080 ms]
                        change: [−8.5448% −8.4501% −8.3570%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

Are there any user-facing changes?

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant