Skip to content

restrict parser numeric fields to ASCII digits#1316

Open
arib06 wants to merge 1 commit into
arrow-py:masterfrom
arib06:parser-ascii-digits
Open

restrict parser numeric fields to ASCII digits#1316
arib06 wants to merge 1 commit into
arrow-py:masterfrom
arib06:parser-ascii-digits

Conversation

@arib06

@arib06 arib06 commented Jul 9, 2026

Copy link
Copy Markdown

Pull Request Checklist

  • 🧪 Added tests for changed code.
  • 🛠️ All tests pass when run locally (run tox or make test to find out!).
  • 🧹 All linting checks pass when run locally (run tox -e lint or make lint to find out!).
  • 📚 Updated documentation for changed code.
  • ⏩ Code is up-to-date with the master branch.

Description of Changes

The numeric fields in the parser (year, month, day, hour, minute, second, subsecond, ISO week and timezone offset) are matched with \d. In Python's default Unicode mode \d matches the whole Unicode decimal-digit category, not just ASCII 0-9, so a value like 2021-01-02 or ٢٠٢١-٠١-٠٢ is accepted for a YYYY-MM-DD field even though those code points are not ISO 8601 digits. int() then happily converts them, so the non-ASCII string parses to a real date and the caller never sees that the input was not what the format claimed. I noticed it while comparing arrow.get against datetime.fromisoformat, which rejects the same strings, so untrusted date input that arrow validates can round-trip differently from the stdlib. Pinning these field regexes to [0-9] keeps the parser to ASCII digits and lines it up with fromisoformat. The timestamp tokens are left as-is and every existing format still parses (full suite green).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant