restrict parser numeric fields to ASCII digits#1316
Open
arib06 wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
toxormake testto find out!).tox -e lintormake lintto find out!).masterbranch.Description of Changes
The numeric fields in the parser (year, month, day, hour, minute, second, subsecond, ISO week and timezone offset) are matched with
\d. In Python's default Unicode mode\dmatches the whole Unicode decimal-digit category, not just ASCII0-9, so a value like2021-01-02or٢٠٢١-٠١-٠٢is accepted for aYYYY-MM-DDfield even though those code points are not ISO 8601 digits.int()then happily converts them, so the non-ASCII string parses to a real date and the caller never sees that the input was not what the format claimed. I noticed it while comparingarrow.getagainstdatetime.fromisoformat, which rejects the same strings, so untrusted date input that arrow validates can round-trip differently from the stdlib. Pinning these field regexes to[0-9]keeps the parser to ASCII digits and lines it up withfromisoformat. The timestamp tokens are left as-is and every existing format still parses (full suite green).