Make Token and TokenKind Copy, drop parse-loop clones#77
Merged
StreamDemon merged 1 commit intoJul 2, 2026
Merged
Conversation
With the lexeme gone, a token is a payload-free kind plus a span — both trivially copyable. Deriving Copy lets every parse loop pass and return tokens by value, so the `.clone()` calls sprinkled through `at`, `eat`, `expect`, `bump`, `peek_kind`, and the recovery helpers all disappear. The PR #71 roadmap sketched this slot as "`at`/`eat`/`expect` take `&TokenKind`"; deriving Copy reaches the same goal (no clones in parse loops) with by-value call sites instead of reference threading, which only became possible after the span-slicing change landed.
There was a problem hiding this comment.
No issues found across 2 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Auto-approved: Refactors token representation to be Copy, removing clone calls in parser loops. No behavior change, all tests pass.
Re-trigger cubic
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CopyonTokenandTokenKind. After Store spans instead of lexeme strings on tokens #76 a token is a payload-free kind (the only payload,Keyword, is alreadyCopy) plus aSpan— both trivially copyable..clone()from the parse loops:at/eat/expect,bump,peek_kind/peek_kind_at,args, the balanced-skip helpers, andrecover_untilnow all pass tokens and kinds by value.can_begin_exprtakesTokenKindby value, so thesend-head check passes it point-free.Third sub-PR of the enhancement wave. Re-slice note: the PR #71 roadmap sketched this slot as "
at/eat/expecttake&TokenKind". DerivingCopyreaches the same goal — no clones in any parse loop — with by-value call sites instead of reference threading; it only became possible once #76 removed the owned lexeme. No behavior change.Related Issue
None (PR #71 roadmap item, re-sliced as described above).
Spec Sections Affected
None — implementation quality only.
Checklist
docs/pages — N/A, no behavior changeBuild Targets Tested
cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspaceall green locally (49 tests).Test Plan
clone_on_copylint (denied in CI) guarantees no stale clone survives.Summary by cubic
Make
TokenandTokenKindCopy and remove clone calls in parser loops by passing tokens and kinds by value. This simplifies the parser and avoids unnecessary allocations with no behavior change.CopyonTokenandTokenKind(token = kind +Span, both copyable)..clone()across parse loops:at/eat/expect,bump,peek_kind/peek_kind_at,args, balanced-skip helpers, andrecover_until.can_begin_exprnow takesTokenKindby value; simplified thesendhead check.Written for commit 45dc99c. Summary will update on new commits.