perf: avoid tokenizing lines that pruning discards

[dexhunter]

Problem

Issue #4947 and PR #5310 removed repeated shift() / pop() work from pruneLinesFromTop and pruneLinesFromBottom. The current implementation still tokenizes every line up front, including lines that are immediately discarded when a large prompt is pruned to a small token budget.

On current main (d0a3c0b626b5bebc3bef4742eec05a0242be0bab), a deterministic 320-line fixture performs 641 Tiktoken.encode calls across top and bottom pruning. Traversing inward from the side that will be retained and stopping when the next line would exceed the budget reduces this to 88 calls (-86.3%).

At the 2,000-line scale from #4947, the same measurement is 4,001 calls on main and 86 with retained-side traversal (-97.9%). All nine runs at each revision produced the same count.

Correctness check

I compared the proposed traversal against current main over 580 top/bottom cases covering empty input, leading and trailing blank lines, newline boundaries, Unicode, single oversized lines, random multiline prompts, and zero or negative limits. All returned strings were byte-for-byte identical.

Proposed change

• Accumulate retained lines from the bottom for pruneLinesFromTop.
• Accumulate retained lines from the top for pruneLinesFromBottom.
• Preserve the existing one-token newline accounting exactly.
• Enable the existing focused pruning tests and keep the production change limited to the two function bodies.

Would this follow-up optimization be welcome?