fix(tts): add streamFlush() API and drain Kokoro buffer on streamStop#1197
Open
msluszniak wants to merge 1 commit into
Open
fix(tts): add streamFlush() API and drain Kokoro buffer on streamStop#1197msluszniak wants to merge 1 commit into
msluszniak wants to merge 1 commit into
Conversation
Member
Author
|
@IgorSwat
because this will not introduce breaking change as the current state. |
msluszniak
changed the title
msluszniak
force-pushed
the
@ms/tts-stream-insert-can-flush
branch
from
7358b19 to
0e0cdb9
Compare
msluszniak
changed the title
msluszniak
force-pushed
the
@ms/tts-stream-insert-can-flush
branch
from
0e0cdb9 to
160cf11
Compare
IgorSwat
approved these changes
May 28, 2026
msluszniak
commented
May 28, 2026
Member
Author
msluszniak
left a comment
msluszniak
left a comment
There was a problem hiding this comment.
I need to add documentation for this function for flushing.
The streaming loop used a timer-based force-flush after `kStreamMaxSkippedIterations` idle iterations. It tried to extract zero characters whenever the buffer held content without an end-of-sentence character — the counter reset on every iteration and `streamStop(false)` hung forever (#1153). The same threshold also fired mid-token for LLM-style streaming, partitioning sentences before they finished. Replace the threshold with caller-driven flushing: - New `streamFlush()` API. Caller signals "drain what's currently buffered, EOS or not" — typically right before `streamStop(false)` for normal apps that ended on un-terminated content. - `streamStop(false)` now drains automatically (equivalent to `streamFlush()` plus auto-stop on empty buffer), so it's guaranteed to return even if the trailing tail has no EOS. - LLM-style callers feeding partial tokens never call `streamFlush()` — model punctuation drives natural EOS partitioning, and the residual tail is drained by `streamStop(false)` when generation completes. The skipped-iteration force-flush and its tuning constant `kStreamMaxSkippedIterations` are removed. Fixes #1153.
msluszniak
force-pushed
the
@ms/tts-stream-insert-can-flush
branch
from
160cf11 to
3f49e51
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Kokoro::streamhung when the buffer held content without an end-of-sentence character: the timer-based force-flush (afterkStreamMaxSkippedIterationsidle iterations) tried to extract 0 characters, so the loop reset its skip counter without draining anything andstreamStop(false)never returned. The same threshold also fired mid-token for LLM-style streaming, partitioning sentences before they finished.Replace the threshold with caller-driven flushing:
streamFlush()API. Caller signals "drain what's currently buffered, EOS or not" — typically right beforestreamStop(false)for normal apps that ended on un-terminated content.streamStop(false)now drains automatically (equivalent tostreamFlush()plus auto-stop on empty buffer), so it's guaranteed to return even if the trailing tail has no EOS.streamFlush()— model punctuation drives natural EOS partitioning, and the residual tail is drained bystreamStop(false)when generation completes.The skipped-iteration force-flush and its tuning constant
kStreamMaxSkippedIterationsare removed.Introduces a breaking change?
Type of change
Tested on
Testing instructions
apps/speechon Android.Text to Speech - LLM Streaming.Related issues
Fixes #1153