CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment

Abstract

LLM-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. Dual-streaming TTS requires accurate text--speech alignment and training sequences that balance synthesis quality and latency. Prior work often relies on GMM--HMM forced alignment (e.g., MFA), which is pipeline-heavy; fixed-ratio interleaving provides weak alignment regularity. We propose CTC-TTS, which replaces MFA with CTC phoneme--speech alignment and introduces a bi-word text--speech interleaving strategy. Two variants are designed: CTC-TTS-L (length-wise token concatenation) for higher quality and CTC-TTS-F (feature-wise embedding stacking) for lower latency. Experiments show that CTC-TTS outperforms strong baselines on streaming synthesis and zero-shot tasks.

Single-speaker Streaming Experiments

alpaca_gpt4_en_55k_28017
traveling by car has several pros and cons. on the positive side, cars offer flexibility and freedom to travel at your own pace and schedule. they provide comfort and privacy, allowing you to carry more luggage and make spontaneous stops. additionally, cars can be more cost effective for short distances, especially when traveling with multiple people. however, there are some downsides as well.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_208606
of course i'd be happy to help. could you please provide a bit more context on what you're looking for? are you searching for a product, a service, a type of professional, or perhaps information on a specific topic? the more details you share, the better i can assist you.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_392805
the reliability of yelp reviews can vary. while many users share genuine experiences, there can also be fake or biased reviews. it's a good idea to look at a variety of reviews, both positive and negative, and consider the overall trend rather than focusing on individual comments.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_177182
it's possible that different political groups have varying media and publication preferences. hustler, being an adult magazine, might attract readers from various backgrounds regardless of political affiliation. it's essential to remember that preferences in entertainment or media do not always align strictly along political lines.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_382721
that's a profound question and philosophers have debated it for centuries. some believe that people are born with a natural disposition that can lean towards good or bad. others argue that it's the environment, upbringing, and experiences that shape a person's morality and behavior.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_364102
filing for separation can vary depending on where you live. in some places, you don't have to file anything to be separated. simply living apart can be enough. however, other jurisdictions may require you to file legal documents to ensure both parties rights and responsibilities are clearly defined.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_414209
if a country like japan doesn't pay back its debts, it can face serious economic consequences. these might include a loss of investor confidence, which can lead to higher interest rates and difficulty borrowing in the future. there could also be a devaluation of the country's currency, making imports more expensive and potentially leading to inflation.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_114380
kickstarter is a crowdfunding platform where people can support creative projects by pledging money to help bring them to life. it allows creators to present their ideas and gather financial support from a wide audience. if a project reaches its funding goal within a certain time frame, it moves forward, and backers pledges are collected. if it doesn't reach its goal, no money is collected.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_451517
the high cost of rent in california is largely due to a combination of factors like high demand, limited housing supply, and strict zoning laws. california's strong economy and desirable climate attract many people, driving up demand for housing. additionally, there are often rigorous building regulations and zoning laws that make it difficult to construct new housing quickly.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX
rlhf_427152
that sounds like a good plan. wax or polish can be a gentle way to improve the appearance and protect the surface you're working on. make sure to clean the area thoroughly before applying the wax or polish for the best results. follow the instructions on the product carefully, and if possible, test it on a small, inconspicuous area first. this way you can ensure it won't cause any unintended harm.
GT
CTC-TTS-L
CTC-TTS-F
LLMVoX

Multi-speaker Continuation Experiments

7127-75946-0006
He has given them with too much grace not to have others still to give, if they are required, which is the case at the present moment.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
2830-3980-0023
These perverters of the righteousness of Christ resist the Father and the Son, and the works of them both.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
61-70970-0000
Young Fitzooth had been commanded to his mother's chamber so soon as he had come out from his converse with the Squire.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
4077-13754-0003
Moreover, had the people been inclined to rebellion what greater opportunity could they have wished?
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
121-127105-0002
cried one of the women. He took no notice of her he looked at me, but as if, instead of me, he saw what he spoke of.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
260-123286-0018
I saw at the Hamburg museum the skeleton of one of these creatures thirty feet in length.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
8463-287645-0010
He worked me very hard he wanted to be beating me all the time.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
4507-16021-0035
True history being a mixture of all things, the true historian mingles in everything.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
4507-16021-0051
In this world, evidently the vestibule of another, there are no fortunate.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
61-70968-0010
Forthwith all ran to the opening of the tent to see what might be amiss but Master Will, who peeped out first, needed no more than one glance.
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD

Multi-speaker Cross-speaker Experiments

common_voice_en_27474919-common_voice_en_27474920
She also took introduction courses in Japanese, Chinese and Arabic.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_31005683-common_voice_en_31005691
A flat body prevents it from being swept away by the current.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_22779827-common_voice_en_22779829
The tail is either very short or docked in the United States.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_18545637-common_voice_en_18545639
She searched in vain for a familiar face.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_19284142-common_voice_en_19284144
It is one of two National Park Service properties in Illinois.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_530085-common_voice_en_530084
The boy began again to read his book, but he was no longer able to concentrate.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_19730681-common_voice_en_19730680
Underwater demolition specialists may still be referred to as underwater demolition teams.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_19727183-common_voice_en_19727187
All listings are with original names.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_188092-common_voice_en_188093
He also tried to remember some good stories to relate as he sheared the sheep.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD
common_voice_en_17444666-common_voice_en_17444668
To summarize, our comprehensive survey discloses the true versatility of their method.
Prompt
GT
CTC-TTS-L
CTC-TTS-F
CTC+ELLA-V
MFA+ELLA-V
MFA+BIWORD