Text to speech

Read these first

• Speech to text

What you will know

• Synthesise speech with controllable pace and tone

  Evaluate intelligibility and naturalness at slow, fast, quiet, and emphatic settings because extreme controls can introduce skipped phonemes, distortion, or unnatural emphasis.

• Stream synthesis so playback starts before generation finishes

  Measure time to first audio and deliberately size the playback buffer; starting early is useless if generation jitter causes audible stalls.

• Measure and budget time-to-first-audio, steady-state generation, buffering, and playback

  A pipeline can meet its average target while violating the user-visible tail latency that determines whether conversation feels interactive.

• Write and apply a consent and provenance policy for a voice-cloning feature

  Retain proof of authorization, label synthetic output where appropriate, and define revocation and abuse handling before accepting a speaker sample.

• Compare two synthesis configurations on a representative script set using latency, intelligibility, naturalness, and cost

  A benchmark that uses only short clean sentences hides long-form drift, pronunciation failures, and the operating expense of higher-quality models.

Chapters

1. 1

   Playback starts on the first chunk and stalls when the buffer drains

   Time to first audio, not total synthesis time, is what makes a voice product feel responsive, because playback begins the moment the first chunk lands. Which fix works after that depends on the regime, and the test is whether steady-state generation is faster than realtime. Below it, one second of audio per 1.2 seconds of wall clock, the buffer drains monotonically and no prebuffer of any size rescues a 30-second utterance; only pipelining does. Above it, say 1.4x with occasional 600ms hiccups, the buffer refills between stalls, an 800ms prebuffer genuinely covers the hiccup, and the real argument is who pays that 600ms on the majority of turns that never stall, which is what an adaptive buffer, small until an underrun is observed, exists to settle.

   Read chapter
2. 2

   Prosody is what you control and what chunking takes away

   Pace, emphasis and tone are steerable until they are not: push the controls to their extremes and you get skipped phonemes, distortion and stress landing on the wrong syllable, so evaluate at slow, fast, quiet and emphatic settings rather than the default. Streaming sentence-sized chunks as the LLM produces them cuts first-audio latency and pipelines generation, at the cost of intonation across the boundary that the synthesiser could not see.

   Read chapter
3. 3

   Every term between the user's last word and the first syllable

   Endpointing 500ms, ASR final 200ms, LLM first token 400ms, TTS first audio 250ms, network 80ms, playout buffer 150ms: name them all before arguing about which model is slow. The largest term here is endpointing, and buying it back means cutting people off when they pause mid-sentence. Budget the tail rather than the mean, since the turn a user remembers is the one that hung.

   Read chapter
4. 4

   Consent for a cloned voice is a recording, not a checkbox

   A checkbox ticked by whoever uploaded the audio authorises nothing, because the person whose voice is being copied never touched it. What establishes authorisation is a verification statement recorded by the speaker themselves, stored with the sample, a revocation path and an abuse process. Decide in advance where output gets labelled synthetic, and be ready to refuse cloning a head of support from three public webinars.

   Read chapter
5. 5

   Benchmarking two synthesisers on scripts that can actually fail

   Short, clean, well-punctuated sentences read at a normal pace will make both providers look fine, which is exactly why that test decides nothing. Build a script set carrying long paragraphs, numbers, acronyms, product names and awkward punctuation, then score latency and cost per hour of audio at your volume. The two subjective axes need instruments rather than impressions: naturalness from a blind, randomised, multi-rater side-by-side with enough raters and clips that a preference is not noise, and intelligibility from an objective proxy, running the synthesised audio back through the recogniser of the previous topic and scoring its word error rate. Your own preference between two voices is a data point of one, collected by someone who wants an answer.

   Read chapter