07. TTS: XTTS-v2

XTTS-v2 offers voice cloning from brief audio samples plus multi-language support. The model processes text and reference audio, producing speech matching the reference voice's characteristics.

Installation from source:

git clone https://github.com/coqui-ai/TTS
cd TTS
pip install -e .

Voice cloning inference:

from TTS.api import TTS

tts = TTS("tts_models/multilingual/multi-modal/tortoise-v2")

# Clone voice from reference audio
tts.tts_to_file(
    text="This audio matches the speaker characteristics from the reference.",
    speaker_wav="reference_voice.wav",
    file_path="output.wav",
    language="en"
)

The reference audio should contain clear speech lasting 3-30 seconds. Longer samples provide more voice characteristics but increase processing time.

Voice embedding persistence across sessions requires saving speaker embeddings:

import numpy as np

# Generate and save embedding
speaker = tts.speaker_manager.speaker_by_name["my_cloned_voice"]
embedding = speaker["embedding"]

np.save("voice_embedding.npy", embedding)

Multi-language support covers dozens of languages with varying quality. Switching languages while maintaining voice clone requires models trained with that speaker across languages—rare in practice.

Generation speed varies significantly by model variant. The tortoise model produces high quality but runs slowly. Fast alternatives sacrifice quality for throughput. Benchmark on target hardware before committing to deployment architecture.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.