A single-model framework for controlling accent strength from audio exemplars while preserving speaker identity.
Zero-shot text-to-speech (TTS) can reproduce an unseen speaker from a short reference recording, but typically entangles speaker identity and accent within the same reference. We introduce DEFINE, an end-to-end framework that decouples these factors by conditioning speaker identity and target accent on separate audio exemplars. A single inference-time guidance weight continuously controls accent strength without retraining. Built on F5-TTS with parameter-efficient LoRA adaptation, DEFINE maps short accent exemplars into a conditioning space using an exemplar encoder supervised through learned accent prototypes, requiring neither accent labels at inference time nor post-synthesis waveform conversion. On seen accents, increasing accent guidance improves accent-probe accuracy from 6.5% to 19.6%. More importantly, a single DEFINE model generalizes accent control beyond its training accent set: across both seen and out-of-domain accents, it matches the accent transfer performance of a two-model TTS–voice-conversion cascade while achieving higher speaker similarity and comparable predicted speech quality. These results demonstrate that speaker identity and accent can be independently controlled from audio exemplars within a single zero-shot TTS model, including for accents unseen during training.
Each row keeps the speaker from the voice prompt and takes the accent from a recording by a different person. Every system receives identical inputs and generates the same unseen sentence.
The same model and exemplar clips at increasing guidance weight. Nothing is retrained between columns; w is one scalar applied at synthesis time.