Decoder-only text-to-speech (TTS) models scale efficiently but remain prone to content hallucinations that arise from weak text–speech alignment during autoregressive generation. We find that robustness is governed by a non-monotone relation to the sharpness of the alignment-bearing attention heads: a moderate degree is best, whereas over-sharpening is no better than the unaligned backbone and even less robust. Guided by this, we present AlignDPO, a post-training method that reaches this moderate regime by folding a lightweight connectionist-temporal-classification (CTC) alignment term into Direct Preference Optimization (DPO), applied only to the chosen samples, with no architectural or inference-time change. On the Seed-TTS-Eval English set, this significantly reduces the content-hallucination and word error rates relative to a strong DPO baseline and lowers the severe content-hallucination rate to ~0.6% (from 4.4%); a listening study further finds it preferred for naturalness over both the backbone and that baseline. Alignment is thus best learned and kept moderate rather than maximized or imposed at decoding.
Same prompt and target text across systems. DPO+M (our champion) internalizes the alignment constraint and removes the skips and repetitions left by Base and plain DPO.
| Target Text | Prompt | Base ($\theta_{\text{base}}$) | Base+M ($\theta_{\text{M}}$) | DPO | DPO+M ($\theta_{\text{DPO+M}}$, ours) |
|---|
An inference-time attention constraint (ACI) forces each alignment head into a narrow window whose width shrinks with the head's sharpness. On an over-sharpened model (near one-hot attention) this window is strangling and the model collapses; on our moderately-sharp champion the same constraint is harmless.
| Target Text | Prompt | DPO+M (over-sharp init) | DPO+M (over-sharp init) + ACI | DPO+M (moderate, ours) | DPO+M (moderate, ours) + ACI |
|---|