machinekind/Matcha-TTS-PL-v2

Matcha-TTS-PL v2 — conversational Polish, stochastic durations

Small, fast, non-autoregressive Polish TTS (Matcha-TTS, 20.9 M parameters + HiFi-GAN; real-time on a small GPU or in a browser) distilled from VoxCPM2, a 2B-parameter TTS, for a humanoid robot. One model per synthetic voice, each with its own fine-tuned vocoder. English words inside Polish sentences are read in English.

New in v2 (compared with Matcha-TTS-PL):

• Conversational training data — polish-synthetic-speech-voxcpm2-v2: what a robot says to a person (reactions, questions, short and long turns in 8 everyday themes) instead of mostly informative multi-sentence passages; VoxCPM2 at classifier-free guidance 2.5.
• Stochastic durations (default): a flow-matching duration predictor (10 Euler steps, in the ONNX graph) draws the rhythm of each utterance, instead of Matcha's deterministic predictor, which averages timing into a regular, metronome-like rhythm. Mean tempo is calibrated to equal the deterministic one; both are shipped.
• 20 000 training steps (v1: 7 500) and a vocoder fine-tuned on this model's mels; mel statistics of the training data in the checkpoint (v1 kept the warm start's, ~15 dB too quiet, compensated by its vocoder).

Family: Matcha-TTS-PL-v2 · Matcha-TTS-PL-Edge-v2 · dataset v2 · demo Space · base: Matcha-TTS-PL-Pretrain · previous version: Matcha-TTS-PL

voice	gender	v2, stochastic durations: UTMOS ↑ / WER ↓ / similarity ↑	v2, deterministic	v1 (Matcha-TTS-PL)	designed from the description
male_voxcpm_2	male	3.65 / 2.7 % / 0.858	3.74 / 2.7 % / 0.862	3.42 / 2.7 % / 0.853	Voice of a dog, fast, energetic, crazy, mid/high pitch.
female_voxcpm_2	female	3.63 / 4.0 % / 0.909	3.62 / 2.7 % / 0.912	3.27 / 3.4 % / 0.910	Female, teenager, energetic, optimistic, confident, ready to help, speaks crip Polish, mid/high-pitch timbre, mid/fast tempo.

12 held-out sentences (samples/<voice>/), ONNX full chain, 4 ODE steps, temperature 0.8, tempo 0.8, neutral style, Mac CPU; UTMOS22, Whisper large-v3-turbo letter error rate, Resemblyzer similarity to the voice's anchor. These move by about ±0.1 between runs (the model samples noise); on 12 sentences the versions are close. Stochastic vs deterministic durations on more text (the 92 held-out conversational texts × 3 seeds = 276 clips, UTMOS22, standard error ≈ 0.02): male_voxcpm_2: full chain, 4 steps 3.75 stochastic vs 3.72 deterministic; female_voxcpm_2: full chain, 4 steps 3.69 stochastic vs 3.68 deterministic. Equal in UTMOS; what changes is the rhythm.

male_voxcpm_2, 92 held-out conversational texts (E79), same chain for all three:

model	quality gate pass	CER	UTMOS22	UTMOSv2	similarity	pitch range (st)	tempo variation (CV)
Matcha-TTS-PL (v1)	92 %	0.5 %	3.59	2.87	0.857	18.1	0.114
v2, deterministic durations	91 %	0.7 %	3.63	2.89	0.855	18.3	0.104
v2, stochastic durations	92 %	1.0 %	3.66	3.00	0.860	19.2	0.115

Gate = the dataset's quality gate (Whisper text match, UTMOS, similarity, tempo, pitch). Pitch range and tempo variation across utterances are where stochastic durations differ most; in our listening they give a less regular, more spoken rhythm.

The voices are synthetic — designed from a text description with VoxCPM2 (OpenBMB, Apache-2.0) and frozen as prompt caches; not recordings or clones of real people. Try them in the browser: spaces/machinekind/matcha-tts-pl-v2. For CPUs (Raspberry Pi / Jetson class), int8 and streaming: Matcha-TTS-PL-Edge-v2.

Files

browser / ONNX (acoustic model + vocoder in one graph, 4 / 2 ODE steps)	PyTorch: Matcha checkpoint (speaker 20), duration predictor	HiFi-GAN generator
onnx/male_voxcpm_2/matcha_pl_sdp_t4.onnx, matcha_pl_sdp_t2.onnx (stochastic), matcha_pl_t4.onnx, matcha_pl_t2.onnx (deterministic), voices.json	model/matcha_pl_v2_male_voxcpm_2.ckpt, model/sdp_v2_male_voxcpm_2.pt	vocoder/male_voxcpm_2/generator
onnx/female_voxcpm_2/matcha_pl_sdp_t4.onnx, matcha_pl_sdp_t2.onnx (stochastic), matcha_pl_t4.onnx, matcha_pl_t2.onnx (deterministic), voices.json	model/matcha_pl_v2_female_voxcpm_2.ckpt, model/sdp_v2_female_voxcpm_2.pt	vocoder/female_voxcpm_2/generator

onnx/voices_index.json lists the voices with their paths and defaults.

• Stochastic graphs (matcha_pl_sdp_t*.onnx): inputs x (phoneme ids, int64 [1, T]), x_lengths, scales = [temperature, length_scale, dur_temperature], spk_emb (float32 [1, 64]: speakers["0"].emb + a style embedding from the voice's voices.json), z_dur (float32 [1, 1, T], standard normal: one draw per phoneme, the rhythm's randomness; fix it to repeat a rendering); outputs wav (22.05 kHz), wav_lengths. Defaults in voices.json: temperature 0.8, length_scale 0.8, dur_temperature 0.667.
• Deterministic graphs (matcha_pl_t*.onnx): the interface of Matcha-TTS-PL — x, x_lengths, scales = [temperature, length_scale], spk_emb.

path	what
english/	English detection and the mixed phonemizer (Python and browser), see below
samples/<voice>/	the 12 held-out sentences (stochastic, 4 steps, defaults) and the VoxCPM2 teacher reading them
data/	style token map, speaker map
ATTRIBUTION.md	every text source of the training data, the vocoder's origin and the base model's data

Quick start (ONNX, Python)

import re, sys, json, numpy as np, onnxruntime as ort, soundfile as sf
sys.path.append("english")                          # english_detect.py + english_words.txt (optional fallback: pip install wordfreq)
from english_detect import mark_english
from matcha.text import text_to_sequence            # Matcha-TTS with the tts-pl cleaners: paste english/polish_cleaners.py and
from matcha.utils.utils import intersperse          # english/polish_mixed_cleaners.py into matcha/text/cleaners.py
text = mark_english("Po meetingu wyślę ci feedback na maila? A w weekend zrobię update.")
text = re.sub(r"\s+\?", "?", text)                # keep '?' glued to the word, as in the training text
ids = np.array(intersperse(text_to_sequence(text, ["polish_mixed_cleaners"])[0], 0), np.int64)
v = "male_voxcpm_2"
vj = json.load(open(f"onnx/{v}/voices.json"))
spk = np.array(vj["speakers"]["0"]["emb"], np.float32) + np.array(vj["styles"]["8"]["emb"], np.float32)   # 8 = neutral style
sess = ort.InferenceSession(f"onnx/{v}/matcha_pl_sdp_t4.onnx")
z_dur = np.random.default_rng().standard_normal((1, 1, ids.size)).astype(np.float32)   # a new rhythm on every call
wav, n = sess.run(None, {"x": ids[None], "x_lengths": np.array([ids.size]), "spk_emb": spk[None], "z_dur": z_dur,
                         "scales": np.array([0.8, 0.8, 0.667], np.float32)})   # temperature, tempo (length_scale), dur_temperature
sf.write("out.wav", wav[0, :n[0]], 22050)

PyTorch (Matcha-TTS code of the tts-pl project, with the duration predictor): load model/matcha_pl_v2_<voice>.ckpt, attach_sdp(load_sdp("model/sdp_v2_<voice>.pt")), then synthesise(..., spks=torch.tensor([20]), styles=torch.tensor([8]), duration_mode="stochastic"); vocoder vocoder/<voice>/generator (HiFi-GAN v1 config vocoder/<voice>/config.json).

Text front end: mark_english() wraps English words in braces (Po {meeting}u wyślę ci {feedback}.), the cleaner phonemizes braced spans with espeak-ng en-us and the rest with pl (with the Polish ending of an inflected English word glued on unstressed, and one-letter prepositions before English words handled by rule). You can brace any word yourself. Questions: keep the ? glued to the word ("kuchni?"), as in the training text — the model follows VoxCPM2's question intonation (wh-questions fall, yes/no questions rise); a space before ? makes wh-questions rise. The browser port produces the training phonemes character for character (tested on 300 sentences).

Training

• Base: Matcha-TTS-PL-Pretrain, the voice's speaker row initialised from the closest real reader; 18 style tokens, the same phoneme table.
• Data: polish-synthetic-speech-voxcpm2-v2 — about 9 800 clips (≈ 11 h) per voice, generated by VoxCPM2 with each frozen voice (anchor + continuation, 10 diffusion steps, guidance 2.5), every clip gated by Whisper (text match), UTMOS ≥ 3.2, speaker similarity ≥ 0.80, tempo and pitch, up to 3 takes per sentence.
• Matcha fine-tune per voice: 20 000 steps, effective batch 64, lr 5e-5, bf16, 2 × RTX 3090 (≈ 1.7 h).
• Stochastic duration predictor: an optimal-transport conditional flow-matching model on the frozen encoder's features, trained 20 000 steps on the voice's monotonic-alignment durations; sampled with 10 Euler steps, durations rounded; length factor calibrated so the mean utterance length equals the deterministic predictor's (in voices.json).
• Vocoder per voice: HiFi-GAN universal v1 fine-tuned on teacher-forced mels of the voice's final Matcha checkpoint for all its training clips, paired with the VoxCPM2 audio, 12 500 steps, lr 2e-5.

Limitations

• One voice per model (no blending). Questions: the model follows the teacher's intonation, which rises on only part of the yes/no questions.
• English detection uses a curated list (≈ 1 900 words and brands, Polish lookalikes excluded); an unlisted English word is read the Polish way unless you put it in braces.
• Synthetic voices: label generated speech as synthetic; don't present the voices as real people.

Licence

CC BY-SA 4.0 (training data includes CC BY-SA texts; the base model is CC BY-SA 4.0). VoxCPM2 is Apache-2.0; HiFi-GAN universal v1 (MIT); espeak-ng (runtime phonemizer) is GPL-3.0. See ATTRIBUTION.md.