FuseAlign multilingual (CJK fine-tune, step 2,500)

Word-level forced aligner: given a recording and its transcript, it predicts each word's start and end on a 1 ms grid plus a presence probability. A reproduction of FuseAlign (Vaidya et al., 2026) by Zyphra: the multilingual model (100,000 steps), fine-tuned 2,500 steps on CJK junction-corrected labels.

Code, usage, long-form alignment and the C++/ggml CLI: github.com/Zyphra/fusealign-multilingual.

Files

File	Size	Use
fusealign-multilingual-cjkft-step2500.safetensors	1.34 GB	PyTorch package (torch.hub, fusealign_hub): FuseAlign fp32, Dia text encoder bf16
fusealign-multilingual-cjkft-step2500-f32.gguf	1.35 GB	ggml port: FuseAlign F32, Dia BF16, Silero VAD F32
fusealign-multilingual-cjkft-step2500-q8_0.gguf	0.51 GB	ggml port: transformer matmuls Q8_0, other Dia tensors BF16, the rest F32

Usage

import soundfile as sf
import torch

aligner = torch.hub.load("Zyphra/fusealign-multilingual", "fusealign", trust_repo=True, device="cuda")
wav, sr = sf.read("speech.wav", dtype="float32")
words = aligner.align(torch.from_numpy(wav), sr, "the transcript of the recording")
# [WordAlignment(word='the', start=..., end=..., presence=..., placed=True), ...]

Evaluation

% of word boundaries off by more than 50 ms (the paper's asymmetric error), against Qwen3-ForcedAligner-0.6B (qwen-asr 0.0.6, bf16, whole clips) on the same clips, transcripts and references:

Corpus	FuseAlign	Qwen3-FA
TIMIT	16.55	20.17
Buckeye	24.58	24.60
AMI-silver	49.38	58.22
TIMIT-long	17.25	20.04
BZNSYP Mandarin (gold)	15.94	28.04
FLEURS Mandarin	17.95	11.06
FLEURS Japanese	14.02	8.38
FLEURS Korean	8.39	11.53
FLEURS Spanish	19.66	11.71
FLEURS Portuguese	9.30	10.24
FLEURS French	13.15	12.35
FLEURS German	13.29	13.91
FLEURS Russian	12.74	11.18

FLEURS references are Montreal Forced Aligner output, not hand labels; BZNSYP's are hand-labelled.

License

Apache-2.0. The checkpoint and GGUFs include the Dia text encoder weights from nari-labs/Dia-1.6B-0626 (Apache-2.0); the GGUFs include Silero VAD weights (MIT).

Citation

@article{vaidya2026fusealign,
  title   = {FuseAlign: Forced Alignment in the Wild},
  author  = {Vaidya, Mithilesh and Bailey, Stephen and Badam, Sumukh and Bendel, Matthew and He, Xingzhe},
  journal = {arXiv preprint arXiv:2609.33650},
  year    = {2026}
}