Paraformer-large (zh) — CoreML (Apple Neural Engine)
CoreML conversion of FunASR's Paraformer-large (Mandarin Chinese) for on-device inference on Apple Silicon, for FluidInference/FluidAudio.
Paraformer is a non-autoregressive ASR model: a SANM encoder, a CIF predictor that emits one acoustic-embedding token per output character, and a parallel (single-pass) decoder. Upstream: iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch.
Files (3 CoreML stages + host CIF)
| File | Precision | Compute unit | Size | Role |
|---|---|---|---|---|
ParaformerPreprocessor.mlmodelc |
FP32 | CPU | 3 MB | front-end: waveform → 560-d LFR features |
ParaformerEncoder.mlmodelc |
FP16 | ANE | 302 MB | SANM encoder (enumerated buckets [128,256,512,1024,1800]) |
ParaformerDecoder.mlmodelc |
FP16 | ANE | 109 MB | parallel decoder (enc 512, tokens 128) |
vocab.json |
— | — | — | 8404 CharTokenizer tokens (array form) |
The CIF predictor runs on the host between encoder and decoder (it emits a
dynamic token count, which a fixed-shape CoreML graph can't express). It's a
small conv1d + linear + sigmoid → integrate-and-fire; a numpy reference
(cif_numpy.py) is in the conversion repo as the Swift blueprint.
Pipeline
waveform → [Preprocessor fp32/CPU] → features [1,T,560]
→ [Encoder fp16/ANE] → enc_out [1,T,512]
→ [host CIF] → acoustic_embeds [1,L,512], token_count L
→ [Decoder fp16/ANE] → logits [1,L,8404]
→ argmax per token → drop sos(1)/eos(2)/blank(0) → CharTokenizer
Both fp16 encoder/decoder are correct on the Neural Engine. The front-end runs FP32/CPU (power-spectrum + log exceed the FP16 range). Run the encoder/decoder with
MLModelConfiguration.computeUnits = .cpuAndNeuralEngine.
Conversion notes
Two SANM-specific fixes were required for fp16/ANE under bucket padding (see the
conversion repo): a fp16-safe attention mask fill (-inf → -1e4), and building
the encoder/decoder pad-masks from the input tensor's seq dim (so
EnumeratedShapes generalize) rather than lengths.max().
Benchmark — AISHELL-1 test (CoreML on ANE)
Full test set (7,176 utts), full-CoreML pipeline on M5 Pro ANE:
| Precision | size (enc+dec) | CER | median RTFx | peak RAM |
|---|---|---|---|---|
| fp16 (default) | 411 MB | 2.12% | 85× | 0.38 GB |
| int8 | 207 MB | 2.12% | 84× | 0.24 GB |
Official Paraformer-large AISHELL-1 ≈ 1.95% CER (the ~0.17 pp gap is fp16 + the fixed-shape decoder padding). int8 weight quantization is accuracy-neutral (CER unchanged), ~half the size/memory.
Reproduces the published Paraformer-large AISHELL-1 number — confirming the conversion (front-end + encoder + CIF + decoder) is faithful.
License & attribution
Weights derive from FunASR's Paraformer-large; the upstream license applies. This repo is a format conversion only (no retraining). See FunASR.