# Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages

Preprint · Yu Zhang, Han, Wei, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Li, Bo, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Rosenberg, Andrew, Rohit Prabhavalkar, Daniel Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman, Bhuvana Ramabhadran, Tara N. Sainath, Moreno, Pedro, Chung‐Cheng Chiu, Johan Schalkwyk and 2 more

- Archive id: arxiv:2303.01037
- Published: 2023-03-02
- arXiv: https://arxiv.org/abs/2303.01037
- Citations: 110
- Source URL: http://arxiv.org/abs/2303.01037
- PDF: https://arxiv.org/pdf/2303.01037
- Provenance: openalex+arxiv

## Abstract

We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages, and fine-tuning on a smaller labeled dataset. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks. We also demonstrate that despite using a labeled training set 1/7-th the size of that used for the Whisper model, our model exhibits comparable or better performance on both in-domain and out-of-domain speech recognition tasks across many languages.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Efficient Training & Inference](https://space-kitty.com/topics/efficient-ai)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.