Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
Yu Zhang, Han, Wei, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Li, Bo, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Rosenberg, Andrew, Rohit Prabhavalkar, Daniel Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman, Bhuvana Ramabhadran, Tara N. Sainath, Moreno, Pedro, Chung‐Cheng Chiu, Johan Schalkwyk, and 2 more
Abstract
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages, and fine-tuning on a smaller labeled dataset. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks. We also demonstrate that despite using a labeled training set 1/7-th the size of that used for the Whisper model, our model exhibits comparable or better performance on both in-domain and out-of-domain speech recognition tasks across many languages.
Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.
Record
- Archive ID
- arxiv:2303.01037
- Published
- Mar 2, 2023
- Updated
- Sep 25, 2023
- arXiv
- 2303.01037
- Categories
- Speech Recognition and Synthesis
- License
- public-domain
- Provenance
- openalex+arxiv