Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR
Mitesh M. Khapra Abhigyan Raman Pratyush Kumar Tahir Javed Kaushal Santosh Bhogale Sai Sundaresan
作者信息
Abstract
Improving ASR systems is necessary to make new LLM-based use-cases accessible
to people across the globe. In this paper, we focus on Indian languages, and
make the case that diverse benchmarks are required to evaluate and improve ASR
systems for Indian languages. To address this, we collate Vistaar as a set of
59 benchmarks across various language and domain combinations, on which we
evaluate 3 publicly available ASR systems and 2 commercial systems. We also
train IndicWhisper models by fine-tuning the Whisper models on publicly
available training datasets across 12 Indian languages totalling to 10.7K
hours. We show that IndicWhisper significantly improves on considered ASR
systems on the Vistaar benchmark. Indeed, IndicWhisper has the lowest WER in 39
out of the 59 benchmarks, with an average reduction of 4.1 WER. We open-source
all datasets, code and models.引用本文复制引用
Mitesh M. Khapra,Abhigyan Raman,Pratyush Kumar,Tahir Javed,Kaushal Santosh Bhogale,Sai Sundaresan.Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR[EB/OL].(2023-05-24)[2026-05-29].https://arxiv.org/abs/2305.15386.学科分类
语言学/印欧语系/计算技术、计算机技术
|国家预印本平台