Document detail
ID
oai:arXiv.org:2408.09491
Topic
Computer Science - Sound Electrical Engineering and Systems...Author
Li, Yangze Wang, Xiong Cao, Songjun Zhang, Yike Ma, Long Xie, LeiCategory
Computer Science
Year
2024
listing date
8/21/2024
Keywords
decoding audio-llm speech audioAbstract
Audio-LLM introduces audio modality into a large language model (LLM) to enable a powerful LLM to recognize, understand, and generate audio.
However, during speech recognition in noisy environments, we observed the presence of illusions and repetition issues in audio-LLM, leading to substitution and insertion errors.
This paper proposes a transcription prompt-based audio-LLM by introducing an ASR expert as a transcription tokenizer and a hybrid Autoregressive (AR) Non-autoregressive (NAR) decoding approach to solve the above problems.
Experiments on 10k-hour WenetSpeech Mandarin corpus show that our approach decreases 12.2% and 9.6% CER relatively on Test_Net and Test_Meeting evaluation sets compared with baseline.
Notably, we reduce the decoding repetition rate on the evaluation set to zero, showing that the decoding repetition problem has been solved fundamentally.
Li, Yangze,Wang, Xiong,Cao, Songjun,Zhang, Yike,Ma, Long,Xie, Lei, 2024, A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition
Analysis of the ICML 2023 Ranking Data: Can Authors' Opinions of Their Own Papers Assist Peer Review in Machine Learning?
peer author-provided analysis rankings scores review learning machine science computer
