Research / Robust speech recognition and spoken search / [12]
ALBAYZIN 2020: searching Spanish speech by text and by example
The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses. Applied Sciences, 2021.
The first internationally open, multi-domain evaluation of search on speech in Spanish that includes both spoken term detection and query-by-example. Results leave clear room for improvement and depend strongly on the data domain.
Two ways to search speech
Search on speech covers two related tasks. In spoken term detection the query is written text. In query-by-example the query is itself a piece of audio. In both, a system returns the files that contain the query, the start and end times, and a score for how likely the detection is.
The evaluation
ALBAYZIN 2020 is the first internationally open, multi-domain evaluation in Spanish that includes both tasks. It used three databases: MAVIR, talks from workshops; RTVE, broadcast news programmes; and SPARL20, sessions of the Spanish parliament.
The paper describes the evaluation, the databases, the metric and the submitted systems, and reports the results with analyses by query type: in or out of vocabulary, one word or several, native or foreign.
What stood out
The most novel submissions were a data augmentation technique for spoken term detection and an end-to-end system for query-by-example. Overall, the results show clear room for improvement, and performance is highly sensitive to a change of data domain.
Why it matters
It continues the 2018 evaluation with a new domain and a second task, so progress in Spanish search on speech can be followed across editions.
Cite
Tejedor J., Toledano D.T., Ramírez J.M., Montalvo A.R., Álvarez-Trejos J.I. The Multi-Domain International Search on Speech 2020 ALBAYZIN Evaluation: Overview, Systems, Results, Discussion and Post-Evaluation Analyses. Applied Sciences, 2021.