Working papers · speech, health & trustworthy AIVigo, 2026

Researcher · ML engineer_

What the voice reveals, and how to keep AI that listens safe

José Manuel Ramírez Sánchez1

1Multimedia Technologies Group (GTM), atlanTTic Research Center, Universidade de Vigo

0000-0003-4700-6592 josemspeechtech

Plate I. The author, Atkinson 1-bit dither.

Abstract

I am a speech and language technology researcher finishing my PhD at the Universidade de Vigo, and an engineer who ships. I build models that read health from the voice: Long COVID, respiratory disease, emotional distress and suicide risk. In industry, I delivered the voice classifier for the COPERIA screening platform at Bahía Software, and I now advise Balidea on agentic AI systems for the Galician Health Service (SERGAS). My focus now is making generative AI that talks with vulnerable people aligned, robust and hard to misuse.

Keywords — speech biomarkers; multimodal learning; self-supervised speech representations; AI safety; alignment; safeguards for generative AI; mental-health AI.

§1 Research lines

Table 1. Lines of work, the modalities involved and the papers behind each.
LineModalityRefs.
Voice as a clinical signal [1][2][7][9]
Mental-health AI and conversational safeguards [5][6][8]
Robust speech recognition and spoken search [3][4][12][13][14][15][16][17]
Language technology for under-resourced languages [10][11]
Alignment and safeguards for generative AI (current) —

§2 Selected papers

  1. [1]

    A short walk makes voice-based screening more reliable: coughs and vowels recorded after exercise reveal post-COVID sequelae better than recordings at rest.

    Vera-López Á. et al. · Frontiers in Medicine · 2026

    Journal Read summary DOI
    Full citation

    Vera-López Á., Tilves-Santiago D., Ramírez-Sánchez J.M., Docío-Fernández L., García-Mateo C., Bustillo-Casado M., García-Caballero A.A. Improving respiratory disease detection through SSL-enhanced acoustic analysis and exercise-rest measurements. Frontiers in Medicine, 2026.

  2. [2]

    A new way to adapt large speech models to small clinical datasets, which beats standard LoRA by 12.4 points on four health tasks and is the only method that detects suicidal ideation in one of them.

    Ramírez-Sánchez J.M. et al. · IberSPEECH · 2026

    Conference Read summary
    Full citation

    Ramírez-Sánchez J.M., Docío-Fernández L., García-Caballero A.A., García-Mateo C. Radial LoRA: Enhancing Parameter-Efficient Fine-Tuning for Clinical Speech Assessment. IberSPEECH 2026, 2026.

  3. [3]

    A system that tells who spoke when in long rural interviews, by fusing two complementary diarizers: second place in the ALBAYZIN 2026 diarization track.

    Ramírez-Sánchez J.M. et al. · IberSPEECH · 2026

    Conference Read summary
    Full citation

    Ramírez-Sánchez J.M., Piñeiro-Martín A., Docío-Fernández L., García-Mateo C. Robust Speaker Diarization under Spontaneous Rural Interviews via Cross-Architecture Fusion: The IberSPEECH 2026 COSER Challenge. IberSPEECH 2026, ALBAYZIN Speech-COSER Challenge, 2026.

  4. [4]

    A speech recogniser for dialectal rural Spanish, built by adapting Whisper with a new low-rank method: second place in the ALBAYZIN 2026 ASR track.

    Ramírez-Sánchez J.M. et al. · IberSPEECH · 2026

    Conference Read summary
    Full citation

    Ramírez-Sánchez J.M., Esperon-Couceiro A., Piñeiro-Martín A., Docío-Fernández L., García-Mateo C. Robust Speech Recognition under Spontaneous Rural Interviews via Radial Low Rank Adaptation: The IberSPEECH 2026 Speech-COSER Challenge. IberSPEECH 2026, ALBAYZIN Speech-COSER Challenge, 2026.

  5. [6]

    A chat agent designed to spot suicide risk factors during a live conversation and to respond safely when it finds them.

    Ramírez Sánchez J.M. et al. · IWSDS · 2025

    Conference Link
    Full citation

    Ramírez Sánchez J.M., Manso Vázquez M., García-Mateo C., Docío-Fernández L., Fernández-Iglesias M.J., Gómez-Gómez B., Pinal B., Brañas A., García-Caballero A. Design of a conversational agent to support people on suicide risk. IWSDS 2025 (International Workshop on Spoken Dialogue Systems Technology), 2025.

  6. [9]

    The first study to test whether voice recordings alone can tell Long COVID patients apart from healthy people. Coughs after exertion worked best.

    Ramírez Sánchez J.M. et al. · JMIR Preprints · 2023

    Preprint Read summary DOI
    Full citation

    Ramírez Sánchez J.M., Docío-Fernández L., García Mateo C., Bustillo-Casado M., García-Caballero A.A. Identifying patients with Long COVID: study of the effectiveness of voice signal analysis and machine learning. JMIR Preprints, 2023.

  7. [16]

    MFCC or PLP? In quiet rooms it barely matters; in noise, MFCC keeps speech recognition working better, while PLP decodes faster. My first paper.

    Ramírez Sánchez J.M. et al. · Revista Ingeniería Electrónica · 2019

    Journal Read summary Link
    Full citation

    Ramírez Sánchez J.M., Montalvo Bereau A.R., Calvo de Lara J.R. Evaluación de Rasgos Acústicos para el Reconocimiento Automático del Habla en Escenarios Ruidosos usando Kaldi. Revista Ingeniería Electrónica, Automática y Comunicaciones (RIELAC) 40(3), 2019.

All papers (17) →

§3 Tutorials

MFCC and PLP from scratch: how machines first learned to listen

Build the two classic speech features step by step in NumPy, watch one frame of real speech travel through both pipelines, and see what noise does to them. The front end every recogniser used before deep learning, and the subject of my first paper.

NumPySciPylibrosaWhisper

Fine-tuning Whisper with Optuna, cross-validation and MLflow

A complete, reproducible fine-tuning loop for Whisper on real speech, with grouped cross-validation, Optuna pruning, nested MLflow runs and a locked test set, ending in an explicit decision against a zero-shot baseline.

WhisperTransformersOptunaMLflowLibriSpeech

Radial LoRA on Whisper, implemented from scratch

Write R-LoRA in a few lines of PyTorch, check it numerically, train a Whisper classifier next to a LoRA baseline, and learn the two things that change in practice: the learning rate and merging.

WhisperPyTorchR-LoRAPEFTscikit-learn

The 10% of the terminal: a lab with four flags

The four skills of the terminal post, practised on a real SSH server inside Google Colab. Each section hides a flag, and a check cell tells you how many you have.

BashOpenSSHtmuxtop

§4 Writing

Radial LoRA: when the direction matters more than the size

R-LoRA is a variant of LoRA that tempers the size of the adapter's input with one learnable exponent per layer. On four clinical speech corpora it beats LoRA by 12.4 points of Macro-F1, and it is a drop-in change for any Transformer that LoRA can adapt.