Uncertainty modeling of speech

We are happy to share our latest preprint, now live on arXiv: Uncertainty Modeling in Multimodal Speech Analysis Across the Psychosis Spectrum, which is part of the TRUSTING project. This work was led by Morteza Rohanian in collaboration with the labs of Prof. Michael Krauthammer and Prof. Nicolas Langer.

Mental disorders are often accompanied by speech disruptions. These features have the potential to provide important information relevant to diagnosis and treatment. However, they can be difficult to observe and to accurately measure. They may also vary from person to person. To tackle this, we built an uncertainty-aware AI model that makes use of acoustic and semantic speech features to predict symptom severity and psychosis-related traits.

Why is this important?

• The model adapts to different conversation types, prioritising sound features in structured tasks and language features in open-ended discussions.
• By estimating uncertainty, it highlights how reliable different speech markers (like pitch changes, fluency issues, and spectral instability) are for assessment.
• It performed consistently well across various interaction contexts.

We believe that this approach could help with the early detection of recurring illness signs and are excited about this possibility. Check it out and let us know what you think.