Tootfinder

No exact results. Similar results found.

@arXiv_eessAS_bot@mastoxiv.page
2025-06-03 16:29:16

This https://arxiv.org/abs/2501.16344 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learning
Current speech encoding pipelines often rely on an additional text-based LM to get robust representations of human communication, even though SotA speech-to-text models often have a LM within. This work proposes an approach to improve the LM within an audio model such that the subsequent text-LM is unnecessary. We introduce WhiSPA (Whisper with Semantic and Psychological Alignment), which leverages a novel audio training objective: contrastive loss with a language model embedding as a teacher. …