VoiceLongMemEval: Do Assistants Remember How You Sounded?

Researchers introduced VoiceLongMemEval (VLME), a benchmark testing whether AI assistants can use paralinguistic cues such as emotion and prosody in long conversations, finding that current models show an 'affect gap' with accuracy boosts of 0.09 to 0.38 when given such metadata. The study, posted on arXiv, evaluated leading frontier and open-weight models, noting that audio-native models extract cues directly from speech (0.354 to 0.412 vs. 0.325 blind).

arXiv:2609.00570v1 Announce Type: new Abstract: With the growing scale of multi-agent architectures and large language models, deployed AI assistants are increasingly tasked with reasoning over long, continuous, multi-session conversation histories. Current benchmarks evaluate this dialogue history as information retrieval over long horizon, temporal reasoning, or knowledge updates, while crucially ignoring the fundamental dynamics of human-agent interaction, i.e. how they said it. To address this gap, we present VoiceLongMemEval VLME benchmark, where every answer depends on paralinguistic metadata emotion labels, prosody descriptors, and voice events attached to conversational turns, which is otherwise unrecoverable from the words alone. Every item passes a three-stage adversarial gate, ensuring that a strong language model fails when given only the transcript. Evaluating leading frontier and open-weight models reveals a pervasive affect gap; providing text-track paralinguistic metadata yields a 0.09 to 0.38 accuracy boost 0.61 to 0.69 when prompted with evidence hints , while standard ASR pipelines systematically discard this signal. Additionally, audio-native models successfully extract these cues directly from speech 0.354 to 0.412 vs. 0.325 blind . Code and dataset will be made available upon acceptance.