arXiv:2610.31685v1 Announce Type: new Abstract: LLM-based multimodal emotion reasoning systems perform well on standard benchmarks, but improving them through instruction tuning can be computationally expensive. Agentic pipelines built from frozen models avoid this cost, but introduce a different challenge: when a prediction is wrong, the system must identify which agent or stage caused the error. We address this problem with an agentic framework for traceable multimodal emotion reasoning and prompt-space adaptation. Modality-specific perception agents analyze visual, acoustic, and lexical inputs and post evidence-cited observations to a shared board. An integration agent combines these observations with retrieved multimodal examples to produce an emotion description, label set, and supporting citations, while an MLLM critic can request one revision before the output is finalized. For incorrect predictions, a separate evidence-grounded verification stage applies deterministic checks to verify that cited evidence and references are valid, and an entailment check to determine whether the final claims are supported by that evidence. Detected failures are then grouped by the responsible component and failure type, converted into targeted prompt patches, and accepted only after validation on a held-out set. This adapts the agent prompts without updating model weights. On the MER2025 dataset, targeted prompt adaptation outperforms the frozen static pipeline on all four MLLMs, ranging from 2B to 38B parameters, and matches or outperforms undifferentiated reflection baselines on each of them: VideoLLaMA3-2B, Phi-3.5-vision-instruct, Qwen2.5-VL-7B-Instruct, and InternVL2.5-38B. It improves F-score by 0.6--2.2 points depending on the backbone, including an increase from 52.3 to 54.5 on Qwen2.5-VL-7B.
An Agentic Approach with Evidence-Grounded Verification for Multimodal Emotion Reasoning in Conversation
A new arXiv paper (2610.31685v1) reports that an agentic framework combining evidence-grounded verification with targeted prompt-space adaptation improved multimodal emotion reasoning F-scores by 0.6 to 2.2 points across four MLLMs on the MER2025 dataset, without updating model weights. The framework uses modality-specific perception agents for visual, acoustic and lexical inputs, an integration agent, an MLLM critic limited to one revision, and a verification stage that applies deterministic citation checks and an entailment check before failures are grouped by component and converted into validated prompt patches. On Qwen2.5-VL-7B-Instruct, F-score rose from 52.3 to 54.5, and the method matched or beat undifferentiated reflection baselines on VideoLLaMA3-2B, Phi-3.5-vision-instruct, Qwen2.5-VL-7B-Instruct and InternVL2.5-38B.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.