cd /news/artificial-intelligence/hierarchical-multimodal-interaction-… · home › topics › artificial-intelligence › article
[ARTICLE · art-159277] src=arxiv.org ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

Hierarchical Multimodal Interaction Reasoning for Dyadic Relationship Recognition

A research team posted arXiv:2610.36249v1, introducing HiRe-DRR, a Hierarchical Multimodal Interaction Reasoning framework for dyadic relationship recognition that outperforms recent image-, video-, and multimodal methods across three dyadic relationship benchmarks. HiRe-DRR decomposes facial, bodily, acoustic, and textual cues into low- and high-level representations, builds homogeneous and heterogeneous cross-participant interactions, and uses a Mixture-of-Interaction Transformer (MoI-Former) with relation-aware sparse routing plus a Hierarchical Conditional Loss (HC-Loss) for coarse-to-fine supervision. The team's routing analysis found distant relationships tend to rely on low-level homogeneous interactions while closer relationships focus on high-level heterogeneous interactions, and released the code at github.com/tw-repository/HiRe-DRR.

by read1 min views2 publishedOct 28, 2026

arXiv:2610.36249v1 Announce Type: new Abstract: Dyadic relationship recognition aims to identify the latent social relationship between two individuals from their multimodal interactions. Existing methods typically project heterogeneous cues into a shared representation and optimise a flat classification objective, overlooking that relationships may rely on different interaction patterns. For instance, distant relationships can often be inferred from direct behavioural correspondence, whereas close relationships are better reflected by coordinated affective, behavioural, and multimodal responses. To address this limitation, we propose HiRe-DRR, a Hierarchical Multimodal Interaction Reasoning framework that organises relationship evidence from three complementary perspectives: feature level, interaction type, and relationship hierarchy. HiRe-DRR decomposes facial, bodily, acoustic, and textual cues into low- and high-level representations and constructs homogeneous and heterogeneous cross-participant interactions. We further introduce the Mixture-of-Interaction Transformer (MoI-Former), which treats diverse interactions as specialised experts and employs relation-aware sparse routing to identify relationship-relevant evidence while preserving contextual interaction information. The proposed Hierarchical Conditional Loss (HC-Loss) exploits the hierarchical nature of social relationships through coarse-to-fine supervision, enabling structured relationship reasoning. Extensive experiments on three dyadic relationship benchmarks demonstrate that HiRe-DRR outperforms recent representative image-, video-, and multimodal methods. Our routing analysis further reveals that distant relationships tend to rely on low-level homogeneous interactions, whereas closer relationships focus on high-level heterogeneous interactions. Our code is publicly available at: https://github.com/tw-repository/HiRe-DRR.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @hire-drr 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/hierarchical-multimo…] indexed:0 read:1min 2026-10-28 · —