{"slug": "iflytek-embodied-omni-technical-report", "title": "iFLYTEK-Embodied-Omni Technical Report", "summary": "IFLYTEK-Embodied-Omni, a unified multimodal foundation model, jointly models vision, language, and action within a single framework to enable general-purpose embodied agents to understand instructions, predict environment evolution, and produce precise control actions. The model uses a brain-cerebellum architecture with shared multimodal self-attention, combining vision-language, video-generation, and action-generation components. A four-stage training strategy leverages action-annotated and action-free embodied videos, human demonstrations, and robot interactions to develop these capabilities.", "body_md": "arXiv:2607.02542v1 Announce Type: new\nAbstract: General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons. Existing approaches typically specialize in visual-language reasoning, video-based world modeling, or action generation, while cascaded pipelines that first synthesize future observations and then infer actions can introduce interface bottlenecks and compound prediction errors. We present iFLYTEK-Embodied-Omni, a unified multimodal foundation model that jointly models vision(videos and images), language, and action within a single Omni framework. Its modality-specific visual-language, video-generation, and action-generation components communicate through shared multimodal self-attention. This design establishes brain-cerebellum collaboration: the vision-language modeland video generation model form a high-level brain for instruction understanding, task planning, progress tracking, and future visual-state prediction, whereas the action generation modelserves as a low-level cerebellum that directly converts planned subgoals and shared multimodal context into executable action chunks. To develop these capabilities, we combine action-annotated and action-free embodied videos from human demonstrations and robot interactions with embodied reasoning, embodied perception, and general-purpose image-text data to construct a comprehensive dataset. We further adopt a four-stage strategy that progressively trains the VLM, VGM, and AGM before jointly fine-tuning the complete model.", "url": "https://wpnews.pro/news/iflytek-embodied-omni-technical-report", "canonical_source": "https://arxiv.org/abs/2607.02542", "published_at": "2026-07-07 04:00:00+00:00", "updated_at": "2026-07-07 04:02:55.875338+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "robotics", "computer-vision"], "entities": ["iFLYTEK", "iFLYTEK-Embodied-Omni", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/iflytek-embodied-omni-technical-report", "markdown": "https://wpnews.pro/news/iflytek-embodied-omni-technical-report.md", "text": "https://wpnews.pro/news/iflytek-embodied-omni-technical-report.txt", "jsonld": "https://wpnews.pro/news/iflytek-embodied-omni-technical-report.jsonld"}}