Just-In-Time Scene Graph Growth: Combating Perceptual Saturation in Long-Horizon Robotics

Researchers introduced JITOMA, a just-in-time framework that reduces perceptual saturation in long-horizon robotics by using a task heatmap and LLM-driven memory activation to grow 3D scene graphs on demand, cutting active graph size and captioning latency while maintaining stable processing under task switching.

arXiv:2607.13245v1 Announce Type: new Abstract: While 3D Scene Graphs 3DSGs provide crucial structured representations for embodied agents, conventional Ahead-of-Time, build-everything-then-filter pipelines conflict with the real-time, low-latency demands of edge platforms, inducing a perceptual saturation effect via severe observation redundancy. To resolve this, we present JITOMA Just-In-Time On-demand Memory Activation , a closed-loop framework that unifies task reasoning, perception, and memory into a just-in-time growth process. Instead of exhaustively mapping the entire environment, JITOMA leverages a top-down task heatmap at the frontend to filter continuous observations, routing minimal streams to maintain a global foundation of low-cost, dormant anchors. Upon a cognitive query, the backend Large Language Model LLM parses the robotic intent to dynamically awaken task-relevant anchors, triggering resource-intensive operations -- such as dense node captioning and functional inference -- exclusively within the activated local subgraph. To evaluate these dynamic capabilities and study perceptual saturation trade-offs, we introduce JITOMA-Bench, a comprehensive suite for long-horizon multi-tasking and complex multi-step reasoning. Extensive experiments demonstrate that JITOMA substantially reduces active graph size and captioning latency, while maintaining stable processing time under long-horizon task switching.