Zhenghao Xing, Ruiyang Xu, Yuxuan Wang, Jinzheng He, Ziyang Ma, Qize Yang, Yunfei Chu, Jin Xu et al.
To address the inefficiency of traditional passive paradigms that uniformly process all frames for long video understanding, this paper proposes OmniAgent, an omni-modal agent that selectively distills audio-visual cues.
Existing long video understanding models rely on a 'watch-it-all' paradigm, causing computational costs to grow proportionally with video duration. Interactive frameworks often rely on global pre-scanning, and their context cost still scales with video length.
OmniAgent formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. It executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. It introduces Agentic Supervised Fine-Tuning and Agentic Reinforcement Learning with TAURA.
OmniAgent achieves state-of-the-art performance across ten benchmarks. Notably, its 7B agent outperforms the 10x larger Qwen2.5-VL-72B on LVBench. It exhibits positive test-time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception.