Jixuan Chen, Jianzhi Shen, Haoqiang Kang, Zhi Hong, Qingyi Jiang, Soham Bose, Yiming Zhang, Leon Leng et al.
AgentSpec is a framework that separates modules (perception, memory, reasoning, reflection, action, learning) of LLM agent scaffolds with standardized interfaces, enabling assembly, swapping, and controlled experiments.
Current LLM agents are implemented as tightly coupled pipelines of various modules, making it difficult to isolate the contribution of each component or compare alternative designs. There is a lack of systematic methods to analyze how module interactions affect agent behavior.
AgentSpec standardizes input/output interfaces of each module and introduces a type system to verify compatibility between modules. Experiments are conducted across diverse environments (DeliveryBench, ALFRED, MiniGrid, RoboTHOR) by swapping reasoning, memory, reflection, and reinforcement learning modules with different backbone models.
Agent performance is governed by module compatibility and interaction effects rather than individual module strength. Structured multi-granularity memory improves long-horizon state tracking; reasoning and memory interact non-uniformly across environments; reflection shows a trade-off between correction and cost; RL-trained policies perform best when optimized with deployment-time scaffold structure.