Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Saravalle et al.
To enable real-time tactile-reactive robot hand manipulation policies, we propose a large-scale tactile dataset and a variable-rate Mixture-of-Transformers architecture, overcoming limitations of existing VLA models and achieving over 30% performance improvement on 12 tasks.
For dexterous robotic manipulation, the ability to dynamically react to tactile signals is essential, but existing VLA models overlook the tactile modality or rely on static cues due to scarcity of tactile data, lack of standardized evaluation, architectural constraints in current VLA models, and limitations of static tactile encoders.
1) We built a 100-hour tactile-rich dataset via a data-efficient collection recipe that prioritizes elementary motor primitives. 2) To effectively exploit high-frequency tactile signals without sacrificing existing VLA capabilities, we introduced a novel temporal tactile VQ-VAE encoder and a variable-rate Mixture-of-Transformers (MoT) architecture.
On 12 manipulation tasks requiring delicate force control and deformable object manipulation, our method achieved over 30% higher average success rate than the strongest baseline. This sets a new benchmark for tactile-reactive manipulation and represents a significant advance in robotic dexterity research.