On November 10, the same day Claude Fable 5.2 was released, Ant Lingbo Technology open sourced the basic tactile model LingBot-Touch. A general large model for the digital world and an embodied basic model for closed-loop physical operations came into public view on the same day.

The relationship between these two technical routes is the focus of current discussions in the embodied intelligence industry. In Ant Lingbo's view, the general large model and the embodied basic model are closer to a 1+1>2 synergistic relationship: the former enhances the understanding and reasoning of tasks and environments, and the latter complements physical perception and closed-loop operation capabilities. Only when the two go hand in hand can they jointly accelerate the maturity of the embodied industry.

The open source LingBot-Touch is based on nearly 5,000 hours of real tactile data pre-training and covers three mainstream sensing principles: magnetic tactile, visual tactile and piezoresistive. It can convert different signals into the same set of tactile representations for use by different dexterous hands and action models, allowing the robot to "see clearly and think clearly" while further complementing the tactile perception required for fine operations.

Previously, Ant Lingbo's universal brain has covered high-precision spatial perception, highly dynamic environment interaction and generalizable dexterous operations. This time, LingBot-Touch has been connected to LingBot-VLA, extending the sensing capabilities to the "fingertips", making the robot "touch more sensitive", and the hand-eye coordination ability of the native universal brain has been further improved.

26 real machine tests, mission success rate increased by 14 to 26 percentage points

As general-purpose large models continue to advance, robots' ability to understand complex tasks and reason about operation steps is increasing. But in the real physical world, every time a robot performs an action, it must sense changes in objects and the environment, and then correct the next action. In many delicate operations, the robot can clearly see where the object is and can reach its hand to the correct position, but it may not be able to successfully complete the task. After the finger touches the object, it is difficult to judge whether the force is appropriate and whether the object is slipping. It is difficult to judge by vision alone and requires touch to fill in the position.

Taking the dexterous hand-tightening of a bottle cap as an example, the depth information can help the finger to align the bottle cap; after starting to turn, the bottle cap may have become loose, or the fingertip may be slipping along the cap wall. The two situations are similar in the picture, but require different movement adjustments. The contact position of the fingertips, the magnitude of the force and its changes are often obscured by fingers and objects, making it difficult to obtain vision, but they are important basis for judgment of action models.

In order to verify the help of tactile representation in real operations, Ant Lingbo conducted 26 real machine tests on two dexterous hands, Sharpa Wave and XHand1, covering fine operations such as turning book pages, twisting bottle caps, and identifying mahjong tiles. After accessing LingBot-Touch, the average task success rate of the four action models of LingBot-VLA, π0.5, Diffusion Policy and T-Rex is increased by about 14 to 26 percentage points compared with that without touch. Taking T-Rex to control Sharpa Wave to twist bottle caps as an example, the task success rate is 56.25% without using tactile sensation, and the original signal of the sensor is directly input to 62.5%. After connecting to LingBot-Touch tactile representation, the success rate reaches 90%.

(Caption: After accessing LingBot-Touch, the success rates of the four action models on the two dexterous hands have reached their highest levels)

Unify tactile representation and reuse "feel" across hardware, models, and tasks

Installing a tactile sensor on a robot does not mean that it can make good use of its sense of touch. At present, commercial tactile sensors are still in an early stage of development as a whole. Mainstream routes such as opto-tactile, magnetic tactile, and piezoresistive have their own strengths and limitations. For different costs and usage scenarios, multiple technical routes may coexist for a long time. What’s even more troublesome is that the data collected by these sensors is difficult to directly generalize: the same contact will form different original signals under different sensing principles and contact layouts. Once the model only learns to interpret one of them, after replacing the sensor or dexterous hand, the existing data and tactile experience may be difficult to reuse, requiring re-acquisition, training and adaptation.

Therefore, the key to large-scale implementation of haptics is to free tactile understanding from dependence on physical signals from specific sensors, so that data from different sources can be jointly used for training, and the accumulated tactile capabilities can be reused across sensors, dexterous hands, models, and tasks.

To this end, LingBot-Touch sets up coding entrances according to the sensing principle, first processes different original signals, and then maps the contacts of different layouts to a unified fingertip coordinate space. The shared model learns the contact position, force and their changes over time, and outputs a unified tactile token. As a result, data collected by people wearing tactile gloves and teleoperated dexterous hands can be used for training together, and different action models can also directly call trained models to extract contact features. This architecture also leaves room for expansion in the future to access new tactile sensors.

(Caption: LingBot-Touch encodes magnetic tactile, visual tactile, and piezoresistive tactile signals into a unified tactile Token for use by different action models)

On XHand1, the team further verified the reuse of tactile capabilities across hardware: the magnetic tactile sensor model used by XHand1 is not in the pre-training data, and the magnetic tactile experience learned by the model all comes from data collected by people wearing gloves. After switching to this dexterous hand, the team kept the LingBot-Touch parameters unchanged and only used XHand's operation demonstration training action model to complete the migration. At the same time, LingBot-Touch is connected to four different action models in this real machine test and is not bound to the Ant Lingbo self-developed model. It provides a universal perception capability between the tactile sensor and the action model, rather than a dedicated module for a certain hardware or a certain strategy. For the industry, sensor and dexterous hand manufacturers can select hardware based on cost and task requirements, and the entire machine team can also retain its own action strategy, thereby reducing the repeated investment in re-developing tactile capabilities every time a hardware and algorithm combination is replaced.

In order to take into account both perception capabilities and end-side computing power, Ant Lingbo provides models with two capacities: The 26 real machine tests were all completed using the lightweight version of LingBot-Touch-Tiny with approximately 5.3 million parameters. After accessing LingBot-VLA, π0.5 and T-Rex, the number of model parameters only increased by approximately 0.13% to 0.16%; the larger-capacity LingBot-Touch-Small It shows the potential for further improvement in the perceptual test - when the pre-training data increases to nearly 5,000 hours, the model can still benefit from the new data, indicating that there is room for continued expansion and improvement in tactile pre-training.

Adhere to embodied nativeness and advance in coordination with general large models

Fable 5.2 was unveiled on the same day as LingBot-Touch, which to some extent is a microcosm of the evolution of the current embodied intelligence technology route. Regarding the impact of the accelerated evolution of general large models on the robot’s “brain” route, Zhu Xing, CEO of Ant Lingbo, said: “With Claude Fable 5.2 and GPT-6 Astra The advancement of the digital world's large language model has indeed made robots more capable of complex task reasoning and physical operations. We are very concerned about this and are actively absorbing it. However, the practice in the industry tells us that to truly break through the closed loop of operations, the hard nut of physical operation generalization still cannot be bypassed, and it still depends on the breakthrough of the embodied basic model itself. "

In his opinion, the general large model and the embodied basic model are closer to the synergistic relationship of 1+1>2: the former enhances the understanding and reasoning of tasks and environments, and the latter improves physical perception and closed-loop operation capabilities. The two break through each other and promote each other, jointly accelerating the maturity of the embodied intelligence industry.

"The quality of modal input largely determines the quality of action output. In other words, without good input, there will be no good output." Zhu Xing said that Ant Lingbo will adhere to the embodied native route, continue to explore new upper limits for the generalization of physical operations, create a cross-configuration universal brain for the robot industry, promote the alignment of more physical modalities such as vision, depth, touch and even temperature in a unified model, and continue to open source to the community while promoting commercialization.

Currently, LingBot-Touch’s technical reports, models, and code are open source, and developers can access and download them on Hugging Face, GitHub, and the Magic Community.

Leifeng.com Leifeng.com (public account: Leifeng.com)

edit

Resend Email Modify Email