For Google DeepMind’s latest robotics model, switching bodies may be easier than sweeping a countertop. The company says Gemini Robotics 2 can control several robot configurations, bringing it closer to a single AI that works across different machines.  

Yet in its published tests, one humanoid completed a countertop sweeping task just 32% of the time, a reminder of the gap between adapting to a body and using it reliably. The result offers a useful reality check alongside the promise of a model that can move between machines. 

Shown in late July, Gemini Robotics 2 controls a robot’s legs, torso, arms and fingers, while its predecessor largely operated humanoid robots from the waist up. 

DeepMind reports that the same version of the model ran three configurations, namely Apollo 2 fitted with SharpaWave hands, Apollo 2 with Inspire hands, and Franka Duo with a Robotiq gripper. Adapting it to a new body takes just a few hours of data, according to the company. 

But there is something missing from that versatility. The model does not receive a sense of touch. 

A hand controlled by Gemini can have 22 degrees of freedom, giving it considerable flexibility, without feeding the model to tactile information about what it is holding. Take, for example, picking up a grape has no touch signal warning that its skin is about to split. 

One challenge is finding enough data to teach that skill. The internet has given AI a lot of data to learn what things look like, but it does not offer much about how they feel. 

As per Matei Ciocarlie, a Columbia University roboticist and co-founder of the robot-hand company Tangent Robotics in his interview with Scientific American, he states that there is “essentially no tactile data or force data” online. 

DeepMind’s own results mirror the challenge. Tasks requiring delicate, coordinated finger movements were its weakest category, trailing whole-body movement and tasks using grippers. 

Even judging how a task is going remains difficult. The company’s separate reasoning model, Gemini Robotics ER 2, scored 57.4% on a test of assessing task progress. DeepMind says that was better than the other models it tested against, though it leaves considerable room for improvement. 

As per Carolina Parada, DeepMind’s vice president and head of robotics, as reported by Scientific American, faster feedback remains an ongoing focus. Lead software engineer Kanishka Rao explained the broader approach as a bet that enough experience interacting with the world will produce more general abilities, much as large amounts of training data helped advance language models. 

There are reasons to think the approach could work, but switching bodies adds another challenge. 

James Marshall, director of the Center for Machine Intelligence at the University of Sheffield, said a general model capable of controlling different body types is plausible, pointing to human adaptability. Moving between a four-legged robot, a humanoid and a drone, however, would demand much more data. Researchers still do not fully understand how brains solve such problems. 

For now, access is limited. Gemini Robotics ER 2 is available through Google AI Studio and in private preview on the Gemini Enterprise Agent Platform. The model that controls physical actions and its on-device version remain restricted to early-access partners. DeepMind has not announced when either will become more widely available.