In October 2019, OpenAI wowed the world when it released a video of a robotic hand manipulating a Rubik’s Cube, solving the puzzle in under four minutes, as the machine’s fingers rapidly and dexterously twisted and turned the plastic block.
But the experiment, run by one of the world’s best-funded AI labs, wouldn’t have been possible without technology from a small UK company that began life as a ramshackle group of hobbyist robot makers in the 1980s.
“It was pretty jaw-dropping,” said Rich Walker, director of Shadow Robot, the company that sold the robot hand to OpenAI. “It was one of those points where you just think, ‘OK, the state of the world has changed in some way. Different things are now possible.’”
Shadow Robot sold its first robotic hand to researchers at Bielefeld University in the early 2000s. Nearly two decades later, OpenAI showed that algorithms trained on some 13,000 years of simulated practice were starting to make fine motor manipulation possible. And, as huge sums of money go into developing robots designed to operate in homes, warehouses, hospitals and labs, Walker believes those machines will need touch to do work that demands dexterity.
“Take anything that comes in plastic packaging. That requires opening a fiddly bag: you have to be able to tear it open, take things out correctly and put it in something else. That’s something that’s currently bottlenecked by humans doing it,” he said. “If robots don’t have manipulation skills, they’re going to have real trouble getting a lot of things done.”
Shadow Robot has been building artificial hands for decades, but now startups such as Touchlab and Adexis are developing technologies ranging from highly sensitive electronic skin to new world models built specifically to crack the elusive and somewhat abstract problem of robotic touch.
The task is a complex one. Robotic hands have to process streams of data from sensors that interact with multimodal models, built on scarce training data, while scientists are still working on defining the concept of “dexterity” in a way that can be coded into math.
The data and definition gap
One reason robotic touch has lagged behind machine vision is data. Where huge quantities of video are available on the internet to train vision models, far less data exists for touch.
“Vision had the advantage of YouTube. The scale of YouTube accumulating video is absolutely incredible, so that’s a huge amount of resource to work from,” said Walker.
“Manipulation is about ‘contact interaction’; you’ve got to be touching something and moving it and understand why you’re doing that. There are people who are letting us record manipulation data at scale to try to label it and categorize it, but if you think about contact and touch, we don’t even have very good words to describe that.”
A collage of Shadow Robot’s early hobbyist team, with Rich Walker pictured [R]. Credit: Shadow Robot
What’s hard to describe is even harder to code. “Touch is still relatively poorly understood. There’s no engineering science of touch that lets you say, ‘If I want to handle an object like this, I need sensing like that to be able to do it well,’” said Walker. “When we talk to people about dexterity, it’s one of the challenges we get into. No one can really define dexterity very well.”
Enter ‘proprioception’
But none of this has stopped startups from trying. Edinburgh-based Touchlab, founded in 2018, is developing algorithms built around “proprioception,” the largely unconscious sense of where our body parts are in space, sometimes called our sixth sense.
“Vision and depth sensing are essential for navigating and approaching an object,” Touchlab CEO and founder Zaki Hussein told The Infinite Loop. “These sensors share one basic physical limitation, though: occlusion. At the exact moment a robot’s hand grasps an object, the hand itself blocks the camera’s view.”
“Touchlab’s technology picks up right where vision goes blind. Proprioception, the robot’s awareness of where its fingers and hand are in space, is just as important: it anchors each touch to a precise location on the object,” he said.
On the hardware side, Touchlab has developed proprietary robotic “skin,” a material covered in sensors that can read the position, pressure and 3D direction of whatever its robotic hands are holding.
Touchlab’s e-skin, fitted to tactile “fingertips” on a grabber. Credit: Touchlab
Hussein said this helps give the robot a dynamic grip that prevents the object from slipping or being crushed, via measuring “spatial pressure distribution directly at multiple points of contact with high temporal fidelity.”
A robotic inflection point
Combining these technologies in one system is no mean feat, but Hussein said that as humanoid robots increasingly operate in spaces designed for humans, more dexterous machines will be a must.
“Historically, robots operated in highly structured environments like automotive assembly lines, where touch wasn’t strictly necessary. Today, the rapid acceleration of humanoid and general-purpose robots is changing the paradigm, as these platforms move into unstructured environments,” he told The Infinite Loop.
“Touch enables true dexterity, allowing robots to adjust their grip dynamically to prevent slip, modulate force to avoid crushing delicate items, and operate safely in unstructured, human-centric environments.”
This industry pressure to bring robots into more environments built for humans is beginning to usher in a new wave of startups. London-based Adexis is developing what it calls the “somatosensory cortex for physical AI” — a world model that will help robots perceive the sense of touch — and has raised funding from high-profile investors including Speedinvest and ElevenLabs.
Shadow Robot, meanwhile, has kept building for researchers. In May 2024, it unveiled its three-fingered “DEX-EE” hand for advanced AI research developed in collaboration with Google DeepMind.
The DEX-EE hand, developed by Shadow Robot and Google DeepMind. Credit: Shadow Robot
“We live in this slightly weird world where most of our customers have been people called ‘Professor’ who are doing state-of-the-art research in robotics, but that’s now shifting,” Walker said.
“We’re looking at some problems in manufacturing and some problems in science, where essentially there are lots of things being done by people that are pretty boring, repetitive tasks. We’re trying to work out how we can build the right robot hand to do those.”
This momentum, Walker believes, is a natural progression for physical AI. Touch might be one of the toughest senses to instill in robots, but it is also the first sense humans develop.
“There’s research in developmental psychology and early-stage development of neurology that suggests that being able to touch is how you learn to move in the womb,” Walker said. “That sort of contact happening is pretty fundamental to our cognitive development.”
