Why is it that every time I try to insert a USB plug it’s backward? Shouldn’t it be right at least half the time by dumb luck? Whatever my problem is, a dextrous new robot doesn’t have it. The robot’s advantage is that its fingertips don’t just feel—they see, too.
Researchers at Northeastern University and (where else?) MIT created the plug-savvy bot. They started with an existing factory-worker robot called Baxter and gave it a pair of pinching fingers. Then on one finger, they added a shrunken-down and adapted version of a sensing technology called GelSight, invented a few years ago.
GelSight creates a precise sense of touch by, essentially, combining it with vision. It uses a small box (below) surrounding a rubber surface that’s coated in metallic paint. A different color of light shines across the surface from each of the box’s four sides: red, green, blue, and white. The rubber acts like the pad of the robot’s finger. As it presses against an object, a camera looking down from the top of the box observes how the four colors of light are reflected by the bending of the metallic pad. Then an algorithm uses that information to create a three-dimensional map of the object’s surface.
“Basically we convert touch information to images,” says lead author Rui Li, a PhD student at MIT. This gives the device an advantage over other kinds of sensors, because it can have a very high resolution—as high as the resolution of the camera built into it.
To test how well GelSight works as a robot’s fingertip, Li and his coauthors asked the newly outfitted Baxter bot to pick up a USB cable dangling from a hook and plug it in. “USB insertion is a very fine manipulation that requires sub-millimeter accuracy, which can be very challenging,” Li says.
Each time the robot picked up the USB cable, it had to feel for the embossed USB symbol on the plug, then use that information to align the plug precisely over the hole. (To treat their sensor gently, the team didn’t make Baxter push the plug all the way down—partway in was good enough.) After the researchers trained the robot, it succeeded on 34 out of 36 attempts to plug in the USB.