Hands and brains are two of the most complex and hard-to-duplicate attributes of humans. They are part of what distinguish people from robots. But, a new R&D initiative hopes to change that by improving the dexterity of machines and enhancing human-robot interaction.

The goal of the Human AugmentatioN via Dexterity (HAND) project is to develop robots capable of intelligent and versatile grasping, fine motor skills and hand-eye coordination. The multi-institutional collaboration, led by Northwestern University, recently received $26 million from the National Science Foundation to launch an Engineering Research Center (ERC) dedicated to studying robotic manipulation in applications ranging from manufacturing to caregiving.

In addition to Northwestern, several other schools are involved in the 10-year project, which comprises nearly 30 faculty members with a variety of expertise. Key academic partners include Carnegie Mellon, Florida A&M and Texas A&M, with additional faculty support from the Massachusetts Institute of Technology (MIT), Syracuse University and the University of Wisconsin-Madison.

The ERC, housed under the auspices of Northwestern’s McCormick School of Engineering, aims to build a variety of cutting-edge hardware and software tools that are versatile, robust and easy to integrate with existing technology. The interdisciplinary center involves experts in fields such as artificial intelligence, haptics, human-robot interaction, machine perception, manipulation, materials, modeling and soft robotics.

“This is a historic milestone that builds on [our] well-recognized expertise in robotics and human-machine systems,” says Eric Perreault, Ph.D., vice president for research at Northwestern. “The HAND [project] is bold and visionary. It will have a long-lasting, positive effect on manufacturing, food processing, healthcare and many other areas that rely on dexterous manipulation.”

“The focus of our engineering research center is on making robots easier to use,” adds Kevin Lynch, Ph.D., a professor of mechanical engineering and director of Northwestern’s Center for Robotics and Biosystems.

“Traditionally, most research activity has focused on robotic locomotion and navigation, and less on manipulation,” explains Lynch, who serves as HAND’s research director. “Our focus is on end effectors. We want to give robotic arms humanlike dexterity.”

While robots already play an important role in manufacturing, Lynch says their full potential has been limited. Developing hands that are as versatile and dexterous as their human counterparts will enable robots to adopt new grasping capabilities and allow smaller manufacturers to automate a variety of low-volume, high-mix production tasks.

“Parallel and suction grippers have limitations,” claims Lynch. “Human hands are capable of so much more, because they have many degrees of motion and have numerous tactile sensors embedded in their fingertips. They can perform complex things such as using chopsticks, shuffling cards, screwing a cap onto a jar or writing with a pen.

“All those tasks require rolling at the fingertips, which we refer to as in-hand manipulation,” says Lynch. “Most things that humans do involve some kind of motion of the object relative to the hand. The sensing and control of our hands also makes up a huge portion of the human brain’s volume.

“However, when it comes to robotics, we are very far away from achieving that same sort of sensory motor integration,” Lynch points out. “That means all the intelligence we see in modern AI, such as ChatGPT, can’t be expressed through physical actions and manipulation. The sensing and control of hands is a major bottleneck toward unleashing the power of AI.”

Lynch and his colleagues plan to address robotic dexterity challenges through multiple thrusts. For instance, they’re developing new robotic hands with new actuation technologies and tactile sensors. They are also working on artificial intelligence and human interfaces related to robotic dexterity and manipulation.

“We currently use off-the-shelf components in our lab, including commercial robot arms, cameras and sensors,” says Lynch. “We’re using virtual reality headsets and haptic gloves to teleoperate robots through that interface. But, over the next few years, we will be replacing those components with new hands and interfaces that we are currently developing.”

HAND is focusing on three key areas: hands, intelligence and human interface. Engineers at Northwestern are tackling AI and intelligence dexterity, while their counterparts at Carnegie Mellon address hand hardware, including design, actuation and sensing. MIT engineers are in charge of human interface issues, such as how to make robots easier to use and perform useful tasks.

“Our long-term goal is to have a commercially available dexterity system that comes with hands and a human interface that can be easily connected to any standard industrial robot,” says Lynch. “You’ll be able to attach our hands to any robot arm.

“The devices will be able to perform dexterous skills straight out of the box instead of requiring months of integration time,” explains Lynch. “Assembly applications will include adhesive dispensing, screwdriving and soldering, with minimal initial programming effort required.”

The technology is being tested at several test beds. One is associated with the Shirley Ryan AbilityLab in Chicago, which is focused on helping people with motor disabilities so that they can be more independent at home. Test beds at Carnegie Mellon and MIT are focused on manufacturing applications. Another facility at Texas A&M is focused on handling rare or high consequence materials, such as biological agents.

“There are many areas that we still need to have significant advances in to achieve the vision of the center,” says Lynch. “One thing we have to address is improved actuation so that robotic hands are not so brittle and heavy. Reducing weight while increasing strength and dexterity are actuator problems that we need to address.

“Another challenge we face is making skin soft and durable,” adds Lynch. “We need to develop multifunctional materials that provide both actuation and sensing capabilities, while also providing a soft surface for interaction. We’ll be investigating technologies such as electrostatic clutches and liquid crystal elastomers. Then, once we address all those issues related to hands, we have to connect the intelligence.”