Jonathan Cheng, MD, FACS, Associate Professor of Plastic Surgery, Director, Nerve Lab, Chief of Pediatric Hand, Peripheral Nerve, and Microvascular Surgery at UT Southwestern Medical Center talks about robotics and amputees.
Interview conducted by Ivanhoe Broadcast News in May 2018.
We’re looking at, I think, some really remarkable developments in the interface between robotics and amputees. Could you talk a little bit about what you’re finding?
Dr. Cheng: That’s exactly right. The robotic hand systems that have been available, the really state of the art technology, has been around for a few years now. But what we’ve found over time is that the ability to connect those to the patient, to the human who is using it is actually still lagging behind. And so we’re part of a major effort that the Department of Defense, DARPA which is the Defense Advanced Research Projects Agency (the space age research arm of the Department of Defense) is aggressively working to close that gap so we’re really able to use all of the functions that are available in the robotic hands that are available.
What kind of progress are you making?
Dr. Cheng: We’ve made a huge amount of progress. We have a very, very specific approach which is to place electrical interfaces inside the residual nerves that are still left behind in the residual limb after someone has an amputation. Those electrodes, because of where we place them and how we place them, and how the specific electrodes interact with the nerve that they’re implanted within, allow us to be able to listen to the nerves and to read out commands for controlling the individual fingers of the robotic hand by decoding the signals that are still inside the nerve, and talking to the nerves by stimulating them electrically in order to give various different sensations. A sensation of touch, the sensation of movement, even going so far as to provide an improved sense of where the hand is in space. If you were to close your hand and try to touch the tips of your fingers together or touch your nose, that’s the kind of sensation that we think that we’re getting to have the ability to impact. So that when somebody has an amputation and uses the robotic hand, it’s much, much more lifelike function than what’s available currently.
Currently what’s available is three different motions one at a time, no sensation. And what we are trying to do is: to take patients who have existing robotic hands, retrofit them to give them the ability to feel the sense of touch and the sense of motion; to be able to use those to improve their sense of where their hand is in space; and then even to read out from the nerves, which is a very unique thing that very few groups have really had much success on. On a long-term basis, being able to record from the nerves and use those nerve signals in order to control the fingers of the robotic hand. If you were to imagine what that might allow you to do, it would be things like playing the piano or typing on a keyboard. In order to have that dexterity, you have to be able to control all the fingers individually and know what they’re doing and feel them.
We’re talking about sensory restoration similar to the electrical devices that currently exist for patients who are born with deafness, who have cochlear implants placed in order to restore their ability to hear (which has been FDA-approved for twenty or thirty years). And retinal implants, which are newer, which are for patients with retinal blindness that stimulate their nervous system and allow them to see. And so really we view our effort as another sensory restoration option for those who have lost the ability to feel because they’ve lost their hand. And so when they get a robotic hand, they replace their physical hand and it moves similarly, but there’s no feeling. It really impacts their ability to interact with the world because the hands are kind of the leading edge of our interface with the world.
So it sounds like sort of the most generalized way of saying it is you’re creating machines that can feel?
Dr. Cheng: Machines that are extensions of the human body. As we move forward, the goal is for the robotic hand that replaces the natural native hand to be less like using a tool or an appliance, and to be more like replacing part of your body with something that does the same things. And we’ll get closer and closer to that over time as the technology improves.
What are you finding from the feedback that you’re getting from the research project? Are some of your patients, some of the amputees, are they feeling things that they haven’t felt before?
Dr. Cheng: Sure. The patients that have participated in our study so far have only been for short periods of time (several months), and we’ve recently gotten approval to extend those periods of time so that we can see how the implants that they have continue to work over time. They’ve been feeling things, even from the first week after they received the implants. They have an implantation surgery to place the electrical connections into their nerves, and then within the first week they come back to the lab for an experimental session. We turn on the stimulation, plug them in to the electrical systems that we have and they’re able to feel the tips of their fingers or the palm of their hand. One of our patients felt like her thumb and her index finger were coming together like this (demonstrates pinching the thumb and index finger together) in order to pinch something. And that’s a sensation of movement, and all we would have to do is add in the sensation of touch once they meet, to be a totally normal sensation of your fingers closing around something.
You’ve seen a lot of this in action in real life happening is it like a miracle?
Dr. Cheng: I wouldn’t say that, it’s something that we’ve been working on for a long time. And as part of this effort our team, which consists of numerous individuals each with their own individual expertise because obviously it takes a lot of people, with a lot of dedication in different areas of expertise, to make something like this happen. We talk a lot about what we know about the science, what we don’t know, how we’re going to meet those challenges and what we think that the solutions are. There’s always a prediction about what’s going to happen, and we’re always very excited to see when those predictions come true. If we find something different and new, we work immediately to try to figure out what’s going on and what that means about how the nervous system is wired up.
What about the person who is experiencing this?
Dr. Cheng: The person who is experiencing it, it’s like restoring part of their own body. I think that for the patients that we have… you know, we have had folks who have had their hands amputated for ten years, twenty years in some cases …and it’s really an eye opener that those nerves are still working in the same ways and still carry the same signals after that amount of time. I wouldn’t say that we’re giving them some kind of miraculous sensation, just trying to restore them back to normal and ultimately that’s the goal.
If you think this is a medical breakthrough or leading to a medical breakthrough would you tell me what that is and tell me why.
Dr. Cheng: It really is a big deal because I think that being able to restore a major sense….we have five senses and arguably the ones that allow us to function in the world and be part of society, take care of our family, make relationships, do our jobs…. Those are sight, hearing and touch. Being able to restore the sense of touch, I think, is a really, really important thing. And that’s not just feeling. That’s movement, and that’s knowing where your hands are when you’re not looking at them. That is all part of what makes us who we are, and allows us to interact with the world, and to manipulate objects and make connections and hold our partner’s hand, or our grandchild’s hand, or our children’s hands. It’s really that human connection that we’re after in restoring touch. There’s another piece beyond that, so we have the ability to restore touch for any patient with an amputation.
We’re talking about really why this is a breakthrough and what it’s doing. How exciting is it to you as a researcher and also what it’s doing for the patients.
Dr. Cheng: What we’re doing really is a breakthrough because with what we’ve learned about where to place the interfaces in the nerves, and how to use them, and what electronics to connect them to, we’ve gained the ability to talk to the nerves and provide the sense of touch, and the sense of movement and being able to know where your hands are without looking at them. And that sensation of feeling is one of the major senses that we have. You know we, have five senses: hearing and seeing are equally important or more important, but being able to use your hands and to feel with them is essential to functioning in the world. To being able to be a social person and interact with others, build relationships, hold hands with your partner. Being able to manipulate objects, interact with the world: at work, using tools, making dinner. Being able to restore that, I think, that is a major breakthrough and then we have added on top of that. So those sensations are available for anybody with an amputation who uses a prosthetic. They may have a hook or they may have a lifelike-appearing hand that doesn’t move and we can still add sensation to those types of prostheses. But adding on top of that for patients who have robotic hands, that actually move in a lifelike way… we have now gained the ability to listen to the nerves, get the control signals that control the hand that’s no longer there, that used to be there. Those signals are still available after decades with the hand not being there anymore. We can actually read those out, decode them, and use those to control the robotic hand so that each of the digits can move individually. Which really has not been available at all with any of the very sophisticated systems that are available to patients today. And to bring that to reality, those two things in combination: feeling and being able to move each of your fingers individually, gets us toward this Luke Skywalker, Star Wars ideal where you have a robotic hand that does everything your natural hand does. That’s using it with dexterity, being able to control your fingers and feel what they’re doing. Play the piano, type on a keyboard. For the servicepeople the goal is to get them back to their service, to get them back to their military occupation. So the example that they use is for an infantryman being able to disassemble and clean and reassemble his firearm.
After we’ve gotten that injury to the point where they’ve lost their hand, so they can get back to working with a rifle again?
Dr. Cheng: It would be their choice if they choose to do so. And so that part of the impetus for the project, is what we’re doing… is that there are a number of young men and women, servicepeople that have been injured in the line of duty through the conflicts in the Middle East. And the federal government, the US government has made a commitment to them to provide them lifelong restoration for their injuries. What we’re doing is furthering that goal, making it so we can restore them to something closer to the function that they had before their injury.
As this field has been developing, it’s my understanding the way it used to be they would try to control the robotic arm or hand, would be to go up into the brain and then almost work from there down, whereas apparently you’re working from the nerve ending in the hand.
Dr. Cheng: Right. That’s research also.
Is this an improvement on the old way of doing it?
Dr. Cheng: The old way of doing this is to control the robotic hand using the muscles that remain in the limb. There are contacts that are placed on the skin, and those contacts record the electrical signals that are given off by the muscle as the muscles contract. That allows the ability to control three different movements, one at a time. By placing the electrical contacts inside the nerves, right now we think that we’re able to obtain fifteen different movements, and with the right kind of control scheme, all at once. So you can do like all of us do: we close our fingers and bend our wrists at the same time, or open our fingers and straighten our wrists at the same time. Some of the other research that has preceded ours has been to place electrical implants inside the brain or on the surface of the brain, then use those signals in order to give sensation and to provide movement. But that definitely was a major advance when it occurred, an incredible amount of work and scientific understanding goes into that, but it may not be applicable for everyone.
It may not be as specific as what you’re getting, down to the individual fibers right?
Dr. Cheng: Well the brain is very dynamic and the brain is changing all the time. And that’s less true for the nerve fibers that are in the arm. So in the arm it’s more like wiring, and in the brain it’s more like a landscape that changes all the time. Because the brain is constantly responding to the environment, where the nerves in your forearm are doing the same thing every day.
So by working with the nerves in the forearm you’re getting closer to where you need to be, I guess in terms of reestablishing those contacts.
Dr. Cheng: Right. In some ways it’s more straightforward because it’s like tapping into the wiring that goes to the outlet of your house. But in some ways it’s more challenging because the signals in the brain are much higher strength than the ones in the arm, and so it actually requires very, very specialized strategies to be able to hear those and to filter out all the noise.
Do you use amplifiers?
Dr. Cheng: It’s a special kind of amplifier, traditional amplifiers just reduce the noise and our collaborator has an ingenious way of eliminating the contaminants, the extra signals, but keeping the nerve signals. And it’s really been a major advance.
What else do we need to talk about?
Dr. Cheng: So I am the Co-founder of a company called Nerves Incorporated, together with Ed Keefer who is the principal investigator on this project that you’ll meet later today. The two of us have been working together on research for almost ten years. And it’s part of my disclosure to let you know in this conversation that I have what’s a conflict of interest because of my involvement with the company. And so there are very special specific controls set in place in order to oversee my interaction with this research.
But it’s not stopping you from doing the research?
Dr. Cheng: It’s not.
So what’s the goal of the company beyond the research, or is it just to apply the research in a broader way? I guess it’s controlled more by the company.
Dr. Cheng: The goal of the company is to do the research and apply the research. And really our focus is on developing things that can help patients. All the research that is being done in healthcare and in medicine is ultimately with the goal of helping patients. But our particular approach is more direct than that. Because everything that we do really is with an eye on, “How is this going to help patients, and how many people is this going to help?” because our goal is to help as many people as possible.
So we’re going to meet Shawn and he’s one of your patients. But he didn’t get injured in a military conflict, he had an industrial accident at one time.
Dr. Chen: Right. Two of our four subjects so far have been military veterans, although neither of the veterans had their injury during their service. All four of them actually were industrial injuries. I think that, that really points to how critical the hands are in our livelihoods and our daily lives. These injuries have occurred because of the importance of the hands in daily life, restoring them will have an equal impact, I think, on their ability to be restored closer to whole.
So for Shawn it is my understanding that he had all the wiring and the electrodes, everything was in there for a period of time?
Dr. Cheng: Right.
It’s not there anymore?
Dr. Cheng: Right. What we’re doing right now is purely research. It’s to interact with the nerves in a specific way, in order to see what we can hear and what we can tell the nervous system, and how that changes over a period of time. So far our subjects have been implanted for around three months. We’ve expanded that to six months, and eventually we may get out to even a year or more as part of the research part of what we’re doing.
So how many people have gone through this so far?
Dr. Cheng: Four, we’ve had four subjects so far.
Is that like four nationally?
Dr. Cheng: There are a couple of other centers in the United States that are doing similar work, but we’re very lucky to be one of the select groups that was picked by DARPA to participate in this work.
So Shawn is one of four people to experience this high level of feeling again in his hand?
Dr. Cheng: Right. Shawn is part of a very select group that was very, very carefully screened in order to make sure that they are able to participate fully. They’re all extremely intelligent and committed individuals who are willing to give of their time and to make the commitment and the sacrifice to be part of gaining new knowledge that can help others.
It’s like you gave him something and then you had to take it away right?
Dr. Cheng: Right.
So how challenging is that?
Dr. Cheng: It’s challenging. I think that Shawn was perfectly ready and willing to keep going when the study was over. But all of our subjects so far have entered into the study knowing that it’s really a temporary process at this point. But all of them are still candidates and will be considered very highly as we move forward to something that can be applied to all patients, that can be left in for an extended period of time.
How soon do you think you will get to that point?
Dr. Cheng: Our goal is three years to be able to get to a solution that can be applied widely to a large number of patients.
Would that be three years from now or three years from when you started?
Dr. Cheng: We have the knowledge today in order to start working toward a clinical solution, something that can be a reality for a large number of patients. And if we started today it would take three years. Right now we’re in the phase of seeking additional funding in order to support that effort. Part of that is through charitable gifts. Part of that is through seeking additional federal grants. We really want to carry this forward in a way that allows us to have as much control over the process as possible, and to be able to serve as many people as possible.
Groups like what Wounded Warriors or something like that?
Dr. Cheng: Potentially Wounded Warriors and there are a number of charitable foundations, and also we’re very fortunate in our local area to have a lot of individuals. The medical center that we’re sitting in, and the hospital that we’re sitting in right now, are a testament to that, how the individuals in our community really are committed to the broader good. Also there’s federal funding which is through the NIH and through DARPA and through the US military.
So overall you must feel pretty good about the project you’re involved in and your commitment to all this.
Dr. Cheng: It’s been a real pleasure and a privilege to work on this, to be provided the opportunity to do such incredible exciting scientific work. We started all this in animals and doing it in the lab, doing research studies with the goal of helping people. We’ve gotten to the point where we can actually apply these strategies to patients, and to have them tell us directly what their experiences are: how they feel about the importance of what we’re doing, and what impact it could have on their lives. And I think when you talk to Shawn he’ll be able to give you a sense of that. We really feel strongly that being able to feel will have just an incredible number of benefits to how he goes about his life, and how he interacts using his prosthesis, and what it will mean for his daily living.
If you have an artificial hand and you have a Coke in your hand and you’re not paying attention you might just squeeze the can and crush it, is that—
Dr. Cheng: Absolutely. And I think that I’ve heard a lot of amputees that use robotic hands say that they have to be very careful when they shake someone’s hand because it’s possible to squeeze too hard. And that’s not a good way to greet your friend, or to meet somebody new. We actually do some testing in the lab that demonstrates that… because you know, my kids and their friends do cup stacking… and whenever you do cup stacking it seems like this very simple activity, but it requires squeezing the cup hard enough to pick it up, but not too hard… because if you squeeze it too hard, you pick up the entire stack of cups. When Shawn was using his current robotic hand with the addition of sensation, he was actually able to do that way, way faster and with greater facility than without sensation. And so even during that short period of time, we could see that it would be something enormously beneficial.
Since he had the benefit of three months I guess, he experienced things in those three months that he hadn’t experienced in a long time. Has he been able to retain anything from that period of time? Or is he back to three movements?
Dr. Cheng: He’s back to normal. And so I think that’s a testament to how compatible what we’re doing is with the way that the nervous system exists, because after we’re done it it’s like we were never there. So far that’s what we’ve seen in the subjects that have completed the trial.
It’s not creating any damage or anything like that?
Dr. Cheng: Well inherently there is some, but it’s something that I think is not interfering with their existing function. We’ve had two of the four subjects that had pain, that they had since the time of their injury. For a period of years… a decade, and during the time that they were involved in the trial their pain was actually better. Then after a time, once the interfaces were removed, their pain returned. It’s hard to know whether that’s because of the sensations that we were giving them. They seemed to think so, but I think that remains to be determined.
You hear about phantom pain, is that what we’re talking about here to some degree?
Dr. Cheng: It’s hard to say. There are lots of different kinds of nerve pain, phantom pain being one of them. But a lot of these patients still have injuries in the nerves that are remaining from the time of their injury. And so that can also be painful. A lot of the circumstantial evidence that we have, from taking care of patients with nerve injuries, seems to suggest that giving them back the sensation of movement can help to change their perception of phantom pain. But right now, with the particular interfaces that we’re using, it’s hard to know because patients come in once a week and we do the stimulation, but it’s for a very short period of time. So it’s not that they’re going home and experiencing the sensation of moment every day for three months, which we may ultimately find gives them an improvement in their phantom pain.
Anything else that you were thinking of saying?
Dr. Cheng: I think that something that always bothered me, is that in the previous conversations on this particular topic I don’t think we’ve emphasized enough that it takes a huge team of dedicated individuals in order to do this. You know, we live in a day and in a time that nobody can do something like this on their own. In our group, we have in some ways one of each. So we have me, who is a surgeon and I also do laboratory work on nerve injury and nerve repair, basic research. My collaborator (Edward Keefer, PhD), who is a neuroscientist with expertise in electrophysiology, which is basically talking to and listening to the nervous system. Cynthia Overstreet is a sensory neurophysiologist who understands how the brain perceives sensation, how to code the electrical signals to give specific kinds of sensation, how to test whether the sensations are the same each time, where they’re located. Zhi Yang is an electrical engineer and a bioengineer at University of Minnesota. He’s an electrical engineering expert and a signal processing expert, and has absolutely groundbreaking electronics that he’s created that allow us to listen to the nerves. A fellow that we’re collaborating with named Dan Merrill is somebody who has participated in lots of different projects to bring electrical interfaces for human use, for health reasons… to clinical reality. So it really takes all these various parts in order to bring something like this and make it so lots of patients can benefit from it.
It’s kind of like an electrician.
Dr. Cheng: Right. Living wires.
END OF INTERVIEW
This information is intended for additional research purposes only. It is not to be used as a prescription or advice from Ivanhoe Broadcast News, Inc. or any medical professional interviewed. Ivanhoe Broadcast News, Inc. assumes no responsibility for the depth or accuracy of physician statements. Procedures or medicines apply to different people and medical factors; always consult your physician on medical matters.
If you would like more information, please contact:
Parick Wascovich, UTSW
214-648-3404
Patrick.Wascovich@UTSouthwestern.edu
Sign up for a free weekly e-mail on Medical Breakthroughs called
First to Know by clicking here.
