A Brain Implant Let Paralysed People Speak and Wave at the Same Time, Through an Avatar, for the First Time

Conversation is not only words. It is the hand that comes up as you say hello, the nod that goes with yes, the shrug that changes the meaning of a sentence. Brain-computer interfaces have restored speech to people who cannot speak and movement to people who cannot move, but always one at a time, through separate systems. A paper published on Monday in Nature Neuroscience describes the first that does both at once.

What was done

The work comes from Edward Chang's laboratory at the University of California, San Francisco, which has led the field for a decade. Three people with paralysis of the vocal tract and body, from amyotrophic lateral sclerosis or from a stroke in the brainstem, had an electrocorticography array, a sheet of electrodes, placed on the surface of the motor cortex, the strip of brain that plans movement of the face, mouth, arms and hands. The team recorded from it while the participants attempted to speak, attempted to make gestures such as a wave or a thumbs-up, or attempted both together, and trained machine-learning models to turn the patterns into phrases and movements. The output is a full-body animated avatar of the user, rendered in real time, which says the words and makes the movement together. Two of the three participants were able to control it, and one reached full accuracy on both speech and gesture in three of the assessments, according to Science's report.

Why it matters

"Conversation is about much more than the words being spoken," Chang said. "It's a multilayered, dynamic process involving the whole motor cortex." The technical point is that speech and gesture are planned in overlapping tissue and can be read from the same electrodes without interfering with each other, which nobody had shown. The human point was made by Debara Tucci, who directs the US institute that funded the work: "These promising results give me hope that in the future, patients with severe paralysis will be able to recapture the holistic nature of human communication." A person who can only type, or only produce synthesised speech, communicates. A person whose avatar waves as it greets you is present in the room in a way that a text box is not.

What it is not

It is not a product. The system in the paper is wired: the implanted electrodes connect through the skull to external processors, which confines it to the laboratory. Chang's group says a fully implanted wireless version will be tested soon, and that is the step that turns a demonstration into a treatment. Three participants is three participants, and the paper does not claim more. The vocabulary and the gesture set are limited, and the avatar's repertoire is a handful of movements rather than the full expressive range of a human upper body. All of that is the normal shape of a first paper in a field that has, in five years, gone from decoding single words to decoding whole sentences at conversational speed, and now to decoding what the hands are doing at the same time.

For readers in this region the relevant fact is that ALS and brainstem stroke are no rarer here than anywhere else, and that the Gulf's neurosurgical centres are already among the sites being considered for trials of implanted systems. When the wireless version arrives, the question will be the one that follows every such advance: whether it is a device that a health system pays for, or a demonstration that appears on the news. The paper is Chang et al., "Simultaneous speech and gesture decoding for multimodal communication in paralysis", Nature Neuroscience, 14 September 2026.