A new study published in the journal Neuron provides evidence that giving people immediate, real-time feedback on their success during a motor task can significantly improve their ability to control machines. This real-time reinforcement tends to be especially helpful when users have limited visual or physical sensations to guide their movements. The research suggests a promising strategy to enhance rehabilitation technologies for stroke patients and improve the functional use of advanced prosthetic limbs.
Scientists and engineers are actively developing clinical interventions that help individuals recover or replace lost physical abilities. These interventions often rely on human-machine interfaces. A human-machine interface is a system that allows a person to interact with or control a computer or robotic device, such as a virtual reality platform used for stroke rehabilitation or a robotic prosthetic hand for an amputee.
To operate these technologies, a user must generate a specific sequence of physical movements to achieve a goal, such as reaching out to grasp a glass of water. A major challenge in this field is that users often perform these actions with limited sensory feedback. When a person uses a robotic hand, they do not feel the natural sensation of touch, and stroke patients often experience neurological deficits that impair their vision or physical sense of their own body movements.
Sensory uncertainty makes it difficult to control devices smoothly and accurately. To address this problem, scientists have tried adding artificial sensory feedback to these technologies. Even with artificial feedback, the sensory information remains incomplete because of noisy sensors or slight delays in transmitting the data.
One proposed solution is to use mechanisms of reinforcement learning. Reinforcement learning is a process where a person or system learns to make better decisions by receiving feedback about success or failure. In daily life, this might look like earning a high score in a video game or hearing a pleasant chime when completing a digital task, which directly signals whether a goal was met and carries built-in motivational value.
In most previous movement studies, reinforcement feedback was only given at the very end of a task. This endpoint feedback can be confusing during complex, continuous actions. If a person fails at a multi-step movement, a single failure signal at the end does not explain which specific part of the movement went wrong.
Pierre Vassiliadis, a researcher at University College London who conducted this work during his time at the École Polytechnique Fédérale de Lausanne in Switzerland, wanted to explore how timing impacts this learning process. “We were interested in a simple question: in many motor tasks, people only receive feedback after the movement is over, but real movements unfold continuously,” Vassiliadis said.
“We wondered whether giving an explicit success signal in real time, while someone is moving, could help people control a human-machine interface more effectively, especially in situations where normal sensory feedback is limited, as is often the case in rehabilitation or prosthetic control,” Vassiliadis said. The researchers proposed that delivering these success and failure signals continuously would provide better guidance.
To test this idea, the researchers designed a series of five experiments involving a total of 106 participants. In the first three experiments, healthy young adults completed a continuous tracking task. Participants squeezed a specialized handgrip device to control a digital cursor on a computer screen, attempting to adjust their grip force to keep the cursor inside a moving target for exactly seven seconds.
The scientists manipulated the visual feedback by hiding the cursor for different amounts of time. In full vision conditions, the cursor was always visible. In low vision conditions, the cursor was only visible for about 35 percent of the trial.
To test real-time reinforcement, the target on the screen changed colors based on the participant’s performance. The target turned green to indicate success and red to indicate failure at any given moment. This color change happened in real time and was personalized for each participant.
A software program constantly calculated the user’s average error over the past few trials, requiring them to constantly beat their own recent performance to see the green success signal. Control trials featured targets that flashed random, uninformative colors.
The first experiment involved 24 healthy adults and showed that reducing visual feedback strongly impaired the ability to control the cursor. Introducing real-time reinforcement improved the participants’ performance across all conditions. The benefits were substantial in the low vision condition and relatively small in the full vision condition.
The researchers also measured skill retention, finding that real-time reinforcement helped participants maintain their improved motor skills specifically when they trained under low visual feedback. “A very simple signal, here, a real-time success cue based on performance, can help people learn to control a device better,” Vassiliadis said.
“We found that this was particularly helpful when visual or somatosensory feedback was reduced, which is important because many patients and assistive technologies operate under exactly those conditions,” Vassiliadis told PsyPost. “In other words, a small and inexpensive change in feedback design may make human-machine interfaces easier to use and train.”
The second experiment asked the same 24 participants to continue practicing the task with more varied levels of visual feedback to confirm that the performance gains were not simply due to participants reaching the absolute limit of their physical abilities. The third experiment involved an entirely new group of 24 healthy participants. This separate group replicated the original findings, confirming that real-time reinforcement reliably improves motor control when visual information is scarce.
The fourth experiment explored whether these benefits extend to other types of technology and other physical senses. A new group of 40 healthy participants controlled the screen cursor using electrical signals generated by their muscles. The researchers attached surface electrodes to the participants’ biceps, requiring them to tense their muscles without moving their arms to maneuver the cursor.
Because the participants kept their arms completely still, they lacked their natural physical sense of movement. To replace this missing sense, the researchers provided artificial touch feedback using a small motorized device pressed against the palm of the participant’s hand. The motorized device applied physical pressure that matched the amount of muscle force the participant was producing.
The scientists then selectively reduced the visual feedback on the screen or the physical pressure on the hand. The researchers found that reducing either the visual or the touch feedback worsened the participants’ ability to control the muscle interface. Adding the real-time color changes indicating success or failure significantly reduced these errors.
The real-time reinforcement improved performance across all conditions where sensory feedback was limited. This suggests that the strategy works for different types of machine interfaces.
In the fifth experiment, the scientists tested the strategy on a clinical population. They recruited 18 older adults who had experienced a stroke that caused long-term physical movement impairments. The patients completed the original handgrip task using their impaired hands, and the researchers personalized the physical difficulty of the task to match each patient’s specific capabilities.
Similar to the healthy participants, the stroke patients showed significant improvements in their real-time motor control when they received reinforcement under low vision conditions. Surprisingly, the real-time reinforcement actually impaired the patients’ performance when they had full vision of the cursor. The researchers suspect that when full visual information is already available, the flashing success and failure colors might act as a distracting overload of information for patients with brain lesions.
Unlike the healthy younger adults, the stroke patients did not show lasting retention of the motor skill after the training sessions ended. Their performance returned to the baseline level once the reinforcement colors were removed. This lack of retention may be related to age-related learning deficits or specific brain changes caused by the stroke itself.
To understand how real-time reinforcement changes human behavior, the researchers analyzed the physical variability of the participants’ movements. When human beings practice a physical skill, they tend to adjust their movements based on what happened previously. If a movement fails, people usually try something different, which increases the variability of their actions in a process known as exploration.
If a movement is successful, people tend to repeat that exact same action, which reduces their physical variability in a process known as exploitation. The data revealed that under conditions of low visual feedback, real-time reinforcement caused participants to strongly exploit their successful actions. They locked into a winning strategy and repeated it accurately, whereas under full vision, the reinforcement caused them to explore more often after a failure.
“Our information-theoretic analyses suggested that the main effect of real-time reinforcement was not to make people explore more after failure, but to help them exploit success more effectively by stabilizing motor commands that had just worked,” Vassiliadis said.
“We also found that this success-related stabilization was linked to later learning, which points to a specific mechanism by which reinforcement can improve motor skill acquisition,” Vassiliadis said.
While the study provides evidence for a new training strategy, it contains several limitations. The training sessions were incredibly short, with some conditions lasting for fewer than 20 attempts. This short timeframe likely explains why the stroke patients did not retain their new skills after the practice ended, meaning future research will need to test whether longer training sessions spread over multiple days can produce permanent physical improvements.
Another limitation involves how the researchers manipulated the physical and visual senses. Intermittently turning off a screen cursor or a robotic touch sensor creates a predictable environment for a science experiment, but real-world sensory loss caused by nerve damage or strokes is much more chaotic. Scientists will need to test real-time reinforcement in more unpredictable, naturalistic settings to confirm its clinical usefulness.
The study, “Real-time reinforcement for human-machine interface control,” was authored by Pierre Vassiliadis, Daniel Leal Pinheiro, Lisa Fleury, Silvestro Micera, Solaiman Shokur, and Friedhelm C. Hummel.