In Augmented reality i Virtual reality news
July 16, 2021 – Facebook reality labs (FRL), Facebook’s augmented and virtual reality (AR / VR) research team, celebrates new results published by University of California, San Francisco (UCSF) The New England Journal of Medicine, which show how someone with severe speech loss has been able to write what he meant almost instantly, simply trying to speak. In other words, UCSF has restored a person’s ability to communicate by decoding brain signals sent from the motor cortex to the vocal tract. FRL stated that the study marks an important milestone for the field of neuroscience and concludes Facebook’s collaboration with UCSF for years Chang Lab.
Established in 2017, Facebook’s Reality Labs (FRL) Brain-Computer Interface (BCI) project began with an ambitious long-term goal: to develop a quiet, non-invasive speech interface that would allow people to write just by imagining the words they they mean.
The FRL team has advanced in this mission over four years, investing in the exploration of head-mounted optical BCI as a potential input method for the upcoming computer platform, essentially a way to communicate in AR / VR with voice speed and writing discretion.
However, in a long blog post, FRL stated that Facebook has no interest in developing products that require electrodes implanted in the brain and that, while it still believes in the long-term potential of head-mounted BCI optical technologies, it has now decided to focus its immediate efforts in a different neuronal interface approach that has a path to the market in the shorter term: wrist-based devices powered by electromyography (EMG).
FRL explained how EMG works: When you decide to move your hands and fingers, the brain sends signals through the arm through motor neurons, telling them to move in specific ways in order to perform actions such as hitting or slide. EMG can capture and decode these signals to the wrist and translate them into digital commands for a device. In the short term, these signals will allow users to communicate with their devices with a highly reliable, subtle, customizable and adaptable degree of control to many situations. The company added that as this area of research evolves, “EMG-based neural interfaces have the potential to dramatically expand the bandwidth with which we can communicate with our devices, opening up the possibility of things such as high speed typing “.
We are still in the early stages of unlocking the potential of wrist-based electromyography, but we believe it will be the basic input of RA glasses and applying what we have learned about BCI will help us get there faster. ”
FRL Research Director Sean Keller commented: “We are developing more natural and intuitive ways to interact with AR glasses that are always available, so we don’t have to choose between interacting with our device and the world around us. . We are still in the early stages of unlocking the potential of wrist-based electromyography, but we believe it will be the basic input of RA glasses and applying what we have learned about BCI will help us get there faster. ”
Speech was the focus of FRL’s BCI research, as it has an intrinsically high bandwidth: you can speak faster than you can write. But discourse is not the only way to apply this research, according to FRL. The fundamental work of the BCI team can also be leveraged to enable intuitive wrist-based controls. As a result, FRL no longer seeks a research path to develop a silent, non-invasive speech interface, but pursues new forms of intuitive control with EMG, as it focuses on wrist-based input devices for AR / VR.
“As a team, we’ve realized that the real-time biofeedback and decoding algorithms we use for BCI optical research can accelerate what we can do with wrist-based EMG,” said Emily Mugler, research manager at FRL neural engineering. “We really want you to be able to intuitively control our next-generation bracelets in the first few minutes of wearing them. To use a subtle control scheme with confidence, you need your device to give you feedback, which confirms that it understands your goal … Applying these BCI research concepts to EMG can help control wrist-based be intuitive and helpful from the start. “
FRL concluded by saying that it will continue to share more as the work progresses and that by the end of this year it will share more about how haptic wearables will add another dimension to the next computing platform and “improve our ability to establish presence and learn a new interaction paradigms “.
To learn more about Facebook Reality Labs and its research on augmented and virtual reality, click here.
Image credit: Facebook Reality Labs