Brain Computer Interface (BCI) for Speech Decoding
Brain-computer interfaces allow us to capture brain activity, analyze these signals, and translate them into commands for a computer to act out. This technology is what allows individuals to control prosthetic limbs or motorized wheelchairs with their thoughts. For many researchers, the aim of brain-computer interfaces is to empower users and restore function where it has been lost. This fast-growing field is the future of rehabilitation and quality-of-life improvement.
In the field of brain-computer interfaces (BCIs), speech synthesis is a unique application that has the potential to improve the quality of life for individuals with speech impairments. For those with conditions such as Amyotrophic Lateral Sclerosis (ALS) and Locked-In Syndrome, communication with others is difficult despite cognitive faculties being intact. The development of a BCI that can synthesize speech from thoughts in real time may empower individuals with impaired speech to communicate despite physical limitations. For example, individuals recovering from brainstem strokes may regain speech function, allowing them to communicate more freely during their rehabilitation process. Furthermore, patients with ALS can utilize this BCI to maintain their ability to communicate throughout their disease progression, thereby improving their quality of life.
Magnetoencephalography (MEG) is a non-invasive neuroimaging technique with high spatiotemporal resolution (<0.2 ms and <5 mm), offering a uniquely exceptional brain signal source. When our neurons fire, there is an electrical current. This electrical current generates a magnetic field. MEG amplifies that magnetic field and uses it to provide information about brain activity. This research project aims to use the University of Florida’s state-of-the-art MEG technology and integrate it with sophisticated machine learning algorithms, notably large language models like generative pre-trained transformers (GPT), to create a real-time BCI speech synthesis tool.
We aim to develop a deep learning framework for instant speech synthesis from neuromagnetic brain activity by extracting speech articulation and phonetic features from the MEG brainwave patterns. Previous research has some success in using brain activity captured from intracranial recording and non-invasive neuroimaging, such as through functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and MEG, in combination with machine learning models and language models to reconstruct speech. This project will use UF’s local MEG machinery to gather a large body of recording data with the intent of improving the current efficacy of MEG for speech synthesis.