Ověření toho, co nám bylo řečeno: Rekonstrukce řeči z elektroencefalografických (EEG) dat

Abstract

Previous studies have shown that it is possible to reconstruct speech fromintracranial recordings of the brainactivity of the listeners. However the use of invasive recording techniques is limited to animals, and to humans who are undergoing certain clinical treatments. To explore the feasibility of performing speech reconstruction with non-invasive neural recordings, we have measured the brain activity of subjects who listened to samples of continuous natural speech using a technique called electroencephalography (EEG). In the past, EEG studies have focused on the reconstruction of the speech envelope, but to dat e, itis unknown if EEG data reflectmore speech features than just the envelope tracking. To explore if the EEG recordings are sensitive to the spectrotemporal features of speech, we have designed a series of tests to investigate how much phonetic information can be recovered from the reconstructions of the speech spectrogramthat were estimated fromthose recordings. Our results show that spectrogramreconstructions from EEG data contain enough spectrotemporal features to allow for the discriminationof consonants and non-consonants, and for the identification of averaged representationsof certain phonemes. These results indicate that EEG recordings are sensitive to more than just thetracking of the speech envelope, and that the features contained in them could allow for the estimation of its phonetic content.

Previous studies have shown that it is possible to reconstruct speech fromintracranial recordings of the brainactivity of the listeners. However the use of invasive recording techniques is limited to animals, and to humans who are undergoing certain clinical treatments. To explore the feasibility of performing speech reconstruction with non-invasive neural recordings, we have measured the brain activity of subjects who listened to samples of continuous natural speech using a technique called electroencephalography (EEG). In the past, EEG studies have focused on the reconstruction of the speech envelope, but to dat e, itis unknown if EEG data reflectmore speech features than just the envelope tracking. To explore if the EEG recordings are sensitive to the spectrotemporal features of speech, we have designed a series of tests to investigate how much phonetic information can be recovered from the reconstructions of the speech spectrogramthat were estimated fromthose recordings. Our results show that spectrogramreconstructions from EEG data contain enough spectrotemporal features to allow for the discriminationof consonants and non-consonants, and for the identification of averaged representationsof certain phonemes. These results indicate that EEG recordings are sensitive to more than just thetracking of the speech envelope, and that the features contained in them could allow for the estimation of its phonetic content.