Multimodal Integration of EEG and Near-Infrared Spectroscopy for Robust Cross-Frequency Coupling Estimation

Neuroimaging techniques have had a major impact on medical science, allowing advances in the research of many neurological diseases and improving their diagnosis. In this context, multimodal neuroimaging approaches, based on the neurovascular coupling phenomenon, exploit their individual strengths to provide complementary information on the neural activity of the brain cortex. This work proposes a novel method for combining electroencephalography (EEG) and functional near–infrared spectroscopy (fNIRS) to explore the functional activity of the brain processes related to low-level language processing of skilled and dyslexic seven-year-old readers. We have transformed EEG signals into image sequences considering the interaction between different frequency bands by means of cross-frequency coupling (CFC), and applied an activation mask sequence obtained from the local functional brain activity inferred from simultaneously recorded fNIRS signals. Thus, the resulting image sequences preserve spatial and temporal information of the communication and interaction between different neural processes and provide discriminative information that allows differentiation between controls and dyslexic subjects with an AUC of 77.1%. Finally, explainability is improved by introducing an easily comprehensible representation of the SHAP values obtained for the classification method in the brainSHAP maps.