- Speaker: Dr. Momiao Xiong, Houston, Texas, USA
- Title： “SAIR-2-10: High-quality Video Reconstruction from Brain Activity”
- Date：9:00pm US East time, 06/01/2024
- Date：10:00am Beijing time, 07/01/2024
- Zoom URL: https://jmu-edu.zoom.us/j/2327715678
- Key words:AI, Deep learning, Neuroscience
Title: High-quality Video Reconstruction from Brain Activity
Background
Reconstructing human vision from brain activities has been an appealing task that helps to understand our cognitive process. Even though recent research has seen great success in reconstructing static images from non-invasive brain recordings, work on recovering continuous visual experiences in the form of videos is limited. In this work, Chen et al. (2023) proposeMinD-Video that learns spatiotemporal information from continuous fMRI,data of the cerebral cortex progressively through masked brain modeling, multimodal contrastive learning with spatiotemporal attention, and co-training with an augmented Stable Diffusion model that incorporates network temporal inflation. They show that high-quality videos of arbitrary frame rates can be reconstructed with MinD-Video using adversarial guidance. The recovered videos were evaluated with various semantic and pixel-level metrics. They achieved an average accuracy of 85% in semanticclassification tasks and 0.19 in structural similarity index (SSIM), outperforming the previous state-of-the-art by 45%.
Bio
Momiao Xiong, Ph. D, Professor in Department of Biostatistics snd Data Science , University of Texas, School of Public Health. Dr. Xiong graduated from the Department of Statistics at the University of Georgia in 1993. From 1993 to 1995, Dr. Xiong was postdoctoral fellow at the University of Southern California working with Michael Waterman.
Research Interesting
Causal Inference, Artificial Intelligence, Brain Decoding, Hiperdimensional Computing, Statistic Genetics and Bioinformatics . Intelligence is the ability to perceive information and to retain it as knowledge to be applied towards adaptive behavior within a changing environment. Many applications run machine learning algorithms to perform cognitive tasks.The learning algorithms have been shown effectiveness for many tasks, e.g., object tracking , speech recognition , image classification , etc. However, the high computational complexity and memory requirement of existing deep learning algorithms hinder usability to a wide variety of real-life embedded applications where the device resources and power budget is limited. Redesign the algorithms themselves using strategies that more closely model the ultimate efficient learning machine: the human brain. Hyperdimensional computing (HDC) is one such strategy developed by interdisciplinary research.It is based on a short-term human memory model, Sparse distributed memory, emerged from theoretical neuroscience . HDC is motivated by the understanding that the human brain operates on highdimensional representations of data originating from the large size of brain circuits.
