收录解读
FPED introduces a functional-network-prior mixture-of-experts framework for fMRI visual decoding, avoiding the common practice of flattening brain signals into unstructured vectors.
Different functional networks are modeled as experts, with adaptive routing estimating their contributions to visual semantic reconstruction and interpretation.
The method preserves more neurobiological structure while still connecting to CLIP-like semantic spaces and image reconstruction objectives.
For the repository, FPED is valuable as a NeuroAI bridge: it uses AI architectures to respect brain network topology while making decoding more interpretable.
原始摘要与中文对照
中文对照翻译
摘要。从功能性磁共振成像（fMRI）中重建视觉图像是脑解码中的一项基本任务，为理解人类感知机制和开发先进的脑机接口（BCI）提供了关键途径。然而，目前大多数方法只是简单地将来自局部视觉皮层的fMRI信号展平为一维（1D）向量，并将其直接映射到对比语言-图像预训练（CLIP）等潜在空间中。这种范式不仅破坏了大脑固有的网络拓扑结构——导致神经科学可解释性有限——而且忽视了其他分布式功能网络在处理高级视觉语义方面的协同贡献。为了解决这些局限性，我们提出了FPED，一个功能网络先验引导的专家混合（MoE）框架，用于可解释的脑解码。FPED明确地将不同的功能脑网络建模为专门的专家，并采用自适应路由来捕捉它们对视觉语义理解的互补贡献。与传统的同质解码范式不同，我们的框架结合了神经生物学基础的先验知识，以实现结构化和可解释的网络级表征学习。实验结果表明，FPED仅用0.68B参数就实现了极具竞争力的语义重建性能。学习到的路由动态揭示了功能脑网络与模态特定语义处理之间具有生物学意义的对应关系，提供了透明的神经科学可解释性。这表明，脑网络感知的专家建模是连接神经解码和受生物学启发的AI的一个有前景的方向。关键词：脑解码 · 专家混合 · 脑功能网络
原始摘要
Abstract. Visual image reconstruction from functional Magnetic Resonance Imaging (fMRI) is a fundamental task in brain decoding, providing a crucial pathway for understanding human perceptual mechanisms and developing advanced brain-computer interfaces (BCIs). However, most current methods simply flatten fMRI signals from localized visual cortices into one-dimensional (1D) vectors, mapping them directly into latent spaces such as that of Contrastive Language-Image Pre-training (CLIP). This paradigm not only disrupts the inherent network topology of the brain—leading to limited neuroscientific interpretability—but also overlooks the synergistic contributions of other distributed functional networks in processing high-level visual semantics. To address these limitations, we propose FPED, a Functional-Network Prior-Guided Mixture of Experts (MoE) framework for interpretable brain decoding. FPED explicitly models different functional brain networks as specialized experts and employs adaptive routing to capture their complementary contributions to visual semantic understanding. Unlike conventional homogeneous decoding paradigms, our framework incorporates neurobiologically grounded priors to enable structured and interpretable network-level representation learning. Experimental results demonstrate that FPED achieves highly competitive semantic reconstruction performance with only 0.68B parameters. The learned routing dynamics reveal biologically meaningful correspondence between functional brain networks and modality-specific semantic processing, providing transparent neuroscientific interpretability. This suggests that brain network-aware expert modeling is a promising direction for bridging neural decoding and biologically inspired artificial intelligence. Keywords: Brain Decoding · Mixture-of-Experts · Brain Functional Networks