收录解读
放射学AI目前多局限于窄任务、数据密集且对领域转移脆弱,基础模型有望提供可迁移的数据高效方案,但现有工作规模有限、评估片面且假设单一通用模型。
本文提出了nnFoundation,包含互补的卷积与Transformer两种3D放射学基础模型,在210万CT/MRI/PET影像体积上训练,并可通过后处理动态对齐模型拓扑,适配不同数据集特性。
在108个分割、检测、分类、报告生成、图像检索任务及领域转移、外部合作、低数据和低计算场景下评估,卷积模型在局部任务占优,Transformer在全局推理和冻结特征任务领先,整体超越此前3D基础模型和从头训练。
该工作表明3D放射学的可迁移性能取决于互补架构与数据集感知适配而非单一通用模型,模型已集成至nnU-Net和nnDetection便于部署,但摘要中未报告具体量化指标,限制了验证强度的直接可测性。
原始摘要
Radiological artificial intelligence has advanced rapidly, yet most systems remain narrowly task-specific, data-intensive, and fragile under domain shift. Foundation models promise more transferable and data-efficient solutions, but existing approaches are limited in scale, evaluated narrowly, and often assume that a single pretrained model can support diverse downstream tasks. Here we present nnFoundation, complementary convolutional and transformer-based 3D radiological foundation models. Developed within the Human Radiome Project (THRP), nnFoundation is trained on 2.1 million CT, MRI, and PET image volumes from 125 institutional and public datasets. We evaluate them across 108 tasks spanning segmentation, detection, classification, report generation, and image retrieval, including evaluations under domain shift, by external partners and in low-data and low-compute regimes. Across all task types, our convolution- and transformer-based nnFoundation models consistently outperform both prior 3D foundation models and training from scratch, establishing state-of-the-art performance for radiological imaging. However, performance follows a consistent task-dependent structure: the convolutional nnFoundation model dominates spatially localized tasks, whereas the transformer-based nnFoundation model excels in tasks requiring global semantic reasoning and in frozen-feature settings. Dynamically aligning the foundation model topology with the dataset characteristics post-hoc further improves transfer across heterogeneous 3D settings. These results show that transferable 3D radiological performance is governed not by a single universal model, but by the interplay of scalable pretraining, complementary architectures, and dataset-aware adaptation. We release nnFoundation models integrated into nnU-Net and nnDetection, enabling immediate application across established radiology workflows.