A tour of biomedical image segmentation using deep learning
What is this tutorial about?
Years ago, when I first was first introduced to the field of biomedical image analyis, I was scared by the number of papers out there. In particular, I was overwhelmed by how different research disciplines were organized and structured within biomedical imaging. I had no clue where to start, i.e., which key papers, books to read, and where to go next from there. Therefore, here, I present a nice overview of medical image segmentation using deep learning (I plan to make another set of videos soon for segmentation methods before the deep learning era). In this two part tutorial, we will follow a systematic approach starting from U-Net and its variants to build a solid foundation and eventually by the end of part 2 of the tutorial, we will be able to talk about advanced architectures such as Transformers. I have designed the course in a way accessible to someone who wants to see how how the field is organized and how does one build their basics and then is able to systematically build on that. Thus, we will have a whirlwind tour of state of the art papers in deep medical image segmentation. Here, are the time-stamped links and full lectures follow thereafter.
Tutorial 1 covers the following topics:
- Building foundational building blocks
- Domain adaptation and generalization
- Meta-learning
- Domain Disentanglement
- Generative modeling and synthesis
Tutorial 2 covers the following topics:
- Shape-priors, topology-aware and geometrically-constrained networks
- Representation learning, self-supervised and multi-task learning
- Weakly-supervised and few-shot learning
- Model distillation (student-teacher networks)
- Transformers
Tutorial 1
Tutorial 2
Papers that we covered in this tutorial
- U-Net: Convolutional Networks for Biomedical Image Segmentation. Olaf Ronneberger, et al. 2015
- Deep Residual Learning for Image Recognition. He et. al 2015
- Jurdi et al., High-level prior-based loss functions for medical image segmentation: A survey 2021
- nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Isensee, Fabian, et al.
- Fully Automated 3D Segmentation and Diffeomorphic Medial Modeling of the Left Ventricle Mitral Valve Complex in Ischemic Mitral Regurgitation. Aly et. al 2021
- Automated deep learning segmentation of high-resolution 7 T ex vivo MRI for quantitative analysis of structure-pathology correlations in neurodegenerative diseases.Khandelwal, Pulkit, et al.
- Eckermann et al. 2021, Three-dimensional virtual histology of the human hippocampus based on phase-contrast computed tomography
- FreeSurfer. Fischl et al. 1999, 2000, 2012
- FastSurfer. Henschel et al. 2020
- DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy Christian Wachinger et al. 2017
- Martin Reuter et al. 2005, Laplace–Beltrami spectra as ‘Shape-DNA’ of surfaces and solids
- Attention U-Net: Learning Where to Look for the Pancreas Oktay et al. 2018
- Feature re-calibration: Squeeze and excitation networks. Hu et al. 2018, Roy et al. 2018/19
- AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy. Zhu et al. 2018
- Spherical U-Net on Cortical Surfaces. Zhao et al. 2018
- Deep Label Fusion: multi-atlas with deep learning. Xie et al. 2021, 22
- Li et al. 2017, Deeper, Broader and Artier Domain Generalization
- Ganin et al. 2015, Domain-Adversarial Training of Neural Networks
- Finn et al. 2017, Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Khandelwal and Yushkevich 2020, Domain Generalizer: A Few-shot Meta Learning Framework for Domain Generalization in Medical Imaging
- Dou et al. 2019, Domain Generalization via Model-Agnostic Learning of Semantic Features
- Liu et al. 2020, Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains
- Park et al. 2020, Contrastive Learning for Unpaired Image-to-Image Translation
- Diverse Image-to-Image Translation via Disentangled Representations.Li et al. 2018, 19
- Liu et al. 2022, Learning disentangled representations in the imaging domain
- Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation. Yang et al. 2019
- Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion. Chen et al. 2020
- Unsupervised Deformable Registration for Multi-modal Images via Disentangled Representations. Qin et al. 2019
- Semi-supervised Meta-learning with Disentanglement for Domain-generalized Medical Image Segmentation. Liu et al. 2021
- F. S´egonne and B. Fischl 2007
- Spine and Individual Vertebrae Segmentation in Computed Tomography Images using Geometric Flows and Shape Priors. Khandelwal, Collins and Siddiqi. 2019 and 2021
- Topology-Preserving Deep Image Segmentation. Hu et al. 2019
- clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation. Suprosanna Shit et al. 2020
- PialNN: A Fast Deep Learning Framework for Cortical Pial Surface Reconstruction. Ma et al. 2021
- TopoFit: Rapid Reconstruction of Topologically-Correct Cortical Surfaces. Hoopes et al. 2022
- DeepCSR: A 3D Deep Learning Approach for Cortical Surface Reconstruction. Cruz et al. 2021
- 3D Self-Supervised Methods for Medical Imaging. Taleb et al. 2020
- UNETR: Transformers for 3D Medical Image Segmentation. Hatamizadeh et al. 2021
- Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis. Tang et al. 2022
- nnFormer: Volumetric Medical Image Segmentation via a 3D Transformer. Zhou et al. 2022
- Affine Medical Image Registration with Coarse-to-Fine Vision Transformer. Mok et al. 2022
- TransMorph: Transformer for unsupervised medical image registration. Chen et al. 2022