ArXiv Preprint
Deep learning empowers the mainstream medical image segmentation methods.
Nevertheless current deep segmentation approaches are not capable of
efficiently and effectively adapting and updating the trained models when new
incremental segmentation classes (along with new training datasets or not) are
required to be added. In real clinical environment, it can be preferred that
segmentation models could be dynamically extended to segment new organs/tumors
without the (re-)access to previous training datasets due to obstacles of
patient privacy and data storage. This process can be viewed as a continual
semantic segmentation (CSS) problem, being understudied for multi-organ
segmentation. In this work, we propose a new architectural CSS learning
framework to learn a single deep segmentation model for segmenting a total of
143 whole-body organs. Using the encoder/decoder network structure, we
demonstrate that a continually-trained then frozen encoder coupled with
incrementally-added decoders can extract and preserve sufficiently
representative image features for new classes to be subsequently and validly
segmented. To maintain a single network model complexity, we trim each decoder
progressively using neural architecture search and teacher-student based
knowledge distillation. To incorporate with both healthy and pathological
organs appearing in different datasets, a novel anomaly-aware and confidence
learning module is proposed to merge the overlapped organ predictions,
originated from different decoders. Trained and validated on 3D CT scans of
2500+ patients from four datasets, our single network can segment total 143
whole-body organs with very high accuracy, closely reaching the upper bound
performance level by training four separate segmentation models (i.e., one
model per dataset/task).
Zhanghexuan Ji, Dazhou Guo, Puyang Wang, Ke Yan, Jia Ge, Xianghua Ye, Minfeng Xu, Jingren Zhou, Le Lu, Mingchen Gao, Dakai Jin
2023-02-01
