ArXiv Preprint
Multi-sequence MRIs can be necessary for reliable diagnosis in clinical
practice due to the complimentary information within sequences. However,
redundant information exists across sequences, which interferes with mining
efficient representations by modern machine learning or deep learning models.
To handle various clinical scenarios, we propose a sequence-to-sequence
generation framework (Seq2Seq) for imaging-differentiation representation
learning. In this study, not only do we propose arbitrary 3D/4D sequence
generation within one model to generate any specified target sequence, but also
we are able to rank the importance of each sequence based on a new metric
estimating the difficulty of a sequence being generated. Furthermore, we also
exploit the generation inability of the model to extract regions that contain
unique information for each sequence. We conduct extensive experiments using
three datasets including a toy dataset of 20,000 simulated subjects, a brain
MRI dataset of 1,251 subjects, and a breast MRI dataset of 2,101 subjects, to
demonstrate that (1) our proposed Seq2Seq is efficient and lightweight for
complex clinical datasets and can achieve excellent image quality; (2)
top-ranking sequences can be used to replace complete sequences with
non-inferior performance; (3) combining MRI with our imaging-differentiation
map leads to better performance in clinical tasks such as glioblastoma MGMT
promoter methylation status prediction and breast cancer pathological complete
response status prediction. Our code is available at
https://github.com/fiy2W/mri_seq2seq.
Luyi Han, Tao Tan, Tianyu Zhang, Yunzhi Huang, Xin Wang, Yuan Gao, Jonas Teuwen, Ritse Mann
2023-02-01
