ArXiv Preprint
We propose an image synthesis mechanism for multi-sequence prostate MR images
conditioned on text, to control lesion presence and sequence, as well as to
generate paired bi-parametric images conditioned on images e.g. for generating
diffusion-weighted MR from T2-weighted MR for paired data, which are two
challenging tasks in pathological image synthesis. Our proposed mechanism
utilises and builds upon the recent stable diffusion model by proposing
image-based conditioning for paired data generation. We validate our method
using 2D image slices from real suspected prostate cancer patients. The realism
of the synthesised images is validated by means of a blind expert evaluation
for identifying real versus fake images, where a radiologist with 4 years
experience reading urological MR only achieves 59.4% accuracy across all tested
sequences (where chance is 50%). For the first time, we evaluate the realism of
the generated pathology by blind expert identification of the presence of
suspected lesions, where we find that the clinician performs similarly for both
real and synthesised images, with a 2.9 percentage point difference in lesion
identification accuracy between real and synthesised images, demonstrating the
potentials in radiological training purposes. Furthermore, we also show that a
machine learning model, trained for lesion identification, shows better
performance (76.2% vs 70.4%, statistically significant improvement) when
trained with real data augmented by synthesised data as opposed to training
with only real images, demonstrating usefulness for model training.
Shaheer U. Saeed, Tom Syer, Wen Yan, Qianye Yang, Mark Emberton, Shonit Punwani, Matthew J. Clarkson, Dean C. Barratt, Yipeng Hu
2023-03-03
