In Journal of medical Internet research ; h5-index 88.0
Generative models, such as DALL-E 2 (OpenAI), could represent promising future tools for image generation, augmentation, and manipulation for artificial intelligence research in radiology, provided that these models have sufficient medical domain knowledge. Herein, we show that DALL-E 2 has learned relevant representations of x-ray images, with promising capabilities in terms of zero-shot text-to-image generation of new images, the continuation of an image beyond its original boundaries, and the removal of elements; however, its capabilities for the generation of images with pathological abnormalities (eg, tumors, fractures, and inflammation) or computed tomography, magnetic resonance imaging, or ultrasound images are still limited. The use of generative models for augmenting and generating radiological data thus seems feasible, even if the further fine-tuning and adaptation of these models to their respective domains are required first.
Adams Lisa C, Busch Felix, Truhn Daniel, Makowski Marcus R, Aerts Hugo J W L, Bressem Keno K
2023-Mar-16
DALL-E, artificial intelligence, creating images from text, diagnostic imaging, generative model, image creation, image generation, machine learning, medical imaging, radiology, text-to-image, transformer language model, x-ray
