ArXiv Preprint
Medical imaging analysis plays a critical role in the diagnosis and treatment
of various medical conditions. This paper focuses on chest X-ray images and
their corresponding radiological reports. It presents a new model that learns a
joint X-ray image & report representation. The model is based on a novel
alignment scheme between the visual data and the text, which takes into account
both local and global information. Furthermore, the model integrates
domain-specific information of two types -- lateral images and the consistent
visual structure of chest images. Our representation is shown to benefit three
types of retrieval tasks: text-image retrieval, class-based retrieval, and
phrase-grounding.
Gefen Dawidowicz, Elad Hirsch, Ayellet Tal
2023-03-21
