ArXiv Preprint
Generated synthetic data in medical research can substitute privacy and
security-sensitive data with a large-scale curated dataset, reducing data
collection and annotation costs. As part of this effort, we propose UniXGen, a
unified chest X-ray and report generation model, with the following
contributions. First, we design a unified model for bidirectional chest X-ray
and report generation by adopting a vector quantization method to discretize
chest X-rays into discrete visual tokens and formulating both tasks as sequence
generation tasks. Second, we introduce several special tokens to generate chest
X-rays with specific views that can be useful when the desired views are
unavailable. Furthermore, UniXGen can flexibly take various inputs from single
to multiple views to take advantage of the additional findings available in
other X-ray views. We adopt an efficient transformer for computational and
memory efficiency to handle the long-range input sequence of multi-view chest
X-rays with high resolution and long paragraph reports. In extensive
experiments, we show that our unified model has a synergistic effect on both
generation tasks, as opposed to training only the task-specific models. We also
find that view-specific special tokens can distinguish between different views
and properly generate specific views even if they do not exist in the dataset,
and utilizing multi-view chest X-rays can faithfully capture the abnormal
findings in the additional X-rays. The source code is publicly available at:
https://github.com/ttumyche/UniXGen.
Hyungyung Lee, Wonjae Kim, Jin-Hwa Kim, Tackeun Kim, Jihang Kim, Leonard Sunwoo, Edward Choi
2023-02-23
