ArXiv Preprint
Cytopathology report generation is a necessary step for the standardized
examination of pathology images. However, manually writing detailed reports
brings heavy workloads for pathologists. To improve efficiency, some existing
works have studied automatic generation of cytopathology reports, mainly by
applying image caption generation frameworks with visual encoders originally
proposed for natural images. A common weakness of these works is that they do
not explicitly model the structural information among cells, which is a key
feature of pathology images and provides significant information for making
diagnoses. In this paper, we propose a novel graph-based framework called
GNNFormer, which seamlessly integrates graph neural network (GNN) and
Transformer into the same framework, for cytopathology report generation. To
the best of our knowledge, GNNFormer is the first report generation method that
explicitly models the structural information among cells in pathology images.
It also effectively fuses structural information among cells, fine-grained
morphology features of cells and background features to generate high-quality
reports. Experimental results on the NMI-WSI dataset show that GNNFormer can
outperform other state-of-the-art baselines.
Yang-Fan Zhou, Kai-Lang Yao, Wu-Jun Li
2023-03-17
