ArXiv Preprint
Robust semantic segmentation of intraoperative image data could pave the way
for automatic surgical scene understanding and autonomous robotic surgery.
Geometric domain shifts, however, although common in real-world open surgeries
due to variations in surgical procedures or situs occlusions, remain a topic
largely unaddressed in the field. To address this gap in the literature, we (1)
present the first analysis of state-of-the-art (SOA) semantic segmentation
networks in the presence of geometric out-of-distribution (OOD) data, and (2)
address generalizability with a dedicated augmentation technique termed "Organ
Transplantation" that we adapted from the general computer vision community.
According to a comprehensive validation on six different OOD data sets
comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs
semantically annotated with 19 classes, we demonstrate a large performance drop
of SOA organ segmentation networks applied to geometric OOD data. Surprisingly,
this holds true not only for conventional RGB data (drop of Dice similarity
coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the
latter's rich information content per pixel. Using our augmentation scheme
improves on the SOA DSC by up to 67 % (RGB) and 90 % (HSI) and renders
performance on par with in-distribution performance on real OOD test data. The
simplicity and effectiveness of our augmentation scheme makes it a valuable
network-independent tool for addressing geometric domain shifts in semantic
scene segmentation of intraoperative data. Our code and pre-trained models will
be made publicly available.
Jan Sellner, Silvia Seidlitz, Alexander Studier-Fischer, Alessandro Motta, Berkin Özdemir, Beat Peter Müller-Stich, Felix Nickel, Lena Maier-Hein
2023-03-20
