In Medical image analysis
PURPOSE : Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill.
METHODS : To achieve this goal, a dataset with 33 laparoscopic cholecystectomy videos from three surgical centers with a total operation time of 22 h was created. Labels included framewise annotation of seven surgical phases with 250 phase transitions, 5514 occurences of four surgical actions, 6980 occurences of 21 surgical instruments from seven instrument categories and 495 skill classifications in five skill dimensions. The dataset was used in the 2019 international Endoscopic Vision challenge, sub-challenge for surgical workflow and skill analysis. Here, 12 research teams trained and submitted their machine learning algorithms for recognition of phase, action, instrument and/or skill assessment.
RESULTS : F1-scores were achieved for phase recognition between 23.9% and 67.7% (n = 9 teams), for instrument presence detection between 38.5% and 63.8% (n = 8 teams), but for action recognition only between 21.8% and 23.3% (n = 5 teams). The average absolute error for skill assessment was 0.78 (n = 1 team).
CONCLUSION : Surgical workflow and skill analysis are promising technologies to support the surgical team, but there is still room for improvement, as shown by our comparison of machine learning algorithms. This novel HeiChole benchmark can be used for comparable evaluation and validation of future work. In future studies, it is of utmost importance to create more open, high-quality datasets in order to allow the development of artificial intelligence and cognitive robotics in surgery.
Wagner Martin, Müller-Stich Beat-Peter, Kisilenko Anna, Tran Duc, Heger Patrick, Mündermann Lars, Lubotsky David M, Müller Benjamin, Davitashvili Tornike, Capek Manuela, Reinke Annika, Reid Carissa, Yu Tong, Vardazaryan Armine, Nwoye Chinedu Innocent, Padoy Nicolas, Liu Xinyang, Lee Eung-Joo, Disch Constantin, Meine Hans, Xia Tong, Jia Fucang, Kondo Satoshi, Reiter Wolfgang, Jin Yueming, Long Yonghao, Jiang Meirui, Dou Qi, Heng Pheng Ann, Twick Isabell, Kirtac Kadir, Hosgor Enes, Bolmgren Jon Lindström, Stenzel Michael, von Siemens Björn, Zhao Long, Ge Zhenxiao, Sun Haiming, Xie Di, Guo Mengqi, Liu Daochang, Kenngott Hannes G, Nickel Felix, Frankenberg Moritz von, Mathis-Ullrich Franziska, Kopp-Schneider Annette, Maier-Hein Lena, Speidel Stefanie, Bodenstedt Sebastian
2023-Feb-21
Endoscopic vision, Laparoscopic cholecystectomy, Surgical data science, Surgical workflow analysis
