In Ophthalmology science
PURPOSE : To develop a method for objective analysis of the reproducible steps in routine cataract surgery.
DESIGN : Prospective study; machine learning.
PARTICIPANTS : Deidentified faculty and trainee surgical videos.
METHODS : Consecutive cataract surgeries performed by a faculty or trainee surgeon in an ophthalmology residency program over 6 months were collected and labeled according to degrees of difficulty. An existing image classification network, ResNet 152, was fine-tuned for tool detection in cataract surgery to allow for automatic identification of each unique surgical instrument. Individual microscope video frame windows were subsequently encoded as a vector. The relation between vector encodings and perceived skill using k-fold user-out cross-validation was examined. Algorithms were evaluated using area under the receiver operating characteristic curve (AUC) and the classification accuracy.
MAIN OUTCOME MEASURES : Accuracy of tool detection and skill assessment.
RESULTS : In total, 391 consecutive cataract procedures with 209 routine cases were used. Our model achieved an AUC ranging from 0.933 to 0.998 for tool detection. For skill classification, AUC was 0.550 (95% confidence interval [CI], 0.547-0.553) with an accuracy of 54.3% (95% CI, 53.9%-54.7%) for a single snippet, AUC was 0.570 (0.565-0.575) with an accuracy of 57.8% (56.8%-58.7%) for a single surgery, and AUC was 0.692 (0.659-0.758) with an accuracy of 63.3% (56.8%-69.8%) for a single user given all their trials.
CONCLUSIONS : Our research shows that machine learning can accurately and independently identify distinct cataract surgery tools in videos, which is crucial for comparing the use of the tool in a step. However, it is more challenging for machine learning to accurately differentiate overall and specific step skill to assess the level of training or expertise.
FINANCIAL DISCLOSURES : The author(s) have no proprietary or commercial interest in any materials discussed in this article.
Ruzicki Jessica, Holden Matthew, Cheon Stephanie, Ungi Tamas, Egan Rylan, Law Christine
2023-Mar
AUC, area under the receiver operating characteristic curve, Artificial intelligence, CI, confidence interval, Cataract surgery, Education
