In Annals of surgery ; h5-index 104.0
OBJECTIVE : To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting.
SUMMARY BACKGROUND DATA : To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown.
METHODS : Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to PRISMA-ScR guidelines.
RESULTS : Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2,000; seven of these eight articles (87.5%) had below-average (<0.83) area under the receiver operating characteristic (AUROC) or accuracy. Overall, 29 articles (80.6%) performed internal validation only, five (13.8%) performed external validation, and two (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy.
CONCLUSIONS : Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.
Loftus Tyler J, Altieri Maria S, Balch Jeremy A, Abbott Kenneth L, Choi Jeff, Marwaha Jayson S, Hashimoto Daniel A, Brat Gabriel A, Raftopoulos Yannis, Evans Heather L, Jackson Gretchen P, Walsh Danielle S, Tignanelli Christopher J
2023-Mar-21
