In Canadian Association of Radiologists journal = Journal l'Association canadienne des radiologistes
PURPOSE : To develop and assess the performance of a machine learning model which screens chest radiographs for 14 labels, and to determine whether fine-tuning the model on local data improves its performance. Generalizability at different institutions has been an obstacle to machine learning model implementation. We hypothesized that the performance of a model trained on an open-source dataset will improve at our local institution after being fine-tuned on local data.
METHODS : In this retrospective, institutional review board approved study, an ensemble of neural networks was trained on open-source datasets of chest radiographs for the detection of 14 labels. This model was then fine-tuned using 4510 local radiograph studies, using radiologists' reports as the gold standard to evaluate model performance. Both the open-source and fine-tuned models' accuracy were tested on 802 local radiographs. Receiver-operator characteristic curves were calculated, and statistical analysis was completed using DeLong's method and Wilcoxon signed-rank test.
RESULTS : The fine-tuned model identified 12 of 14 pathology labels with area under the curves greater than .75. After fine-tuning with local data, the model performed statistically significantly better overall, and specifically in detecting six pathology labels (P < .01).
CONCLUSIONS : A machine learning model able to accurately detect 14 labels simultaneously on chest radiographs was developed using open-source data, and its performance was improved after fine-tuning on local site data. This simple method of fine-tuning existing models on local data could improve the generalizability of existing models across different institutions to further improve their local performance.
Mohn Sarah F, Law Marco, Koleva Maria, Lee Brian, Berg Adam, Murray Nicolas, Nicolaou Savvas, Parker William A
2022-Dec-21
artificial intelligence, chest radiographs, fine-tuning, machine learning
