In Ophthalmology ; h5-index 90.0
PURPOSE : To evaluate the performance of retinal specialists in detecting retinal fluid presence in spectral domain optical coherence tomography (SD-OCT) macular volume scans from eyes with age-related macular degeneration (AMD), and to compare performance with the artificial intelligence (AI)-based Notal OCT Analyzer (NOA).
DESIGN : Prospective comparison of retinal fluid grades from human retinal specialists and the NOA on SD-OCT scans from two commonly used devices (Cirrus and Spectralis).
PARTICIPANTS : 1,127 eyes of 651 Age-Related Eye Disease Study 2 10-year (AREDS2-10Y) participants with SD-OCT scans graded for fluid presence/absence by reading center (RC) graders.
METHODS : The AREDS2-10Y investigators graded each SD-OCT scan for the presence/absence of intraretinal and subretinal fluid. Separately, the same scans were graded (with masking to the investigator results) by (i) the NOA, and (ii) RC graders, which were used as the ground truth.
MAIN OUTCOME MEASURES : Accuracy (primary); sensitivity, specificity, precision, F1-score.
RESULTS : Mean participant age was 80.0 years (SD 7.6). Of the 1,127 eyes, retinal fluid was present in 32.8%. For detecting retinal fluid (intraretinal or subretinal), the investigators had accuracy 0.805 (95% CI 0.780-0.828), sensitivity 0.468 (0.416-0.520), specificity 0.970 (0.955-0.981), precision 0.883 (0.829-0.924), and F1-score 0.611. The NOA had accuracy 0.851 (0.829-0.871), sensitivity 0.822 (0.779-0.859), specificity 0.865 (0.839-0.889), precision 0.749 (0.704-0.790), and F1-score 0.784. For detecting intraretinal fluid, the investigators had accuracy 0.815 (0.792-0.837), sensitivity 0.403 (0.349-0.459), and specificity 0.978 (0.966-0.987); the NOA metrics were 0.877 (0.857-0.896), 0.763 (0.713-0.808), and 0.922 (0.902-0.940), respectively. For detecting subretinal fluid, the investigators had accuracy 0.946 (0.931-0.958), sensitivity 0.583 (0.471-0.690), and specificity 0.973 (0.962-0.982); the NOA metrics were 0.863 (0.842-0.882), 0.940 (0.867-0.980), and 0.857 (0.835-0.877).
CONCLUSIONS : In this large and challenging sample of SD-OCT scans obtained with two commonly used devices, retinal specialists had imperfect accuracy in detecting retinal fluid, with low sensitivity. This was particularly true for (i) intraretinal fluid and (ii) low fluid volume appearing on fewer B-scans (i.e., harder to identify). AI-based detection achieved a higher level of accuracy. This AI software tool could assist physicians in detecting retinal fluid, which is important for diagnostic, retreatment, and prognostic tasks in AMD.
Keenan Tiarnan D, Clemons Traci E, Domalpally Amitha, Elman Michael J, Havilio Moshe, Agrón Elvira, Benyamini Gidi, Chew Emily Y