In Journal of chemical information and modeling

Ponting David John, van Deursen Ruud, Ott Martin A

2020-Sep-23

In International journal of computer assisted radiology and surgery

PURPOSE : Machine learning (ML) algorithms are well known to exhibit variations in prediction accuracy when provided with imbalanced training sets typically seen in medical imaging (MI) due to the imbalanced ratio of pathological and normal cases. This paper presents a thorough investigation of the effects of class imbalance and methods for mitigating class imbalance in ML algorithms applied to MI.

METHODS : We first selected five classes from the Image Retrieval in Medical Applications (IRMA) dataset, performed multiclass classification using the random forest model (RFM), and then performed binary classification using convolutional neural network (CNN) on a chest X-ray dataset. An imbalanced class was created in the training set by varying the number of images in that class. Methods tested to mitigate class imbalance included oversampling, undersampling, and changing class weights of the RFM. Model performance was assessed by overall classification accuracy, overall F1 score, and specificity, recall, and precision of the imbalanced class.

RESULTS : A close-to-balanced training set resulted in the best model performance, and a large imbalance with overrepresentation was more detrimental to model performance than underrepresentation. Oversampling and undersampling methods were both effective in mitigating class imbalance, and efficacy of oversampling techniques was class specific.

CONCLUSION : This study systematically demonstrates the effect of class imbalance on two public X-ray datasets on RFM and CNN, making these findings widely applicable as a reference. Furthermore, the methods employed here can guide researchers in assessing and addressing the effects of class imbalance, while considering the data-specific characteristics to optimize imbalance mitigating methods.

Qu Wendi, Balki Indranil, Mendez Mauro, Valen John, Levman Jacob, Tyrrell Pascal N

2020-Sep-23

Class imbalance, Machine learning, Medical imaging, Radiology, X-ray

In Revista da Associacao Medica Brasileira (1992)

Neves Nedy M B C, Bitencourt Flávia B C S N, Bitencourt Almir G V

2020

In European review for medical and pharmacological sciences

Alsharif M H, Alsharif Y H, Chaudhry S A, Albreem M A, Jahid A, Hwang E

2020-Sep

In Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine

STUDY OBJECTIVES : Polysomnography is the gold standard in identifying sleep stages; however, there are discrepancies in how technicians use the standards. Because organizing meetings to evaluate this discrepancy and/or reach a consensus among multiple sleep centers is time consuming, we developed an artificial intelligence (AI) system to efficiently evaluate the reliability and consistency of sleep scoring, and hence the sleep center quality.

METHODS : An interpretable machine learning algorithm was used to evaluate interrater reliability (IRR) of sleep stage annotation among sleep centers. The AI system was trained to learn raters from one hospital, and applied to subjects from the same or other hospitals. The results were compared with the experts' annotation to determine IRR. Intra-center and inter-center assessments were conducted on 679 subjects without sleep apnea from six sleep centers in Taiwan. Centers with potential quality issues were identified by the estimated IRR.

RESULTS : In the intra-center assessment, the median accuracy ranged from 80·3% to 83·3% with the exception of one hospital (designated E) with an accuracy of 72·3%. In the inter-center assessment, the median accuracy ranged from 75·7% to 83·3% when hospital E was excluded from testing and training. The performance of the proposed method was higher for N2, awake, and REM, compared to N1 and N3. The significant IRR discrepancy of hospital E suggested a quality issue. This quality issue is confirmed by the physicians in charge of hospital E.

CONCLUSIONS : The proposed AI system proved effective in assessing IRR and hence the sleep center quality.

Liu Gi-Ren, Lin Ting-Yu, Wu Hau-Tieng, Sheu Yuan-Chung, Liu Ching-Lung, Liu Wen-Te, Yang Mei-Chen, Ni Yung-Lun, Chou Kun-Ta, Chen Chao-Hsien, Wu Dean, Lan Chou-Chin, Chiu Kuo-Liang, Chiu Hwa-Yen, Lo Yu-Lun

2020-Sep-23

inter-center assessments, interrater reliability, intra-center assessments, machine learning, sleep stage scoring

In Psychotherapy research : journal of the Society for Psychotherapy Research

Van Bronswijk Suzanne C, Bruijniks Sanne J E, Lorenzo-Luaces Lorenzo, Derubeis Robert J, Lemmens Lotte H J M, Peeters Frenk P M L, Huibers Marcus J H

2020-Sep-23

cognitive behavioural therapy, depression, external validation, interpersonal psychotherapy, precision medicine, prediction