ArXiv Preprint
Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice
for medical image understanding tasks. However, DNNs are notoriously fragile to
the class imbalance in image classification. We further point out that such
imbalance fragility can be amplified when it comes to more sophisticated tasks
such as pathology localization, as imbalances in such problems can have highly
complex and often implicit forms of presence. For example, different pathology
can have different sizes or colors (w.r.t.the background), different underlying
demographic distributions, and in general different difficulty levels to
recognize, even in a meticulously curated balanced distribution of training
data. In this paper, we propose to use pruning to automatically and adaptively
identify \textit{hard-to-learn} (HTL) training samples, and improve pathology
localization by attending them explicitly, during training in
\textit{supervised, semi-supervised, and weakly-supervised} settings. Our main
inspiration is drawn from the recent finding that deep classification models
have difficult-to-memorize samples and those may be effectively exposed through
network pruning \cite{hooker2019compressed} - and we extend such observation
beyond classification for the first time. We also present an interesting
demographic analysis which illustrates HTLs ability to capture complex
demographic imbalances. Our extensive experiments on the Skin Lesion
Localization task in multiple training settings by paying additional attention
to HTLs show significant improvement of localization performance by
$\sim$2-3\%.
Ajay Jaiswal, Tianlong Chen, Justin F. Rousseau, Yifan Peng, Ying Ding, Zhangyang Wang
2022-12-06
