Practices and problems with publicly available medical datasets
Theo Sourget
Medical imaging papers often focus on methodology, but the quality of the algorithms and the validity of the conclusions are highly dependent on the datasets used. As creating datasets requires a lot of effort, researchers often use publicly available datasets, there is however some issues regarding the tracking of their usage or the evaluation of models trained with them due to potential shortcut learning. In this presentation, I’ll first present a study on the usage of 20 publicly available medical datasets in papers from MICCAI and MIDL detailing the pipeline we developed to detect their presence in scientific articles. In a second part, I’ll present our work on shortcut learning. We challenged the capacity of convolutional neural networks (CNN) to classify chest X-rays and eye fundus images while masking out clinically relevant parts of the image.