Abnormality detection on musculoskeletal radiographs

Type: MA thesis

Status: finished

Date: May 6, 2019 - October 31, 2019

Supervisors: Florian Kordon, Andreas Maier, Jonathan Sperl (Siemens Healthcare GmbH), Lei Wang (Siemens Healthcare GmbH), Sailesh Conjeti (Siemens Healthcare GmbH)

Thesis Description

The primary objective of this thesis project is to develop an algorithm that can determine whether a musculoskeletal X-ray study is normal or abnormal. For this purpose, we only consider X-rays of the upper extremities including the shoulder, humerus, elbow, forearm, wrist, hand, and finger. By abnormalities we consider fractures, hardware, degenerative joint diseases, lesions, subluxations, and other deviations from the standard structural composition and morphology. Given an X-ray image as an input, the devised algorithm should output a labeled image which indicates the presence or absence of an abnormality.  Such a system could be used to enhance the confidence of the radiologist or prioritize subsequent analysis and treatment options.

The task to determine abnormality on musculoskeletal radiographs is particularly critical since more than 1.7 billion people around the globe are affected by musculoskeletal conditions [12]. Since a radiograph is the cheapest, best available and usually the first measure to detect musculoskeletal abnormalities, automatic detection and localization of such potential abnormalities enables a faster initial diagnosis, saves valuable time for physicians, and reduces the number of subsequent diagnostic treatments required on the patient.  This will also reduce the work pressure and fatigue of radiologists [10] which is caused by overwhelming number of X-ray studies they have to diagnose every day [11].

In this project we will use a large public data set called ‘MURA-v1.1’ published by Stanford Machine Learning Group of Stanford University [1]. The data set consists of 14,863 studies from 12,173 patients with a total of 40,561 multi-view radiographic images. Board-certified radiologists from Stanford Hospital manually labeled the radiographs as normal or abnormal. Out of 14,863 studies 9,045 are normal and 5,818 are abnormal.

The project is structured into three parts. First, a learning-based classification algorithm is used to predict whether a radiograph is normal or abnormal [1,2]. Second, anatomical information derived from the dataset’s annotation is incorporated to additionally predict the anatomical origin of the radiograph [3,4,6,7,8]. In a last step, the abnormality is localized and visualized by incorporating the results from the previous steps in combination with targeted feature space analysis. All components should then be combined to a framework capable to predict, localize and visualize musculoskeletal abnormality. Algorithmic development is based on recent advances in deep learning techniques building upon the DenseNet [9] and ResNet [13] neural network architecture. A main aspect of the work is the conception and implementation of an integration strategy of additional anatomical information. It shall also be analyzed to what extent this information can support and improve the classification of abnormal and normal radiographs. Prior work of multi-task/multi-label optimization is investigated and examined for applicability to this project’s task [3,4,5,6,7].  The project is fixed to a six-month period timeline and will be concluded by a detailed project report. Technical implementation of the prototype will be performed within the PyTorch environment for the Python programming language.

 

References

  1. Rajpurkar P., Irvin J., Bagul A., Ding D., Duan T., Methta H., Yang B., Zhu K., Laird D., Ball R., Langlotz C., Shpanskaya K., Lungren M., Ng A. , “MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs” 1st Conference on Medical Imaging with Deep Learning (MIDL 2018)
  2. Guendel S., Grbic S., Gerogescu B., Zhou K., Ludwig R., Meier A., “Learning to recognize abnormalities in chest x-rays with location aware dense networks.” arxiv preprint arXiv:1803.04565 ,2018
  3. Guendel S., Ghesu F., Grbic S., Gibson E., Gerogescu B., Maier A.,“ Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels “ arxiv preprint arXiv:1905.06362 ,2019
  4. Yang, X., Zeng, Z., Yeo, S.Y., Tan, C., Tey, H.L., Su, Y., “A novel multi-task deep learning model for skin lesion segmentation and classification.” arxiv preprint arXiv:1703.01025 ,2017
  5. Vesal S., Ravikumar N., Maier A., ‘‘A Multi-task Framework for Skin Lesion Detection and Segmentation’’ arxiv preprint arXiv:1808.01676 ,2018
  6. Kendall A., Gal Y., Cipolla R.,”Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics” arxiv preprint arXiv:1705.07115 ,2017
  7. Vandenhende S., Brandere B., Gool Luc., “Branched Multi-Task Networks: Deciding What Layers To Share“ arxiv preprint arXiv:1904.02920 ,2019
  8. Berlin L., ”Liability of interpreting too many radiographs.” American Journal of Roentgenology, 175(1):17–22, 2000
  9. Huang G., Liu Z., Weinberger K.Q., and van der Maaten, Laurens, “Densely connected convolutional networks.” arXiv preprint arXiv:1608.06993, 2016.
  10. Lu Y., Zhao S., Chu P.W., and Arenson R.L., “An update survey of academic radiologists’ clinical productivity.” Journal of the American College of Radiology, 5(7):817–826, 2008.
  11. Nakajima Y., Yamada K., Imamura K., and Kobayashi K.. ,”Radiologist supply and workload: international comparison.” Radiation medicine, 26(8):455–465, 2008.
  12. URL http://www.boneandjointburden.org/2014-report.
  13. He K., Zhang X., Ren S., Sun J., “Deep Residual Learning for Image Recognition” arxiv preprint arXiv:1512.03385