Improving Clinical Usability of Deep Learning for Radiographic Diagnosis: Learning under Constraints of Specificity, Data, and Annotation
Computer-Aided Diagnostic (CAD) systems play a central role in medical imaging, especially in radiographic research. This thesis tackles three challenges that limit how usable these systems are in real clinical settings: improving sensitivity at the high specificity thresholds clinics actually operate at, using clinical reports to enable self-supervised learning that supports both classification and localization at scale, and improving precise lesion detection when only limited annotations are available.
For the first challenge, the thesis introduces AUCReshaping, an algorithm that specifically boosts sensitivity at high-specificity operating points, better matching how CAD systems are actually used in practice. The second challenge is addressed through vision-language self-supervised learning: Patch-CLIP adapts classification networks to also produce coarse localization, CXR-CML explicitly clusters the vision-language embedding space to improve performance on rare classes and zero-shot classification, and ALBEF-CXR further refines the alignment between visual and language embeddings to strengthen zero-shot learning. For the third challenge, the thesis presents EM-DETR, a detection framework that learns class-specific “exemplar” features for more robust and precise lesion detection, and extends it to reach state-of-the-art performance using less than 10% of the usual annotation, while also exploring its potential as a foundation model for medical detection tasks more broadly.
Across chest X-ray, mammography, and even a credit card fraud classification benchmark, these methods show substantial gains: AUCReshaping improves sensitivity at 98% specificity by 2 to 9 percentage points; Patch-CLIP improves classification by 1 point and sensitivity at a fixed false-positive rate by 3 to 7 points on chest X-ray datasets; CXR-CML gains 7 points in zero-shot classification, and ALBEF-CXR gains an average of 2 points across 13 major findings in a public chest X-ray dataset. In lesion detection, EM-DETR achieves precision gains of 4 to 16 points across different domains, doubling the true positive rate on an out-of-distribution mammogram dataset, and its extended version reaches near-state-of-the-art classification and detection under minimal annotation, with a foundation-model variant converging faster and adding a further 3-point precision gain on both mammogram and chest X-ray data.
Together, these contributions address core limitations in current CAD systems, making them more reliable, scalable, and clinically usable, and pushing the state of the art in AI-driven diagnostic tools for better patient care.