MIC-CUSP: Multimodal Image Correlations for Ultrasound-Based Prostate Cancer Detection

Transrectal b-mode ultrasound images are used to guide pros-tate biopsies but are rarely used for prostate cancer detection. Cancer detection rates on b-mode ultrasound are low due to the low signal-to-noise ratio and imaging artifacts like shadowing and speckles, resulting in missing upto 52% clinically significant cancers in ultrasound-guided biopsies. Since b-mode ultrasound is widely accessible, routinely used in clinical care, inexpensive, and a fast non-invasive imaging modality, ultrasound-based prostate cancer detection has great clinical significance. Here, we present an automated ultrasound-based prostate cancer detection method, MIC-CUSP (Multimodal Image Correlations for Cancer detection on Ultra-Sound leveraging Pretraining with weak labels). First, MIC-CUSP learns richer imaging-inspired ultrasound biomarkers by leveraging registration-independent multimodal image correlations between b-mode ultrasound and two unaligned richer imaging modalities, Magnetic Resonance Imaging (MRI) and post-operative histopathology images. Second, MIC-CUSP uses the richer imaging-inspired ultrasound biomarkers as inputs to the cancer detection model to localize cancer on b-mode ultrasound images, in absence of MRI and histopathology images. MIC-CUSP addresses the lack of large accurately labeled ultrasound datasets by pretraining with a large public dataset of 1573 b-mode ultrasound scans and weak labels, and fine-tuning with 289 internal patients with strong labels. MIC-CUSP was evaluated on 41 patients, and compared with four clinician-readers with 1–12 years of experience. MIC-CUSP achieved patient-level Sensitivity and Specificity of 0.65 and 0.81 respectively, outperforming an average clinician-reader.