Benchmarking Image Transformers for Prostate Cancer Detection from Ultrasound Data

PURPOSE: Deep learning methods for classifying prostate cancer (PCa) in ultrasound images typically employ convolutional networks (CNNs) to detect cancer in small regions of interest (ROI) along a needle trace region.

However, this approach suffers from weak labelling, since the ground-truth histopathology labels do not describe the properties of individual ROIs.

Recently, multi-scale approaches have sought to mitigate this issue by combining the context awareness of transformers with a CNN feature extractor to detect cancer from multiple ROIs using multiple-instance learning (MIL).

In this work, we present a detailed study of several image transformer architectures for both ROI-scale and multi-scale classification, and a comparison of the performance of CNNs and transformers for ultrasound-based prostate cancer classification.

We also design a novel multi-objective learning strategy that combines both ROI and core predictions to further mitigate label noise.

METHODS: We evaluate 3 image transformers on ROI-scale cancer classification, then use the strongest model to tune a multi-scale classifier with MIL.

We train our MIL models using our novel multi-objective learning strategy and compare our results to existing baselines.

RESULTS: We find that for both ROI-scale and multi-scale PCa detection, image transformer backbones lag behind their CNN counterparts.

This deficit in performance is even more noticeable for larger models.

When using multi-objective learning, we can improve performance of MIL, with a 77.9% AUROC, a sensitivity of 75.9%, and a specificity of 66.3%.

CONCLUSION: Convolutional networks are better suited for modelling sparse datasets of prostate ultrasounds, producing more robust features than transformers in PCa detection.

Multi-scale methods remain the best architecture for this task, with multi-objective learning presenting an effective way to improve performance.

;Comment: early draft, 7 pages; Accepted to SPIE Medical Imaging 2024