Skip to main content

Montage based 3D Medical Image Retrieval from Traumatic Brain Injury Cohort using Deep Convolutional Neural Network

Posted by on Monday, December 10, 2018 in Big Data, Computed Tomography, Deep Learning.

Cailey I. Kerley, Yuankai Huo, Shikha Chaganti, Shunxing Bao, Mayur B. Patel, Bennett A. Landman. “Montage based 3D Medical Image Retrieval from Traumatic Brain Injury Cohort using Deep Convolutional Neural Network.” In SPIE Medical Imaging, International Society for Optics and Photonics, 2019.

Full text: NIHMSID


Brain imaging analysis on clinically acquired computed tomography (CT) is essential for the diagnosis, risk prediction of progression, and treatment of the structural phenotypes of traumatic brain injury (TBI). However, in real clinical imaging scenarios, entire body CT images (e.g., neck, abdomen, chest, pelvis) are typically captured along with whole brain CT scans. For instance, in a typical sample of clinical TBI imaging cohort, only ~15% of CT scans actually contain whole brain CT images suitable for volumetric brain analyses; the remaining are partial brain or non-brain images. Therefore, a manual image retrieval process is typically required to isolate the whole brain CT scans from the entire cohort. However, the manual image retrieval is time and resource consuming and even more difficult for the larger cohorts. To alleviate the manual efforts, in this paper we propose an automated 3D medical image retrieval pipeline, called deep montage-based image retrieval (dMIR), which performs classification on 2D montage images via a deep convolutional neural network. The novelty of the proposed method for image processing is to characterize the medical image retrieval task based on the montage images. In a cohort of 2000 clinically acquired TBI scans, 794 scans were used as training data, 206 scans were used as validation data, and the remaining 1000 scans were used as testing data. The proposed achieved accuracy=1.0, recall=1.0, precision=1.0, f1=1.0 for validation data, while achieved accuracy=0.988, recall=0.962, precision=0.962, f1=0.962 for testing data. Thus, the proposed dMIR is able to perform accurate CT whole brain image retrieval from large-scale clinical cohorts.

Keywords: image retrieval, computed tomography, TBI, CT, traumatic brain injury, deep learning

The dMIR whole brain CT scan retrieval method is presented. The 3D CT scans were resampled to in-plane resolution 512 × 512. If the slice number zis larger than 36, the volume is resampled to 512 × 512 × 36. Otherwise, the 3D volume is resampled to 512 × 512 × zand zero padding is performed to achieve a 512 × 512 × 36 volume. Next, the 512 × 512 × 36 volume was used to form the 2D montage figures. Finally, the 2D montage figures were used to train and test a deep neural network using a 50 layer ResNet.

Tags: , , ,