Skip to main content

Medical-image Analysis and Statistical Interpretation Lab

Scalable quality control on processing of large diffusion-weighted and structural magnetic resonance imaging datasets

Posted by on Thursday, August 28, 2025 in News.

Michael E. Kim, Chenyu Gao, Nancy R. Newlin, Gaurav Rudravaram, Aravind R. Krishnan, Karthik Ramadass, Praitayini Kanakaraj, Kurt G. Schilling, Blake E. Dewey, David A. Bennett, Sid O’Bryant, Robert C. Barber, Derek Archer, Timothy J. Hohman, Shunxing Bao, Zhiyuan Li, Bennett A. Landman, Nazirah Mohd Khairi, The Alzheimer’s Disease Neuroimaging Initiative, The HABS-HD Study Team. “Scalable quality control on processing of large diffusion-weighted and structural magnetic resonance imaging datasets.” PLoS One 2025;20:e0327388. https://doi.org/10.1371/JOURNAL.PONE.0327388.

 

Fig 4. QC app interface. The proposed QC app enforces consistent visualization of pipelines outputs through uniformly generated QC PNG documents for each respective pipeline. The homogenous format of the documents and high success rate for pipelines allow QC users to very quickly cycle through PNGs to catch any abnormalities. Moving through PNG documents can be done either manually with the arrow keys or automatically through the montage feature of the app (not shown). A counter is maintained (red box) so the user can know how many documents are left to view. Most outputs are expected to be good, and thus all QC results are initialized as “Yes” (passes QC) in order to minimize the manual effort in reporting. If the output is not satisfactory, the user can click the corresponding button to indicate their decision (yellow box) and articulate the reason for their verdict with some accompanying text (green box). Any changes are automatically pushed to a CSV document that maintains structured information about the results of the QC for a pipeline and dataset. All QA CSV results documents are formatted the same way and can thus be easily merged across pipelines and datasets so that QC decisions can be shared among team members and collaborators in an easily parseable format.

 

Purpose: Thorough quality control (QC) can be time consuming when working with large-scale medical imaging datasets, yet necessary, as poor-quality data can lead to erroneous conclusions or poorly trained machine learning models. Most efforts to reduce data QC time rely on quantitative outlier detection, which cannot capture every instance of algorithm failure. Thus, there is a need to visually inspect every output of data processing pipelines in a scalable manner.

Approach: We design a QC pipeline that allows for low time cost and effort across a team setting for a large database of diffusion-weighted and structural magnetic resonance images. Our proposed method satisfies the following design criteria: 1.) a consistent way to perform and manage quality control across a team of researchers, 2.) quick visualization of preprocessed data that minimizes the effort and time spent on the QC process without compromising the condition/caliber of the QC, and 3.) a way to aggregate QC results across pipelines and datasets that can be easily shared.In addition to meeting these design criteria, we also provide a comparison experiment of our method to an automated QC method for a T1-weighted dataset of N=1560 images and an inter-rater variability experiment for several processing pipelines.

Results: The experiments show mostly high agreement among raters and slight differences with the automated QC method.

Conclusion: While researchers must spend time on robust visual QC of data, there are mechanisms by which the process can be streamlined and efficient.