January, 2012

Comparison of existing and proposed labeling approaches. In a traditional context, either an expert rater with extensive anatomical knowledge evaluates each dataset (A) or a small set of well-training domain experts who have been instructed by an anatomical expert (B) label each image. Intra- and inter-rater reproducibility analyses are typically performed a per-protocol basis rather than on all datasets. In the proposed WebMILL approach (C), a computer system divides the set of images to be labeled into simple puzzles consisting of a piece of a larger volume and distributes these challenges to a distributed collection of minimally training individuals. Each piece is labeled multiple times by multiple raters. A statistical fusion process simultaneously estimates the true label for each pixel and performance characteristics of each rater.

Foibles, Follies, and Fusion: Web-Based Collaboration for Medical Image Labeling

Jan. 31, 2012—Bennett A Landman, Andrew J Asman, Andrew G Scoggins, John A Bogovic, Joshua A Stein; Jerry L Prince, “Foibles, Follies, and Fusion: Web-Based Collaboration for Medical Image Labeling”, NeuroImage. 2012 Jan 2;59(1):530-9. PMC3195954 † Full text: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3195954/ Abstract Labels that identify specific anatomical and functional structures within medical images are essential to the characterization of...