Sub-Challenge #3

Challenge name

The partial volume effect: How stable is my biomarker?

Purpose and relevance of the challenge

With this challenge we aim to understand how stable are current dMRI metrics to partial volume effects using state-of-the-art measurements. Participants will be provided with dMRI measurements at different resolutions for several voxels selected from different white matter (WM) regions and are asked to return metrics(s) of their choice for each voxel. We encourage metrics, or potential biomarkers, from all signal models or multi-compartment models. Examples of such metrics may include MD, FA, kurtosis, etc. The provided data includes different types of acquisition strategies, such as multi-shell and DSI-like pulsed gradient spin-echo (PGSE)1, as well as double diffusion encoding (DDE)2and double oscillating diffusion encoding (DODE)3, suitable for a large number of fit approaches. In this challenge we will evaluate the stability of current dMRI metrics to partial volume effects. While the overall winner of this subchallenge is reflected by the least change with increasing data downsampling in combination with the maximum differentiation between voxel mostly within white matter and voxels affected by partial volume effects, we will also investigate the ability of different metrics to discriminate ROIs, their reproducibility (through the intra-class correlation coefficient) as well as the correlation between metrics of a given technique. We will also learn whether it is possible to account for partial volume effects in the diffusion models as well as which ROIs are more affected by varying the resolution. This knowledge will enhance our current understanding of dMRI contrast.

Datasets

The challenge includes in-vivo braindata sampled with PGSE in a human volunteer, and with DDE / DODE in mice, ex-vivo. This sub-challenge employs the same datasets as sub-challenge 1, nevertheless the chosen voxels are different.

The two provided datasets are composed as follows:

  • In-vivo human dataset. PGSE acquisition. Data from 6 ROIs with 3 voxels each, representative of different white matter configurations (3 mostly within WM and three at the border of WM/CSF, WM/GM, WM/DGM), extracted from the MASSIVE dataset (1921 unique diffusion-weighted volumes)4is provided. For each voxel, signals include 1300 unique data pointsacquired with a multi-shell strategy, and 621 data pointsacquired with a DSI-like strategy. The two acquisitions were NOT performed with identical imaging parameters. Participants can choose whether they prefer to work with the multi-shell orthe DSI-like acquisition.
  • Ex-vivo mouse dataset. DDE and DODE acquisitions. Data from 6 ROIs with 3 voxels each representative of different white matter configurations (3 mostly within WM and three at the border of WM/CSF, WM/GM, WM/DGM), acquired in a mouse brain ex-vivo, will be provided. The full dataset consists of DDE with 2 different diffusion times and DODE with 5 different frequencies, with 5 b-values and 72 directions each (2520 diffusion weighted volumes in total)3.
  • To study the effect of partial volume, the two datasets have also been downsampled from their original resolution, with the voxel volume increasing by factors of 1.75, 2.75 and 4. The ROIs are placed in similar locations for each resolution.

Link to Data : https://www.synapse.org/#!Team:3388309

Participation (Data given to the participants)

Participants will be provided with the following files. Sample scripts to load the data will be provided for MATLAB.

Note that for each acquisition type, we include files for (1) a protocol description, (2) the acquisition parameters, and (3) the MR signal. For example, the dataset for each sequence is provided in a text file and consists of M columns, where M = 72 is the total number of voxels to be analysed (18 voxels x 4 resolutions – original resolution + 3 downsampled datasets) and N rows, where N is the number of measurements. Note: the voxels are placed in random order.

  1. PGSE_ProtocolDescription.txt: Description of the acquisition parameters for the PGSE sequences
  2. PGSE_shells_AcqParams.txt: Acquisition parameters for all measurements of the PGSE shell dataset. It is a NxA matrix with N rows and A columns, where N is the number of measurements and A is the number of sequence parameters.
  3. PGSE_shells_Signals.txt: Dataset for PGSE shell sequence
  4. PGSE_grids_AcqParams.txt: Acquisition parameters for all measurements of the PGSE grids dataset.
  5. PGSE_grids_Signals.txt: Dataset for PGSE grids sequence
  6. DDE_ProtocolDescription.txt: Description of the acquisition parameters for the DDE sequences
  7. DDE_AcqParams.txt: Acquisition parameters for all measurements of the DDE dataset.
  8. DDE_Signals.txt: Dataset for DDE sequences
  9. DODE_ProtocolDescription.txt: Description of the acquisition parameters for the DODE sequences
  10. DODE_AcqParams.txt: Acquisition parameters for all measurements of the DODE dataset.
  11. DODE_Signals.txt: Dataset for DODE sequences

Submission

Participants are asked to submit a list of voxel-wise metric(s) from a data analysis technique of their choice applied to the provided signals of the human and/or mouse datasets. The metrics for the human and mouse datasets will be evaluated separately. For each dataset, the metrics can be derived based on any of the acquisition protocols provided, using all measurements or a subset of them. Additionally, submissions can contain one, or many, biomarkers of choice from a given technique (all biomarkers will be evaluated separately, but we encourage all measures from a given signal or multi-compartment model to be submitted, to provide more information for the big picture)

Submission for all sub-challenges must be submitted as a single compressed folder with a unique name for each submission (we recommend TeamName_SubmissionName.zip). The results from each data-analysis technique should be send as different submissions. For example, if a team submits metrics for DTI and Kurtosis, then they will make two submissions.Similarly, if the same technique is applied to different subsets of data, e.g. DTI applied to b 500 and to b 1000, it should also be sent as two different submissions with appropriate naming. The information regarding each model should be included in the info.txt file.

Each submission consists of the following files:

  • The dMRI metrics for each voxel are submitted as text files named metrics_{SEQ}.txt, where SEQ = {‘PGSE_grids’, ’PGSE_shells’, ’DDE’, ’DODE’}. A different metrics file should be submitted for each diffusion acquisition. The files should contain of a PxM matrix with P rows and M columns, where M is the number of voxels (72) and P is the number of metrics.
  • The submission should also include a description of the metrics in a text file with the name description_{SEQ}.txt. Each line in these files should contain a brief description of the corresponding metric and the model, e.g.: Mean Diffusivity from a Linear DT fit.
  • The submission should also include a info.txt description/reference to their analysis technique(s) used for computing the dMRI metrics. In addition to a description of the submission, for all sub-challenges, info.txt should also contain the (1) submission name, (2), submission abbreviation, (3), team name, (4) team members who made meaningful contributions, (5) member affiliation, (6) all relevant citations, (7) observations (optional), and (8) relevant discussion points (optional).

The provided estimates can include any metric (e.g. microstructure related, kurtosis, anisotropies, diffusion length scales, time-dependencies, etc) derived from a single technique. Submissions from multiple techniques should be made separately. This will help us better understand how different models cope with partial volume, not just designate a winner for the challenge.

Evaluation

Each submission will be evaluated by looking at the (1) correlation between a given metric and the downsampling factor in ROIs 1-3 which are mostly within white matter regions, as well as (2) the difference between the ROIs 1-3 (mostly within WM) and ROIs 4-6 (affected by partial volume with CSF, GM and DGM), at the lowest resolution. Specifically, for (1) we will compute the slope of the normalized metrics with respect to the downsampling factor for each ROI, then compute the mean absolute values across ROIs. Note: as normalized metrics we use the z-scores (i.e. for each ROI we first subtract the mean across different voxels and resolutions, then divide by standard deviation).For (2) we use only the data from the lowest resolution. First, we compute the absolute difference between the average metrics in ROIs 1-3 and ROI 4-6 and we normalize it by the sum of the standard deviations in these two groups of ROIs. To have similar range for the score in (1)and (2), the difference in (2) is divided by 2, an empirical factor based on a linear DTI fit. The final metric is the score in (1) – (2), and the metric with the lowest overall score is the winner, as it minimizes the correlation with resolution and maximizes the difference between border voxels and voxels mostly within white matter at low resolution. Although the evaluation is focused on metric stability, we will also try to learn more about the general properties of different dMRI techniques, for example how correlated are the metrics from the same model or from different models, and their ability to discriminate between ROIs.

How to get the data

Please see “Registration and Data Access” Page.

Sub-Challenge Chairs

Andrada Ianus <University College London>

Noam Shemesh <Champalimaud Centre for the Unknown>

References

  1. Stejskal, E. O. &amp; Tanner, J. E. Spin Diffusion Measurements: Spin Echoes in the Presence of a Time-Dependent Field Gradient. J. Chem. Phys. 42, (1965).
  1. Shemesh, N., Özarslan, E., Komlosh, M. E., Basser, P. J. &amp; Cohen, Y. From single-pulsed field gradient to double-pulsed field gradient MR: Gleaning new microstructural information and developing new forms of contrast in MRI. NMR Biomed. 23, 757–780 (2010).
  1. Ianuş, A., Jespersen, S., Serradas Duarte, T., Alexander, D.C., Drobnjak, I., Shemesh, N. Accurate estimation of microscopic diffusion anisotropy and its time dependence in the mouse brain. Neuroimage. 183: 934-949 (2018).
  2. Froeling, M., Tax, C. M. W., Vos, S. B., Luijten, P. R. &amp; Leemans, A. ‘MASSIVE’ brain dataset: Multiple acquisitions for standardization of structural imaging validation and evaluation. Magn. Reson. Med. 77, 1797–1809 (2017).