Cross-Modality Domain Adaptation for Medical Image Segmentation
Announcements
A new classification task is introduced in the 2022 edition!
The 2022 edition extends the segmentation task by including multi-institutional data!
Challenge participants will have the opportunity to submit their methods as part of the post-conference MICCAI BrainLes 2022 proceedings.
Aim
This challenge proposes is the second edition of the first medical imaging benchmark of unsupervised cross-modality Domain Adaptation approaches (from contrast-enhanced T1 to high-resolution T2).
Motivation
Domain Adaptation (DA) has recently raised strong interests in the medical imaging community. By encouraging algorithms to be robust to unseen situations or different input data domains, Domain Adaptation improves the applicability of machine learning approaches to various clinical settings. While a large variety of DA techniques has been proposed, most of these techniques have been validated either on private datasets or on small publicly available datasets. Moreover, these datasets mostly address single-class problems. To tackle these limitations, the crossMoDA challenge introduced the first large and multi-class dataset for unsupervised cross-modality Domain Adaptation. Compared to the previous crossMoDA instance, which made use of single-institution data and featured a single segmentation task, the 2022 edition extends the segmentation task by including multi-institutional data and introduces a new classification task.
Tasks
Participants are free to choose whether they want to focus only on one or both tasks.
Task 1
The goal of the segmentation task (Task 1) is to segment two key brain structures (tumour and cochlea) involved in the follow-up and treatment planning of vestibular schwannoma (VS). The segmentation of these two structures is required for radiosurgery, a standard VS treatment. Moreover, tumour volume measurement has also been shown to be the most accurate measurement for evaluating VS growth. The diagnosis and surveillance in patients with VS are commonly performed using contrast-enhanced T1 (ceT1) MR imaging. However, there is growing interest in using non-contrast imaging sequences such as high-resolution T2 (hrT2) imaging, as it mitigates the risks associated with gadolinium-containing contrast agents. Furthermore, in addition to improving patient safety, hrT2 imaging is 10 times more cost-efficient than ceT1 imaging.
Task 2
The goal of the classification task (Task 2) is to
automatically classify hrT2 images with VS according to the Koos grade.
The Koos grading scale is a classification system for VS that
characterises the tumour and its impact on adjacent brain structures
(e.g., brain stem, cerebellum). The Koos classification is commonly
determined to decide on the treatment plan (surveillance, radiosurgery,
open surgery). Similarly to the VS segmentation, Koos grading is
currently performed on ceT1 scans, but hrT2 could be used. For this
reason, we propose an unsupervised cross-
modality classification benchmark (from ceT1 to hrT2) that aims to
determine the Koos grade on hrT2 scans automatically. Only pre-operative
data is used for this task. Again, multi-institutional scans from
centres in London, UK and Tilburg, NL are used in this task.
Evaluation
Task 1
Classical semantic segmentation metrics, in this case, the Dice Score (DSC) and the Average Symmetric Surface Distance(ASSD), will be used to assess different aspects of the performance of the region of interest. These metrics are implemented here. The metrics (DSC, ASSD) were chosen because of their simplicity, their popularity, their rank stability, and their ability to assess the accuracy of the predictions.
Participating teams are ranked for each target testing subject, for each evaluated region (i.e., VS and cochlea), and for each measure (i.e., DSC and ASSD). The final ranking score for each team is then calculated by firstly averaging across all these individual rankings for each patient (i.e., Cumulative Rank), and then averaging these cumulative ranks across all patients for each participating team.
Task 2
Participating teams are ranked based on their Macro-averaged mean absolute error (MA-MAE). Macro-averaged mean absolute error is well-designed for ordinal and imbalanced classification problems. It has been used successfully in other challenges such as SemEval-2017.
Rules
Participants are free to choose whether they want to focus only on one or both tasks.
- No additional data is allowed, including the data released on TCIA and pre-trained models.
The use of a generic brain atlas is tolerated as long as its use is made clear and justified.
Example of tolerated use cases:
- Spatial normalisation to MNI space
- Use of classical single-atlas based tools (e.g., SPM with the standard tissue probability map )
Example of cases that are not allowed:
- Multi-atlas registration based approaches in the target domain - No additional annotations are allowed.
- Manual selection of pseudo-labels is not allowed.
- Participants are free to use the data from one task for the other task.
- Models can be adapted (trained) on the target domain (using the provided target training set) in an unsupervised way, i.e. without labels.
- The participant teams will be required to release their training and testing code and explain how they fine-tuned their hyper-parameters. Note that the code can be shared with the organizers only as a way to verify validity, and if needed, NDAs can be signed.
- The top 3 ranked teams (for each task) will be required to submit their training and testing codes in a docker container for verification after the challenge submission deadline in order to ensure that the challenge rules have been respected.
Feel free to contact us using the forum (preferred option) or directly with questions.
Timeline
| 2nd May 2022: | Release of the training and validation data (see data page) |
| 12th May 2022: |
Start of the validation period. Participants are invited to submit their predictions on the validation dataset |
| 5th August 2022: |
Start of the evaluation period. |
| 10th August 2022: | End of the validation phase |
| 15th August 2022: | End of the evaluation period |
| 18th September 2022: | Challenge results are announced at MICCAI 2022 |
| 30th October 2022: | Participants are invited to submit their methods to the MICCAI 2022 BrainLes Workshop. |
| December 2022: | Submission of a joint manuscript summarizing the results of the challenge to a high-impact journal in the field (e.g., TMI, MedIA) |