The literature screening and final selection were performed according to the PRISMA guidelines28,29. This procedure is summarized in the flow diagram (Fig. 1). Applying the PRISMA procedure, a total of eight tools from 24 original articles have been included in the systematic review.
Excluded toolsAccording to the PRISMA guidelines, inclusion and exclusion criteria must be decided before running the systematic search. In the current review, an additional exclusion criterion was added a posteriori: we decided to exclude tools that are no longer available. This decision was motivated by the following reasons. First, when a tool was no longer available, there was no tool-related website either; this made it impossible to collect some of the information required for the present review. Second, a tool that was no longer available was not relevant to our aim to help clinicians and researchers select the most appropriate option for their investigations. Based on this additional exclusion criterion, two tools were excluded.
The first one, ASSESSA, was initially developed to automatically provide a quantification of GM atrophy and white matter (WM) lesion volume. The focus of this tool was the quantification of hippocampal volume through the learning embeddings for atlas propagation (LEAP)34, an algorithm for the quantification of the regional volume which was developed to enrich clinical trials of Alzheimer's disease in the pre-dementia phase. The clinical tool ASSESSA is no longer available.
The second tool to be excluded, called appMRI, was developed to allow for the automatic statistical analysis of hippocampal volume (http://appmri.org/en/). The tool performed an automated segmentation using FreeSurfer software and then provided a numerical output of left and right hippocampal volumes, together with normative values generated using a reference database of age-matched healthy controls. As for ASSESSA, this tool is no longer available.
Included toolsEight neuroimaging-based clinical tools were identified. Their technical characteristics are summarized in Table 1, while more general information, including how to use each tool and their strengths and limitations, is reported in Table 2.
Two of the eight tools (ADABOOST35 and Qure25) are designed to specifically perform a single type of analysis (hippocampus segmentation and gross abnormality identification, respectively). On the contrary, the other six tools (DIADEM36,37, Icobrain38,39,40,41, Jung Diagnostics27,42,43, NeuroQuant24,44,45,46,47,48,49,50,51, Quantib52,53, volBrain54,55) are designed to extract multiple types of information from the data and/or evaluate multiple disorders.
As reported in Table 2, six of the eight tools obtained at least one certification for medical use (DIADEM, Icobrain, Jung Diagnostics, NeuroQuant, Quantib, Qure). The remaining two tools are not approved for medical use. In particular, ADABOOST35 is present on the neuGrid platform56, a web portal which aims to provide automated algorithms to support the diagnostic assessment of individual patients with neurodegenerative disease from neuroimaging data. The second tool which is not approved for medical use is volBrain26,54,55. The website for this tool explicitly states that it was developed for research purposes, and as such does not hold any certification for medical use.
One tool (DIADEM36,37) has no associated references describing the underlying methodology in detail. The references that are mentioned on the website36,37 describe algorithms to perform parcellation and segmentation with better accuracy than previous approaches. However, it is not clear how are these algorithms are incorporated within the overall tool which performs several additional functions e.g. quantification and labeling. For this reason, we do not report the main characteristics of this tool in the following results description, as they are not present in any scientific reference.
Target disordersAll the identified clinical tools have been developed to support the diagnosis of neurological disorders. In particular, five tools are designed to provide quantitative support to the diagnosis of dementia and in particular of Alzheimer's disease (ADABOOST35, Jung Diagnostistics27,43, NeuroQuant45, Quantib53, volBrain26,54), mild cognitive impairment (MCI) (ADABOOST35, Jung Diagnostics27, NeuroQuant48), or other forms of dementia (Jung Diagnostics43). Furthermore, four tools are designed to support the diagnosis of MS (Icobrain38,39,41, Jung Diagnostics42, Quantib52, volBrain55). In addition, one tool (NeuroQuant) has a parallel version called LesionQuant which has been developed to assist the diagnosis of MS. However, no reference to a scientific publication presenting this alternative version is available on the website. Two tools supported the diagnosis of traumatic brain injury (TBI) (Icobrain40 and NeuroQuant46,49,50,51). Finally, one tool can be used to provide support to the diagnosis of temporal lobe epilepsy (TLE) (NeuroQuant44,47), and one tool (Qure25) is designed to identify different types of intracranial hemorrhages and mass effects in the brain.
Type of analysisAll the identified clinical tools have been designed to perform a region of interest (ROI) analysis measuring pre-defined biomarkers for the target disorder. For instance, we know that dementia (in particular Alzheimer's disease) is associated with atrophy of the hippocampus. Accordingly, two tools are specifically designed to focus on hippocampal volume as a biomarker of this disease (ADABOOST35 and Jung Diagnostics27,43). One additional tool is designed for the investigation of the hippocampus but has not been specifically validated in patients with dementia (volBrain54). Other tools support the diagnosis of dementia through the quantification of both hippocampus volume and general atrophy (NeuroQuant45,48, Quantib53, volBrain26). Finally, one tool performs atrophy quantification (Icobrain41) but has only been validated in patients with MS. As dementia might also be associated with metabolic abnormalities, one tool (PETQuant, a variation of NeuroQuant) performs automatic analysis of metabolic and amyloid based positron emission tomography (PET) images. However, no references are available for this tool.
Similarly, the main pathognomonic feature for MS is the presence of inflammatory WM lesions57. Accordingly, five tools are designed to perform the segmentation of WM lesions and to calculate their volume (Icobrain38,39, JungDiagnostic42, NeuroQuant—no reference available, Quantib52, volBrain55). In addition, as MS has recently been described to be associated with GM atrophy, one tool (Icobrain41) also provides atrophy measurements in patients with MS.
Patients with TBI present with evident traumatic lesions in the brain. A tool (Icobrain40) is therefore designed for intracranial lesion segmentation, cistern segmentation and the evaluation of midline shift. However, mild TBI is not associated with gross brain lesions but with subtle progressive atrophy58. Accordingly, a different tool (NeuroQuant46,49,50,51) has been validated to detect atrophy, structures asymmetry and/or progressive atrophy in patients with TBI.
Patients with TLE are prone to suffer from Mesial Temporal Sclerosis (MTS), involving the loss of neurons and scarring of the deepest portion of the temporal lobe, in particular, the hippocampus59. One tool (NeuroQuant44,47) is therefore designed to detect MTS in patients with TLE through the measurement of the hippocampus volume. Finally, one tool (Qure25) identifies gross abnormalities such as tumors and strokes.
Brain imaging typeThe vast majority of the identified tools analyze magnetic resonance images (MRI) data, in particular, T1-weighted images (ADABOOST35, Icobrain38,39,41, Jung Diagnostics27,42,43, NeuroQuant44,45,47,48, Quantib52,53, VolBrain26,54,55). However, there are a few exceptions. Four tools also require the fluid attenuated inversion recovery (FLAIR) acquisition sequence for the segmentation of WM lesions (Icobrain38,39,41, LesionQuant, a parallel version of NeuroQuant with no reference available, Quantib52, volBrain55). One tool (Qure25) analyzes non-contrast computerized tomography (CT) scans, while one tool (Icobrain40) requires CT scan in the case of suspected TBI. Finally, one tool (PETQuant) analyzes images acquired using positron emission tomography.
Validation datasets and strategiesAll the identified tools can be used to perform a cross-sectional analysis, and thus can be applied to support the diagnosis. Two tools (Icobrain38,41 and Neuroquant46) have also been validated on longitudinal data to predict the natural course of the disease. No tools have been validated to predict the longitudinal response to treatment.
Most tools have been validated using MRI data collected from a single dataset, either freely or private. In a small number of cases, validation is based on the use of multiple datasets. For instance, Smeets et al.41 (Icobrain for MS) used three datasets, two of which are private and the third one is publicly available60; Ochs et al.49, Ross et al.50,51 used data from healthy participants and patients with AD that were part of the ADNI dataset (http://adni.loni.usc.edu/) in combination with scans from patients with TBI which were part of a private dataset; volBrain26,54,55 was validated using healthy participants data from IXI (http://brain-development.org/) and from additional publicly available datasets (http://www.nitrc.org/projects/mni-hisub25; http://cobralab.ca/atlases), AD patients data from OASIS (http://www.oasis-brains.org/), infants data from BSTP (http://brain-development.org), MS data from the MSSEG 2016 (https://www.hal.inserm.fr/inserm-01397806). Qure25 was validated combining scans from 20 different private datasets in India. Finally, Biometrica MS42 (the MS version of Jung Diagnostics) combined real and simulated data. In no case, the strategy adopted to deal with the problem of different scanners and/or different acquisition parameters has been described. The strategy used to validate the tools always consisted of comparing the tool performance with the performance of the gold standard. The gold standard is mainly of three types: a ROI manual delineation by an expert; the performance of previously available software; the performance of an expert radiologist in abnormality identification by visual inspection. The tools that have been validated using the first strategy (i.e. comparison with a manual delineation of ROI) are: ADABOOST35, Icobrain for TBI40, NeuroQuant for sub-cortical segmentation45,48, and Quantib for both sub-cortical structure53 and WM lesions52. The tools that have been validated using the second strategy (i.e. comparison with previous software) are: Icobrain for WM lesion segmentation38,39,41, NeuroQuant for atrophy estimation49, volBrain for volumetry26, WM lesion segmentation55, and hippocampus estimation54. The tools that have been validated using the third strategy (i.e. comparison with visual inspection by an expert radiologist) are: Icobrain for WM lesion segmentation38, Jung Diagnostics for both hippocampus27,43 and WM lesion identification42; NeuroQuant for atrophy identification44,47,50,51. The only apparent exception is Qure25 where the performance of the algorithm has been compared with the results of a medical report, which in turn relies on expert visual inspection as well as other clinical data.
Abnormality inferenceAll identified tools included a control group of disease-free individuals to compare the pathological brain. Five out of the eight tools (ADABOOST35; Icobrain38,39,40; Quantib52,53, Qure25, Jung Diagnostics27,43) rely on machine-learning algorithms to detect brain abnormalities as statistical deviation from the average healthy brain. Two tools rely on classical statistics to identify brains whose structures are statistically different in volume from the analogous structure in the average healthy brain: volBrain26,54,55 and NeuroQuant44,45,46,47,48,49,50,51 detect abnormalities if a brain region volume falls below the 5th percentile or above the 95th percentile of the same region in the average brain.
Strengths and limitationsThe identified tools are characterized by important strengths (see Table 2 for a tool specific description of the strengths and limitations). First, the majority of the tools rely on advanced machine-learning algorithms that offer superior ability to detect complex and distributed patterns in the data61,62 (ADABOOST35; Icobrain38,39,40; Quantib52,53; Qure25; Jung Diagnostics27,43). Second, most of the tools have been licensed for medical use, and this undoubtedly presents an important step toward their translational application in real-world clinical settings. Third, the time from image upload to the report receipt is less than an hour. For instance, using volBrain, results are available in 12 min; using NeuroQuant in 8 min; using Icometrix in 1 h.
However, these tools are also characterized by important limitations. First, they are validated for neurological disorders only; no tool is available for supporting the diagnosis of psychiatric disorders to date. Second, each tool performs a ROI analysis to investigate a single disorder of interest; no tool is available for investigating multiple disorders. Third, all these tools but one (Qure25, which relies on 291,732 images) have been validated on a small number of brain images. Although some of them used fairly large datasets to develop some normative model that could be used to detect abnormalities (e.g. n = 20035 for ADABOOST; n = 600 for volBrain26), the dataset used for validating such model tended to be much smaller (n = 7 MCI, n = 7 AD for ADABOOST35; n = 10 AD for volBrain26). Finally, an important limitation common to all the available tools is that none of them account for inter-scanner variability resulting from differences in scanner provider, magnetic field and acquisition parameters. This is of crucial importance to develop flexible tools that are generalizable to "unseen" scanners i.e. scanners that were not used to train the tool.
0 Komentar