Hi Donald,
With regards to longitudinal analysis, I am not as familiar with the issue you are mentioning. Perhaps some other person on the list can comment.
With regards to your other comments, I think your reading is correct. However, I should mention that the reported accuracies *really* depends on what structures (cytoarchitectural areas versus functional areas versus macro-anatomical areas) and what accuracy measures (e.g., volume of structures versus overlap of structures, etc.) you are looking at. You probably already know this, but I will elaborate anyhow:
The references by Fischl, Hinds, Yeo (from the previous email) are talking about the accuracy of localizing cortical cytoarchitectural and functional areas. Since freesurfer (and most other software packages) uses macro-anatomy (e.g., sulci, gyri, MR intensity) for processing (e.g, alignment of subjects, segmentation) and macro-anatomy does not predict function/cytoarchitecture completely, we expect the accuracy for cytoarchitectural and functional areas to be bounded by the predictive relationship between macro-anatomy and function/cytoarchitecture. In particular, V1 turns out to be very well-predicted by the cortical folds (calcarine), and so when we align sulci/gyri across subjects, there's very small errors in localizing the boundaries of V1. On the other hand, MT turns out to be not as well predicted by the cortical folds, thus the accuracy of MT alignment after freesurfer processing is lower.
Conversely, for macroanatomical structures (stuff that is visibly seen in MR) like superior temporal sulcus and the thalamus (in aparc+aseg), we expect the accuracy to be much higher. Even then the accuracy for individual structures may vary. For example, thalamus is more accurately segmented than the amygdala. For individual thalamic nuclei, which are less visible (if not invisible!) in 3T MR, the accuracy is probably much less.
You also mentioned that some areas would be larger than the true underlying labels for certain individuals, and some would be smaller. That is certainly the case, but I don't think it is explicitly shown in the papers by Fischl, Hinds, Yeo (although you can probably infer that from other papers listed in the links I sent you). As an extreme example, in one of the experiments in Yeo et al. 2010, "ground truth" functional MT+ for each subject was defined by thresholding the fMRI activation map using a size threshold. Therefore, the way the "ground truth" was set up in that particular experiment, any errors in predicted MT+ is due to errors in predicting MT+ location, rather than size difference. On the other hand, if you look at Figure 3 in Fischl et al, 2002 (Whole Brain Segmentation: Automated Labeling of Neuroanatomical Structures in the Human Brain), there is no obvious systemic differences between volumetric measurement of structures obtained by manual labeling versus automated segmentation in FreeSurfer (circa 2002).
--Thomas
On Tue, Dec 20, 2011 at 2:05 AM, MCLAREN, Donald mclaren.donald@gmail.com wrote:
Excellent references. From my reading of these papers, the boundaries of an individuals subject can shift by up to ~7mm. Meaning that some areas would be largers than the standard labels and others would be smaller.
Is my reading of this correct?
In longitudinal analysis of regional volume measures, do you know of anyway to correct for the potential correlation induced by the mislabelling of nodes that go into each regional volume?
Best Regards, Donald McLaren
D.G. McLaren, Ph.D. Postdoctoral Research Fellow, GRECC, Bedford VA Research Fellow, Department of Neurology, Massachusetts General Hospital and Harvard Medical School Office: (773) 406-2464 ===================== This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is intended only for the use of the individual or entity named above. If the reader of the e-mail is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that you are in possession of confidential and privileged information. Any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this e-mail unintentionally, please immediately notify the sender via telephone at (773) 406-2464 or email.
On Sat, Dec 17, 2011 at 9:28 PM, Thomas Yeo ythomas@csail.mit.edu wrote:
There's some reference of test-retest reliability here: http://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferMethodsCitation, although I doubt it's complete. There's also a section on the wiki of paper evaluating freesurfer: http://surfer.nmr.mgh.harvard.edu/fswiki/#Articles.evaluations
What I know off hand is that for cytoarchitectonics and some function (like MT+ and V1), freesurfer registration accuracy ranges from 2-3mm (V1) to 6-12mm (BA44, BA45, MT+). BA2, BA6, etc in between. Here's the references:
Fischl et al. 2008 Cortical folding patterns and predicting cytoarchitecture Hinds et al. 2008 Accurate prediction of V1 location from cortical folds in a surface coordinate system Yeo et al. 2008 Spherical Demons: Fast Diffeomorphic Landmark-Free Surface Registration Yeo et al. 2010 Learning Task-Optimal Registration Cost Functions for Localizing Cytoarchitecture and Function in the Cerebral Cortex
--Thomas
On Sat, Dec 17, 2011 at 5:49 AM, MCLAREN, Donald mclaren.donald@gmail.com wrote:
I've looked through a number of freesurfer papers, but haven't been able to find the answer to what I am looking for and was hoping someone had the references easily available.
(1) What is the reliability of regional volumes generated from freesurfer over time?
(2) What is the accuracy of labels for these volumes from subject to subject (e.g. how close are the freesurfer labels to cytoarchitecture or manually drawing the labels)?
Best Regards, Donald McLaren
D.G. McLaren, Ph.D. Postdoctoral Research Fellow, GRECC, Bedford VA Research Fellow, Department of Neurology, Massachusetts General Hospital and Harvard Medical School Office: (773) 406-2464 ===================== This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is intended only for the use of the individual or entity named above. If the reader of the e-mail is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that you are in possession of confidential and privileged information. Any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this e-mail unintentionally, please immediately notify the sender via telephone at (773) 406-2464 or email.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Hi Donald,
I have a paper almost published on the longitudinal stream including several repeatability measures (although not everything that you ask for). Let me know if you want to take a peek at it before it goes out. I might be able to send a pre-print version.
Best, Martin
On Tue, 2011-12-20 at 11:37 +0800, Thomas Yeo wrote:
Hi Donald,
With regards to longitudinal analysis, I am not as familiar with the issue you are mentioning. Perhaps some other person on the list can comment.
With regards to your other comments, I think your reading is correct. However, I should mention that the reported accuracies *really* depends on what structures (cytoarchitectural areas versus functional areas versus macro-anatomical areas) and what accuracy measures (e.g., volume of structures versus overlap of structures, etc.) you are looking at. You probably already know this, but I will elaborate anyhow:
The references by Fischl, Hinds, Yeo (from the previous email) are talking about the accuracy of localizing cortical cytoarchitectural and functional areas. Since freesurfer (and most other software packages) uses macro-anatomy (e.g., sulci, gyri, MR intensity) for processing (e.g, alignment of subjects, segmentation) and macro-anatomy does not predict function/cytoarchitecture completely, we expect the accuracy for cytoarchitectural and functional areas to be bounded by the predictive relationship between macro-anatomy and function/cytoarchitecture. In particular, V1 turns out to be very well-predicted by the cortical folds (calcarine), and so when we align sulci/gyri across subjects, there's very small errors in localizing the boundaries of V1. On the other hand, MT turns out to be not as well predicted by the cortical folds, thus the accuracy of MT alignment after freesurfer processing is lower.
Conversely, for macroanatomical structures (stuff that is visibly seen in MR) like superior temporal sulcus and the thalamus (in aparc+aseg), we expect the accuracy to be much higher. Even then the accuracy for individual structures may vary. For example, thalamus is more accurately segmented than the amygdala. For individual thalamic nuclei, which are less visible (if not invisible!) in 3T MR, the accuracy is probably much less.
You also mentioned that some areas would be larger than the true underlying labels for certain individuals, and some would be smaller. That is certainly the case, but I don't think it is explicitly shown in the papers by Fischl, Hinds, Yeo (although you can probably infer that from other papers listed in the links I sent you). As an extreme example, in one of the experiments in Yeo et al. 2010, "ground truth" functional MT+ for each subject was defined by thresholding the fMRI activation map using a size threshold. Therefore, the way the "ground truth" was set up in that particular experiment, any errors in predicted MT+ is due to errors in predicting MT+ location, rather than size difference. On the other hand, if you look at Figure 3 in Fischl et al, 2002 (Whole Brain Segmentation: Automated Labeling of Neuroanatomical Structures in the Human Brain), there is no obvious systemic differences between volumetric measurement of structures obtained by manual labeling versus automated segmentation in FreeSurfer (circa 2002).
--Thomas
On Tue, Dec 20, 2011 at 2:05 AM, MCLAREN, Donald mclaren.donald@gmail.com wrote:
Excellent references. From my reading of these papers, the boundaries of an individuals subject can shift by up to ~7mm. Meaning that some areas would be largers than the standard labels and others would be smaller.
Is my reading of this correct?
In longitudinal analysis of regional volume measures, do you know of anyway to correct for the potential correlation induced by the mislabelling of nodes that go into each regional volume?
Best Regards, Donald McLaren
D.G. McLaren, Ph.D. Postdoctoral Research Fellow, GRECC, Bedford VA Research Fellow, Department of Neurology, Massachusetts General Hospital and Harvard Medical School Office: (773) 406-2464 ===================== This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is intended only for the use of the individual or entity named above. If the reader of the e-mail is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that you are in possession of confidential and privileged information. Any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this e-mail unintentionally, please immediately notify the sender via telephone at (773) 406-2464 or email.
On Sat, Dec 17, 2011 at 9:28 PM, Thomas Yeo ythomas@csail.mit.edu wrote:
There's some reference of test-retest reliability here: http://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferMethodsCitation, although I doubt it's complete. There's also a section on the wiki of paper evaluating freesurfer: http://surfer.nmr.mgh.harvard.edu/fswiki/#Articles.evaluations
What I know off hand is that for cytoarchitectonics and some function (like MT+ and V1), freesurfer registration accuracy ranges from 2-3mm (V1) to 6-12mm (BA44, BA45, MT+). BA2, BA6, etc in between. Here's the references:
Fischl et al. 2008 Cortical folding patterns and predicting cytoarchitecture Hinds et al. 2008 Accurate prediction of V1 location from cortical folds in a surface coordinate system Yeo et al. 2008 Spherical Demons: Fast Diffeomorphic Landmark-Free Surface Registration Yeo et al. 2010 Learning Task-Optimal Registration Cost Functions for Localizing Cytoarchitecture and Function in the Cerebral Cortex
--Thomas
On Sat, Dec 17, 2011 at 5:49 AM, MCLAREN, Donald mclaren.donald@gmail.com wrote:
I've looked through a number of freesurfer papers, but haven't been able to find the answer to what I am looking for and was hoping someone had the references easily available.
(1) What is the reliability of regional volumes generated from freesurfer over time?
(2) What is the accuracy of labels for these volumes from subject to subject (e.g. how close are the freesurfer labels to cytoarchitecture or manually drawing the labels)?
Best Regards, Donald McLaren
D.G. McLaren, Ph.D. Postdoctoral Research Fellow, GRECC, Bedford VA Research Fellow, Department of Neurology, Massachusetts General Hospital and Harvard Medical School Office: (773) 406-2464 ===================== This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is intended only for the use of the individual or entity named above. If the reader of the e-mail is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that you are in possession of confidential and privileged information. Any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this e-mail unintentionally, please immediately notify the sender via telephone at (773) 406-2464 or email.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
freesurfer@nmr.mgh.harvard.edu
-
Martin Reuter -
Thomas Yeo