Dear FreeSurfer users,
We wanted to make you aware of a paper recently published in Neuron by Makris et al. (Neuron, 2008, volume 60 issue 1 pp.174 - 188) that may prompt questions by reviewers of studies employing FreeSurfer. Please find attached the figure that the authors published in this paper depicting some inaccurate FreeSurfer surfaces (the top image labeled "CV-FS" uses manual segmentations carried out by some of the authors followed by a surface-deformation that we provided for them, and "FS" on the bottom is the automated FreeSurfer result). Since publication the authors provided us access to the original data. Unfortunately it seems that many of the inaccuracies were probably a consequence of sub-optimal motion correction as well as averaging of multiple T1 volumes in the presence of substantial between-scan subject motion.
Obviously it would have been better if this problem had been identified and communicated to us prior to publication, as we are always happy to help improve the results with any users of our tools. We also feel that this kind of comparison is best done systematically and quantitatively rather than using arbitrary slices that may illustrate a desired point.
We have already been told that in some quarters this paper is being taken to show that automated methods such as FreeSurfer should be used very cautiously. Because of this, we communicated with the authors, and after some discussion they agree that these results are not representative of a widespread inaccuracy of our tools, but rather a localized incident indicative of poor motion correction. We have contributed a joint comment to Neuron to this effect that includes a re-analysis of the data using only a single acquisition per subject to remove motion-correction artifacts, the results of which can be found at:
http://surfer.nmr.mgh.harvard.edu/fswiki/NeuronMakris
We think that it is important to communicate this to a wider audience of FreeSurfer users and readers of relevant journals. Certainly all software has failure modes and users are always encouraged to carefully examine their data. However, if anyone feels that in their experience FreeSurfer surfaces are significantly more accurate than those shown in the attached figure, we encourage you to contribute a comment at neuroncomments@cell.com via email. Feel free to email us if you have any questions.
The FreeSurfer Team