recon-all freezes during mri_fix_topology, FS6.0, Linux, using T2pial and FLAIRpial flags
Hello everyone,
We recently installed FreeSurfer 6.0 at our site and are currently working out which flags and options to use. To learn more about what would suit our needs, we ran a couple of subjects using various flag combinations in order to evaluate the output. One run was initiated on Tuesday Mar 28, and as Friday Mar 31, the recon-all script still had not completed. We obtained similar results on trying to process it again, cf. the attached log file, and also with other subjects on other workstations. It seems that the mri_fix_topology binary is causing this issue -- the top utility indicates that four instances (one for each subject-flag combination, four in total) were running at full blast (100% CPU usage), and recon-all.log reveals that recon-all is waiting for this step to conclude.
I have searched the list, but the posts containing similar issues all seemed to pertain to much older versions of FreeSurfer.
FreeSurfer was invoked using the following command:
$ parallel -a recon-all_test3.txt -j32
The contents of recon-all_test3.txt:
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -T2 /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/sorted/Sag_T2_CUBE_1iso/z4884 -s top3T_0186.fs6.3T.T2pial -all -3T -parallel -T2pial
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -FLAIR /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_CUBE_Flair_TR8/z360 -s top3T_0186.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -FLAIR dicom/top3T_0065/Sag_CUBE_Flair_TR8/Sag_CUBE_Flair_TR8-1.dcm -s top3T_0065.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -T2 dicom/top3T_0065/Sag_T2_CUBE_1iso/Sag_T2_CUBE_1iso-1.dcm -s top3T_0065.fs6.3T.T2pial -all -3T -parallel -T2pial
I would be grateful for any and all clues as to what's wrong here.
1) FreeSurfer version: freesurfer-Linux-centos6_x86_64-stable-pub-v.6.0.0-2beb96c
2) Platform: Ubuntu LTS 14.04.
3) uname -a: Linux picasso 3.13.0-79-generic #123-Ubuntu SMP Fri Feb 19 14:27:58 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
4) recon-all.log: See attachment.
Regards Eirik Svela Institute of Psychiatry, University of Oslo
So these 4 subjects were all processed at the same time, on the same machine, using the parallel flag, and none of them ran to completion? How many cores does this machine have? If you upload one of the subjects I will take a look.
https://surfer.nmr.mgh.harvard.edu/fswiki/FtpFileExchange
I will say that I have seen instances when fs will take days (2 or more) on subjects with really bad quality data. But those were extreme cases.
-Zeke
On 04/03/2017 08:58 AM, Eirik Wixøe Svela wrote:
Hello everyone,
We recently installed FreeSurfer 6.0 at our site and are currently working out which flags and options to use. To learn more about what would suit our needs, we ran a couple of subjects using various flag combinations in order to evaluate the output. One run was initiated on Tuesday Mar 28, and as Friday Mar 31, the recon-all script still had not completed. We obtained similar results on trying to process it again, cf. the attached log file, and also with other subjects on other workstations. It seems that the mri_fix_topology binary is causing this issue -- the top utility indicates that four instances (one for each subject-flag combination, four in total) were running at full blast (100% CPU usage), and recon-all.log reveals that recon-all is waiting for this step to conclude.
I have searched the list, but the posts containing similar issues all seemed to pertain to much older versions of FreeSurfer.
FreeSurfer was invoked using the following command:
$ parallel -a recon-all_test3.txt -j32
The contents of recon-all_test3.txt:
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -T2 /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/sorted/Sag_T2_CUBE_1iso/z4884 -s top3T_0186.fs6.3T.T2pial -all -3T -parallel -T2pial
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -FLAIR /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_CUBE_Flair_TR8/z360 -s top3T_0186.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -FLAIR dicom/top3T_0065/Sag_CUBE_Flair_TR8/Sag_CUBE_Flair_TR8-1.dcm -s top3T_0065.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -T2 dicom/top3T_0065/Sag_T2_CUBE_1iso/Sag_T2_CUBE_1iso-1.dcm -s top3T_0065.fs6.3T.T2pial -all -3T -parallel -T2pial
I would be grateful for any and all clues as to what's wrong here.
- FreeSurfer version:
freesurfer-Linux-centos6_x86_64-stable-pub-v.6.0.0-2beb96c
Platform: Ubuntu LTS 14.04.
uname -a: Linux picasso 3.13.0-79-generic #123-Ubuntu SMP Fri Feb 19
14:27:58 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
- recon-all.log: See attachment.
Regards Eirik Svela Institute of Psychiatry, University of Oslo
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
HI Eirik
usually this means that something large went wrong. The topology correction complexity goes as the square of the size of the largest defect, so can take forever for large defects. The really large ones mean that something else happened - like the skull wasn't stripped properly or the cerebellum is attached. Look at the lh.orig.nofix and the lh.inflated.nofix to see what happened (or rh if it is the right hemi)
cheers Bruce
On Mon, 3 Apr 2017, Eirik Wixøe Svela wrote:
Hello everyone,
We recently installed FreeSurfer 6.0 at our site and are currently working out which flags and options to use. To learn more about what would suit our needs, we ran a couple of subjects using various flag combinations in order to evaluate the output. One run was initiated on Tuesday Mar 28, and as Friday Mar 31, the recon-all script still had not completed. We obtained similar results on trying to process it again, cf. the attached log file, and also with other subjects on other workstations. It seems that the mri_fix_topology binary is causing this issue -- the top utility indicates that four instances (one for each subject-flag combination, four in total) were running at full blast (100% CPU usage), and recon-all.log reveals that recon-all is waiting for this step to conclude.
I have searched the list, but the posts containing similar issues all seemed to pertain to much older versions of FreeSurfer.
FreeSurfer was invoked using the following command:
$ parallel -a recon-all_test3.txt -j32
The contents of recon-all_test3.txt:
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -T2 /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/sorted/Sag_T2_CUBE_1iso/z4884 -s top3T_0186.fs6.3T.T2pial -all -3T -parallel -T2pial
recon-all -i /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_T1_BRAVO_1iso/z127 -FLAIR /net/data/uTOP/3T_BRAVO/dicom/top3T_0186/Sag_CUBE_Flair_TR8/z360 -s top3T_0186.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -FLAIR dicom/top3T_0065/Sag_CUBE_Flair_TR8/Sag_CUBE_Flair_TR8-1.dcm -s top3T_0065.fs6.3T.FLAIRpial -all -3T -parallel -FLAIRpial
recon-all -i dicom/top3T_0065/Sag_T1_BRAVO_1iso/Sag_T1_BRAVO_1iso-1.dcm -T2 dicom/top3T_0065/Sag_T2_CUBE_1iso/Sag_T2_CUBE_1iso-1.dcm -s top3T_0065.fs6.3T.T2pial -all -3T -parallel -T2pial
I would be grateful for any and all clues as to what's wrong here.
- FreeSurfer version:
freesurfer-Linux-centos6_x86_64-stable-pub-v.6.0.0-2beb96c
Platform: Ubuntu LTS 14.04.
uname -a: Linux picasso 3.13.0-79-generic #123-Ubuntu SMP Fri Feb 19
14:27:58 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
- recon-all.log: See attachment.
Regards Eirik Svela Institute of Psychiatry, University of Oslo
freesurfer@nmr.mgh.harvard.edu
-
Bruce Fischl -
Eirik Wixøe Svela -
Z K