Hi FreeSurfer Developers,
I’m running freesurfer from the HCP pipeline developed by the Washington University in St. Louis. It ran just fine on our university’s cluster, but when I migrated the pipeline to Exacloud, Intel’s HPC cluster, the recon-all command began
randomly failing. Occasionally it will work on the first try, but more often than not the same command has to be re-run upwards of 30 times before it will succeed. I’ve provided the recon-all command and the relevant section of the log file including the error
here (attached is the full recon-all.log file):
recon-all -i ./T1w/T1w_acpc_dc_restore_1mm.nii.gz -subjid ${SUBJECT_ID} -sd . /T1w -motioncor -tal airach -nuintensitycor –normalization
…
[perronea@exanode-3-9.local${PWD}/T1w/washu_INV0WC5U4JA/mri/] [2016-11-17 13:08:38] running:
/home/exacloud/lustre1/users/mirandad/usr/local/freesurfer53/mni/bin/make_template -quiet -shrink 3 ./tmp.mri_nu_correct.mni.223851/nu1.mnc ./tmp.mri_nu_correct.mni.223851/1//template.mnc
Transforming slices:......................................................................................Done
Transforming slices:.........................................................................................................................................................mincresample:
posixio.c:210: px_pgin: Assertion `*posp == ((off_t)(-1)) || *posp == lseek(nciop->fd, 0, 1)' failed.
nu_evaluate: crashed while running mincresample (termination status=134)
nu_correct: crashed while running nu_evaluate (termination status=65280)
ERROR: nu_correct
Linux exanode-3-9.local 2.6.32-504.30.3.el6.x86_64 #1 SMP Wed Jul 15 10:13:09 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
recon-all -s washu_INV0WC5U4JA exited with ERRORS at Thu Nov 17 13:08:40 PST 2016
It doesn’t appear than any similar errors have been reported on freesurfer. Our current work-around is to simply re-run the job on different nodes until it works, but this is not a sustainable long-term solution. Any guidance on troubleshooting
this error would be greatly appreciated.
FreeSurfer version: freesurfer-Linux-centos6_x86_64-stable-pub-v5.3.0
Platform: CentOS release 6.5 (FINAL)
uname –a: Linux exalab3.ohsu.edu 2.6.32-504.12.2.el6.x86_64 #1 SMP Wed Mar 11 22:03:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Thanks,
Anders
Anders Perrone,
Research Assistant I
Fair Neuroimaging Lab
503-418-1897
Oregon Health & Science University
Mail code:L470
3181 SW Sam Jackson Park Road
Portland, Oregon 97239-3098
“If there is no solution to the problem then don’t waste time worrying about it. If there is a solution to the problem then don’t waste time worrying about it.” – Dalai Lama XIV