CUDA Error in file '/space/freesurfer/build/minerva/stable/dev/include/ctfactory.hpp' on line 92 : invalid texture reference
Hi all, Here are the details of my freesurfer and linux installations
FREESURFER_HOME: /usr/local/freesurfer
Build stamp: freesurfer-Linux-centos4_x86_64-stable-pub-v5.1.0
RedHat release: CentOS Linux release 6.0 (Final)
Kernel info: Linux 2.6.32-71.29.1.el6.x86_64 x86_64
I successfully ran autorecon2 without gpu on several datasets that we have. However, when I try to use the GPU, I get the following error message: CUDA Error in file '/space/freesurfer/build/minerva/stable/dev/include/ctfactory.hpp' on line 92 : invalid texture reference
The calling sequence and all on-screen messages are listed at the end of this email. A similar error was reported a year ago by a different user ( https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2010-August/015528.ht...) so I am hoping there is a simple fix. Thanks in advance, Hemant
recon-all -subjid 203 -use-gpu -autorecon2 Testing for CUDA device: nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2010 NVIDIA Corporation Built on Thu_Nov__4_12:44:17_PDT_2010 Cuda compilation tools, release 3.2, V0.2.1221
Driver : 4.10 Runtime : 3.20
Acquiring CUDA device Using default device CUDA device: Quadro FX 4600 stable5
NVIDIA driver version info: NVRM version: NVIDIA UNIX x86_64 Kernel Module 285.05.09 Fri Sep 23 17:31:57 PDT 2011
Subject Stamp: freesurfer-Linux-centos4_x86_64-stable-pub-v5.1.0 Current Stamp: freesurfer-Linux-centos4_x86_64-stable-pub-v5.1.0 INFO: SUBJECTS_DIR is /home/hemant/FreeSurferSubjects Actual FREESURFER_HOME /usr/local/freesurfer -rw-rw-r--. 1 hemant hemant 226454 Oct 24 15:05 /home/hemant/FreeSurferSubjects/203/scripts/recon-all.log Linux Lab-3.intelect.local 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64 x86_64 x86_64 GNU/Linux #------------------------------------- #@# EM Registration Mon Oct 24 15:05:52 EDT 2011 /home/hemant/FreeSurferSubjects/203/mri
mri_em_register_cuda -uns 3 -mask brainmask.mgz nu.mgz /usr/local/freesurfer/average/RB_all_2008-03-26.gca transforms/talairach.lta
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2010 NVIDIA Corporation Built on Thu_Nov__4_12:44:17_PDT_2010 Cuda compilation tools, release 3.2, V0.2.1221
Driver : 4.10 Runtime : 3.20
Acquiring CUDA device Using default device CUDA device: Quadro FX 4600 aligning to atlas containing skull, setting unknown_nbr_spacing = 3 using MR volume brainmask.mgz to mask input volume... reading 1 input volumes... logging results to talairach.log reading '/usr/local/freesurfer/average/RB_all_2008-03-26.gca'... gcaAllocMax: node dims 64 64 64 gcaAllocMax: prior dims 128 128 128 gcaAllocMax: max_labels 0 average std = 6.9 using min determinant for regularization = 4.7 0 singular and 1812 ill-conditioned covariance matrices regularized reading 'nu.mgz'... freeing gibbs priors...done. bounding unknown intensity as < 14.9 or > 790.2 total sample mean = 84.4 (994 zeros) ************************************************ spacing=8, using 2772 sample points, tol=1.00e-05... ************************************************ CUDA Error in file '/space/freesurfer/build/minerva/stable/dev/include/ctfactory.hpp' on line 92 : invalid texture reference. register_mri: find_optimal_transform find_optimal_transform: nsamples 2772, passno 0, spacing 8 Abort (core dumped) Linux Lab-3.intelect.local 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64 x86_64 x86_64 GNU/Linux
recon-all -s 203 exited with ERRORS at Mon Oct 24 15:06:11 EDT 2011
On Mon, Oct 24, 2011 at 3:36 PM, Hemant Bokil hemantbokil@gmail.com wrote:
Driver : 4.10 Runtime : 3.20
My best guess would be this mismatch causing a problem. It shouldn't in theory, but...
Richard
HI Richard, Thanks for the response. Does this mean that I should use CUDA 3.2 ? There was some discussion on the group recently about Freesurfer not working with CUDA 4 and one suggestion made was to switch to CUDA 3.2. Thanks, Hemant
On Mon, Oct 24, 2011 at 7:07 PM, R Edgar freesurfer.rge@gmail.com wrote:
On Mon, Oct 24, 2011 at 3:36 PM, Hemant Bokil hemantbokil@gmail.com wrote:
Driver : 4.10 Runtime : 3.20
My best guess would be this mismatch causing a problem. It shouldn't in theory, but...
Richard _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
On Mon, Oct 24, 2011 at 9:28 PM, Hemant Bokil hemantbokil@gmail.com wrote:
Thanks for the response. Does this mean that I should use CUDA 3.2 ? There was some discussion on the group recently about Freesurfer not working with CUDA 4 and one suggestion made was to switch to CUDA 3.2. Thanks,
Probably. NVIDIA always like to claim full backwards compatibility, but I've never been fully convinced.
Richard
Hi Richard and others, I installed the earlier driver and the toolkit, but I sill have the same crash. A portion of the log just before the crash is below. Any thoughts ? Hemant mri_em_register_cuda -skull nu.mgz /usr/local/freesurfer/average/RB_all_withskull_2008-03-26.gca transforms/talairach_with_skull.lta
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2010 NVIDIA Corporation Built on Thu_Nov__4_12:44:17_PDT_2010 Cuda compilation tools, release 3.2, V0.2.1221
Driver : 3.20 Runtime : 3.20
Acquiring CUDA device Using default device CUDA device: Quadro FX 4600 aligning to atlas containing skull, setting unknown_nbr_spacing = 5 reading 1 input volumes... logging results to talairach_with_skull.log reading '/usr/local/freesurfer/average/RB_all_withskull_2008-03-26.gca'... gcaAllocMax: node dims 64 64 64 gcaAllocMax: prior dims 128 128 128 gcaAllocMax: max_labels 0 average std = 23.1 using min determinant for regularization = 53.4 0 singular and 5702 ill-conditioned covariance matrices regularized reading 'nu.mgz'... freeing gibbs priors...done. bounding unknown intensity as < 20.2 or > 943.7 total sample mean = 92.0 (1443 zeros) ************************************************ spacing=8, using 3481 sample points, tol=1.00e-05... ************************************************ CUDA Error in file '/space/freesurfer/build/minerva/stable/dev/include/ctfactory.hpp' on line 92 : invalid texture reference. register_mri: find_optimal_transform find_optimal_transform: nsamples 3481, passno 0, spacing 8 Abort (core dumped) Linux Lab-3.intelect.local 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64 x86_64 x86_64 GNU/Linux
recon-all -s 209 exited with ERRORS at Tue Oct 25 17:13:11 EDT 2011
For more details, see the log file /home/hemant/FreeSurferSubjects/209/scripts/recon-all.log To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting
On Tue, Oct 25, 2011 at 8:41 AM, R Edgar freesurfer.rge@gmail.com wrote:
On Mon, Oct 24, 2011 at 9:28 PM, Hemant Bokil hemantbokil@gmail.com wrote:
Thanks for the response. Does this mean that I should use CUDA 3.2 ?
There
was some discussion on the group recently about Freesurfer not working
with
CUDA 4 and one suggestion made was to switch to CUDA 3.2. Thanks,
Probably. NVIDIA always like to claim full backwards compatibility, but I've never been fully convinced.
Richard _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
freesurfer@nmr.mgh.harvard.edu
-
Hemant Bokil -
R Edgar