Hi Richard,
Hope this finds you well. I followed your instructions and compiled from source (you were right-- compiling Fs from source isn't too bad at all!) I also made your tweak to configure.in and then ran ./configure like so:
./configure --disable-Werror --enable-fermi-gpu --with-cuda="/usr/local/cuda" --with-pkgs-dir=/tmp/centos6-x86_64-packages --prefix=/usr/local/freesurfer
That ran successfully. However, when I then ran 'make -j4', I get the following error and it exits:
/usr/bin/ld: skipping incompatible /usr/local/cuda/lib/libcudart.so when searching for -lcudart /usr/bin/ld: cannot find -lcudart collect2: error: ld returned 1 exit status Makefile:568: recipe for target 'cudadetect' failed make[2]: *** [cudadetect] Error 1 make[2]: Leaving directory '/tmp/freesurfer/gpu_utils' Makefile:1053: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/tmp/freesurfer' Makefile:962: recipe for target 'all' failed make: *** [all] Error 2
It is very strange, because I don't think this is an issue with not being able to find the cuda libraries (i.e. libcudart*) since they are detected by FSL's CUDA wrappers and are exported in my ~/.bashrc:
export PATH=$PATH:/usr/local/cuda/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/lib
Curious to hear your thoughts on this. Also, if it helps to know, I am using cuda toolkit 6.5 and compiling on Debian 8.0.
Thanks in advance, Derek ________________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu [freesurfer-bounces@nmr.mgh.harvard.edu] on behalf of R Edgar [freesurfer.rge@gmail.com] Sent: Wednesday, June 22, 2016 6:14 PM To: Freesurfer support list Subject: Re: [Freesurfer] Any updates on recon-all GPU support?
On 22 June 2016 at 19:32, Derek Pisner dpisner@psychiatry.arizona.edu wrote:
Thanks for the very quick reply. Good to know that someone has successfully compiled GPU binaries for newer CUDA versions. I took a quick look at the developer's guide for building the cuda binaries from source, but it seems like a very involved process? Do your compiled binaries only work for CUDA SDK 7.5 or have you tested them with other CUDA versions?
I only have one Linux box, running CentOS 7 (which isn't officially supported by Freesurfer - I had to tweak CFLAGS quite a bit too) with CUDA 7.5.
I wouldn't say the process is very involved to compile from source, but I did have to tweak things a bit. Specifically, after running ./setup_configure, I had to open up the configure script, and find the section which looks like: ############################################################# # Nvidia CUDA enabling ############################################################ CUDA_DIR="" with_cuda=""
And comment out the with_cuda="" line. Then I run configure with --enable-fermi-gpu --with-cuda="/usr/local/cuda" (plus some other options for my machine). Note that for me: $ which nvcc /usr/local/cuda-7.5/bin/nvcc
Also, would you mind sending me an example binary (e.g. mri_ca_register_cuda) just so I cant test it myself and rule out whether this is just an issue with the way our GPU is configured? (I can't even get cudadetect to acknowledge my GPU as a cuda-enabled device (it just says 0 devices found), despite it successfully acknowledging that the GPU "exists") Please let me know and I cna send you a private email address.
I can do, but there's a lot of things which could go wrong.
Are you able to compile and run the sample CUDA programs which NVIDIA supplies as part of the SDK? If you haven't tried that yet, it would be a better place to start.
Thanks again for your help! And hopefully this thread will help others interested in updating their gpu-capabilities with Freesurfer.
Indeed - one Francis Thomas @arizona.edu was asking similar questions a few months ago.
I'm afraid I'm not doing any extra GPU porting right now - I've been getting concerned about potential drift between the CPU and GPU pipelines, and trying to come up with a better way around the problem.
HTH,
Richard
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.