Hi,

Thank you for your reply.

The CUDA 5.x binaries can be found in the CUDA 5.x legacy drivers for CUDA. 

As long as you set the PATH=/usr/local/cuda/bin:$PATH and LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH, Freesurfer's cudadetect will look into here for the binaries. I think you should note that cudaDetect (belonging to nVidia) is different than Freesurfer's cudadetect (found in usr/freesurfer/bin).

Apologies for the previous misconception.

My issue right now is that the GTX 780 (kepler) cant work with 5.0. Do you know if it can work with 5.5? I might try my hand to see if I can compile for a newer version of CUDA.

Keep me posted.
- Bryan

Hi there,

I'm currently working on compiling FS source code trying to use CUDA 6, and newer GPUs like Titan series. I haven't got lucky yet, but I will keep on trying... In the meantime I would like to download the CUDA 5 binaries. Are these binaries been published? I can't find anything on the FreeSurfer wiki...

In my previous work you mentioned  "Improving Execution Performance of FreeSurfer" we used CUDA 3 binaries that I had compiled with --enable-fermi flag. Just to clarify your concern, we never used openmp on the test setup of that study, we have implemented a scheduler to manage K simultaneous recon-all executions.  

Thank you,


2014-07-02 19:06 GMT+02:00 Chiu, Bryan (PHTH) <bryan.chiu@ubc.ca>:
Hi,

Turns out if you rename libcudart.so.6.0 to the 5.0 in your respective lib64, you can pass the cudadetect for Freesurfer. However after it will crash with a segfault error. This leads me to believe that I installed correctly.

I think Freesurfer only supports up to CUDA 5.0 so far (although I know Freesurfer group stopped supporting the option and left the flags around). Two options exist: Either in a future patch the program can just be enabled to call the newer version of CUDA (or if I can find where it tries to find CUDA from, edit it to accept any version), or to install any version of CUDA less than or equal to 5.0

To others: The OpenMP and -use-gpu flag are quite promising. I took a recon down from 12 hours to 3.5 hours using the -openmp 8 flag for a 4 core processor (i7 4770k). I am currently trying to see if I can speed up faster using the CUDA enabled option. There was a paper titled "Improving the Execution Performance of Freesurfer" which implements a resource scheduler on top of the CUDA and openMP option. No comment on the accuracy of the recon-all using all of these in combination.

- Bryan

Date: Mon, 30 Jun 2014 16:34:45 +0000
From: "Chiu, Bryan (PHTH)" <bryan.chiu@ubc.ca>
Subject: Re: [Freesurfer] recon-all CUDA error (Chiu, Bryan (PHTH))
To: "freesurfer@nmr.mgh.harvard.edu" <freesurfer@nmr.mgh.harvard.edu>
Message-ID:
        <680D9B8D5FEAAF40A1A5878265424CA44A59499D@s-itsv-mbx06p.ead.ubc.ca>
Content-Type: text/plain; charset="iso-8859-1"

Hi all,

I am still working on this problem. Does anyone know if Freesurfer is asking for CUDA 5.0, except that I have CUDA 6.0 installed?

My next steps will be to find out if Freesurfer is calling an older version of CUDA that isn't installed, or trying the CUDA samples. I really want to get this to work.

-Bryan

Date: Fri, 27 Jun 2014 17:59:37 +0000
From: "Chiu, Bryan (PHTH)" <bryan.chiu@ubc.ca>
Subject: [Freesurfer] recon-all CUDA error
To: "freesurfer@nmr.mgh.harvard.edu" <freesurfer@nmr.mgh.harvard.edu>
Message-ID:
        <680D9B8D5FEAAF40A1A5878265424CA44A594981@s-itsv-mbx06p.ead.ubc.ca>
Content-Type: text/plain; charset="iso-8859-1"

Hi,

I'm fairly novice at Linux and I've been trying to setup CUDA for recon-all. I get an error as follows:

[TLA@COGMOB-linuxPC ~]$ recon-all -s bert -use-gpu -openmp 8
Testing for CUDA device:
/usr/freesurfer/bin/mri_em_register_cuda: error while loading shared libraries: libcudart.so.5.0: cannot open shared object file: No such file or directory
Linux COGMOB-linuxPC 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

recon-all -s bert exited with ERRORS at Fri Jun 27 10:53:29 PDT 2014

For more details, see the log file
To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting

Any idea what is going on? I setup CUDA and my Nvidia drivers according to this tutorial (http://aspratyush.wordpress.com/2012/05/06/install-nvidia-cuda-on-centos-6/). With, of course some slight changes to install the latest toolkit.

Thanks in advance,

- Bryan

_______________________________________________
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.




-- 
Jordi Delgado Mengual
PIC (Port d'Informació Científica)