Hi Richard,

That was really helpful. All the details were very clear and I was able to get it up and running except for some small hiccups which I fixed with a small hack. 

The issue I faced was after following all your instructions when I cam to the step of "make j4" it threw an error saying that it was not able to find -lcudart. And it looked like it was looking for it in "/usr/local/cuda/lib" when the directory itself didn't exist. I fixed this by editing another line in the configure.in file. I changed from CUDA_LIBS="-L$CUDA_DIR/lib $LIB_CUDA -lcudart" to CUDA_LIBS="-L$CUDA_DIR/lib64 $LIB_CUDA -lcudart". Was this change right? 

Also I noticed that the speedup wasn't extremely huge. The time for one recon-all run went down from 7 hrs 45 mins to 5hrs 20 mins. My understanding is that this is due to only certain modules being cuda-ised (if there is a word like that!) like the mri_ca_register  and the mri_em_register. Are there any other modules that are parallelized currently other than these?

With regard to the code base I guess I'm also using the read-only git repository of the main development trunk as per the instructions on the freesurfer webpage.

Once again thank you very much for that detailed information.

Best,
Tyson


On Mon, Mar 21, 2016 at 6:47 PM, R Edgar <freesurfer.rge@gmail.com> wrote:
On 21 March 2016 at 19:59, Francis Tyson Thomas
<francistthomas@email.arizona.edu> wrote:

> The first part was smooth. I got it done pretty quickly (installing CUDA and
> setting it up!). However for the freesurfer portion I have two questions,
>
> 1. Bruce mentioned in another email thread that v6 beta should be out in a
> week or so after testing is completed. Should I wait for that version,
> assuming I get the source code for v6 when I download it as per the
> instructions on the freesurfer page.

Which version of the source code do you have access to? I'm using the
read-only git repository of the main development trunk, sending
patches back to Zeke, which he puts into CVS. I'm not sure which of
those are making it over to v6.

> 2. Also, would I have to compile and build all modules (mri_em_registe,
> mri_ca_register etc or can I limit to just these two when I build it again.

I imagine you'll have to run configure again. When you do, the key
options you want are:
 --enable-fermi-gpu --with-cuda="/usr/local/cuda"
Note that on my machine:
[rge21@cudastation ~]$ which nvcc
/usr/local/cuda-7.5/bin/nvcc
[rge21@cudastation ~]$ ls -alF /usr/local/cuda
lrwxrwxrwx. 1 root root 8 Jan  9 16:30 /usr/local/cuda -> cuda-7.5/
So if your CUDA installation went somewhere else, you'll have to
adjust that --with-cuda path.

Then when you compile, make sure the nvcc being used really is the one
you've set (things will likely fail horribly if this isn't the case).
If it isn't, you'll have to look in the configure file for:

#############################################################
# Nvidia CUDA enabling
############################################################
CUDA_DIR=""
with_cuda=""

and then comment out
# with_cuda=""
or it will override the directory you set on the command line.

Let me know if you need further help,

Richard
_______________________________________________
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.