On 7 March 2016 at 11:25, Francis Tyson Thomas francistthomas@email.arizona.edu wrote:
That information was a lot helpful. At this point I'm currently trying to reduce the recon-all processing time as much as possible and for this reason I was looking to get the -use-gpu flag working. I'm currently running a freesurfer v6 beta version on Ubuntu 14.04.4. With regard to the graphic card it is a dual K2200 configuration (I guess they are running in sli configuration - although I'm not completely sure).
When you mentioned you compiled everything, I believe you were referring to compiling CUDA 7.5 for Ubuntu 14.04. Because after seeing the link - https://developer.nvidia.com/cuda-gpus
- we settled for CUDA 5 since it was the compatible version mentioned for
K2200. Does that mean CUDA 7.5 is backwards compatible with a slight tinkering and can be used with freesurfer 6.0 ?
I think that you might have misunderstood the NVIDIA page.
It lists the K2200 as a Compute Capability 5 GPU (just like my K1200). The Compute Capability refers to the hardware: http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#compute-capa... Compute capability 5 is also called "Maxwell," while the work I did on Freesurfer was around the time of Compute Capability 2, known as Fermi. In CPU terms, GPU compute capabilities are a bit like Ivy Bridge vs Sandy Bridge (although I think that GPU features vary more than the CPU ones).
In order to program the GPU, you need the NVIDIA CUDA Toolkit, which contains the required compiler (nvcc). The current version of this is 7.5. I installed it by following the instructions on NVIDIA's website: http://docs.nvidia.com/cuda/index.html This was quite straightforward (certainly moreso than it was five or six years ago, when you could never be sure that your X11.conf would survive).
Either with the toolkit, or as a separate install, you can get a lot of examples from NVIDIA. I'd suggest grabbing those, and making sure that you can compile them. The "DeviceQuery" one will probe your PCIe bus, and report what GPUs it finds.
I however tried to setup CUDA 5 following the instructions in the link - http://www.unixmen.com/how-to-install-cuda-5-0-toolkit-in-ubuntu/ - however, I'm not able to get it running. I keep getting the following error "Unable to acquire CUDA device". Does this sound familiar ?
I suspect (although I wasn't following things at the time) that the Toolkit v5 was before Maxwell cards were released. If so, then it wouldn't know what to do with the GPUs.
For the record, since I was writing the CUDA bits so long ago (for the volume side of things - I didn't do the surface accelerations), they only use Fermi features. For this reason, you'll want to make sure that you have --enable-fermi-gpu when you run configure (and make sure that it's picking out your CUDA installation - I had to tweak the configure script for this).
If you can share some more information in setting this up it will be great since the amount of time recon-all takes is quite too long for running multiple datasets. Most importantly we are concerned about the hippocampal segmentation in freesurfer 6 rather than recon-all and so speeding this up would be extremely helpful.
I don't know if those portions benefit from CUDA acceleration at this time. I focused on mri_em_register and mri_ca_register. Even if other programs (which ones are they?) can be linked against some of the accelerated routines, there is no guarantee of speed up - the time to shuffle data to and from the GPU is typically greater than the speedup of any one routine.
Hope this helps,
Richard