Hi,
I wonder how much graphic card memory is needed by the CUDA commands. We have two Quadro NVS 295's with 256MB memory and the CUDA commands all report "out of memory" errors. I know the cards are kind of outdated but I want to make sure a new card would meet the task before purchasing.
A separate question is whether the current CUDA implementation supports spreading the work to multiple GPU's if they are present?
Thanks.
On Mon, Feb 4, 2013 at 5:50 PM, Zhongtian Dai dai@uchicago.edu wrote:
I wonder how much graphic card memory is needed by the CUDA commands. We have two Quadro NVS 295's with 256MB memory and the CUDA commands all report "out of memory" errors. I know the cards are kind of outdated but I want to make sure a new card would meet the task before purchasing.
I never checked exactly how much GPU RAM was consumed (having 3 GiB cards has that effect on you ;-) ), but.... 256^3 = 16 M. So each volume takes 16 * sizeof(T) MiB of RAM. I don't recall offhand how many volumes GCAMorphGPU had to allocate, but it was quite a few (at least 6 float volumes for sure). I can easily see 256MiB being insufficient (particularly when you remember that the CUDA context itself consumes some - small - amount of RAM on the card).
A separate question is whether the current CUDA implementation supports spreading the work to multiple GPU's if they are present?
No - at least not in the stuff I did. But you can run multiple recon-alls, with each using a different GPU. And with enough GPU RAM, you can probably run multiple recons on the same GPU with reasonable speedups, since there are plenty of portions of the pipeline which are still on the CPU.
HTH,
Richard
Hi Richard,
Thanks for the speedy response. I think we will go with GTX 660 Ti with 2GB GPU RAM. We don't have a lot of subjects (as we hope to have) to analyze, so maybe we will try splitting left and right hemispheres to two graphic cards if our CPU RAM doesn't deplete. How do you designate the cuda commands to a certain GPU?
On Mon, Feb 4, 2013 at 5:04 PM, R Edgar freesurfer.rge@gmail.com wrote:
On Mon, Feb 4, 2013 at 5:50 PM, Zhongtian Dai dai@uchicago.edu wrote:
I wonder how much graphic card memory is needed by the CUDA commands. We have two Quadro NVS 295's with 256MB memory and the CUDA commands all
report
"out of memory" errors. I know the cards are kind of outdated but I want
to
make sure a new card would meet the task before purchasing.
I never checked exactly how much GPU RAM was consumed (having 3 GiB cards has that effect on you ;-) ), but.... 256^3 = 16 M. So each volume takes 16 * sizeof(T) MiB of RAM. I don't recall offhand how many volumes GCAMorphGPU had to allocate, but it was quite a few (at least 6 float volumes for sure). I can easily see 256MiB being insufficient (particularly when you remember that the CUDA context itself consumes some - small - amount of RAM on the card).
A separate question is whether the current CUDA implementation supports spreading the work to multiple GPU's if they are present?
No - at least not in the stuff I did. But you can run multiple recon-alls, with each using a different GPU. And with enough GPU RAM, you can probably run multiple recons on the same GPU with reasonable speedups, since there are plenty of portions of the pipeline which are still on the CPU.
HTH,
Richard
On Mon, Feb 4, 2013 at 6:54 PM, Zhongtian Dai dai@uchicago.edu wrote:
Thanks for the speedy response. I think we will go with GTX 660 Ti with 2GB GPU RAM. We don't have a lot of subjects (as we hope to have) to analyze, so maybe we will try splitting left and right hemispheres to two graphic cards if our CPU RAM doesn't deplete. How do you designate the cuda commands to a certain GPU?
You'll have to ask Nick that - I don't remember, I'm afraid.
Richard
The environment variable 'FREESURFER_CUDA_DEVICE' defines the device number of the GPU device. Although I dont think you will see much benefit in splitting the surface processing across two gpus, as the CUDA surface code is not as optimal as the volume-based mri_ca_register code for parallel processing.
Nick
On Mon, 2013-02-04 at 19:31 -0500, R Edgar wrote:
On Mon, Feb 4, 2013 at 6:54 PM, Zhongtian Dai dai@uchicago.edu wrote:
Thanks for the speedy response. I think we will go with GTX 660 Ti with 2GB GPU RAM. We don't have a lot of subjects (as we hope to have) to analyze, so maybe we will try splitting left and right hemispheres to two graphic cards if our CPU RAM doesn't deplete. How do you designate the cuda commands to a certain GPU?
You'll have to ask Nick that - I don't remember, I'm afraid.
Richard _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
freesurfer@nmr.mgh.harvard.edu
-
Nick Schmansky -
R Edgar -
Zhongtian Dai