On Tue, 2010-11-23 at 15:03 +0100, Marcus N Schmidt wrote:
I should note that we have a single GeForce GTX 470 card on the workstation and have installed the latest CUDA drivers and toolkit (3.2).
Is there a limit to how many processes can use CUDA at the same time? I just setup a new Linux workstation and am delving into FreeSurfer w/ CUDA. I'm getting the following error:
Acquiring CUDA device Using default device CUDA Error in file 'devicemanagement.cu' on line 46 : all CUDA-capable devices are busy or unavailable. Linux rietveld 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:20 EST 2010 x86_64 x86_64 x86_64 GNU/Linux
None of the code I've written (which includes devicemanagement.cu) imposes limitations - I actually often run more than one GPU job at once on the card. The driver software timeslices the entire card between processes. However, the cards themselves might be set so that only one process can use them at a time (using the nvidia-smi tool) - that's entirely up to the system administrator.
I would not recommend running more than one CUDA process per card routinely - especially not on a low end card like that - since the memory allocations might end up failing, killing the job.
Regards,
Richard