Opps... I forgot the attachment. Here it is.
-Zeke
-------- Original Message -------- Subject: Re: [Freesurfer] Open MP parallel processing scaling factor? Date: Wed, 20 Aug 2014 15:24:14 -0400 From: Z K zkaufman@nmr.mgh.harvard.edu To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu, Satrajit Ghosh satra@mit.edu, aoshea@ufl.edu
Andrew,
I basically agree with everything Satra is saying. Just as an added piece if info, Ive included a chart which did a systematic study of recon-all processing times with different number of cpus in parallel.
These results are a bit old (and might be for low-res data) but the general idea is that there is a point of diminishing returns after using 4 processors in parallel. The gain from using more processors may be worth it for you.
There are other tricks to getting faster recons (like processing left and right hemishperes in parallel) but that is a bit more tricky and requires a custom script. I can provide additional information if you think you would like to pursue that avenue
-Zeke
On 08/20/2014 11:21 AM, Satrajit Ghosh wrote:
hi andrew,
i haven't done many systematic comparisons but there are a few practical considerations to take into account. in recon-all, i believe a few steps are affected by the openmp option and that creates resource underutilization. processors run idle when those steps are not being run.
in my ad hoc analysis i have found i can process a single subject in about 5 hours using openmp 8, but that holds up 8 processors for that subject. the same subject can be processed in about 12 hours one 1 processor. say i have 16 processors, i can process 16 subjects in say 12 hours using 1 processor per recon. however, using 8 per recon would take about 40 hours, 2 subjects every 5 hours.
so on our cluster, we tend to process with openmp 1 or 2 depending on the average load on the cluster. this is also dependent on your hardware, amount of memory, cluster scheduler, etc.,.
if i really need speed on an individual case i go with -openmp 8.
cheers,
satra
On Wed, Aug 20, 2014 at 11:04 AM, O'Shea,Andrew <aoshea@ufl.edu mailto:aoshea@ufl.edu> wrote:
Hello all, Traditionally I have only processed FS data using a single core per person, but processing many people at once. Now we have caught up with the backlog of scans, we have a continuos trickle of scans coming in 1 by 1. I was wondering if anyone has tested how the speed-up of open-mp varies with number of cores used simultaneously. For example how much faster is using 100 cores versus 10? I am trying to find a sweet spot of resource usage and speed. Thanks! -Andrew _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu <mailto:Freesurfer@nmr.mgh.harvard.edu> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer