Hi Bruce, No I did not specify the # of open mp threads on the recon-all cmd line. These run times were obtained by running one subject per core. for example cc2.8xlarge has 8 cores and so we ran 8 subjects at once; Thanks for the info about the # of open mp threads options; I will look into it.

One other note: Bruce, Nick did you improve the memory management in 5.2 ? On our local machine we noticed we can run 6 subjects simultaneously even though we only have 12gb of ram. I thought some of the them might crash since we only have 2gb per subject but no crashes so far over 30 subjects.. Mehul

On Fri, Feb 1, 2013 at 7:12 PM, Bruce Fischl fischl@nmr.mgh.harvard.eduwrote:

Hi Mehul

did you specify the # of open mp threads on the recon-all cmd line? cheers Bruce

On Fri, 1 Feb 2013, Mehul Sampat wrote:

Hi Folks,

...

Just wanted to share our experience with running FS 5.2-beta on Amazon Web Services (AWS). Basically, AWS has multiple instance types (http://aws.amazon.com/ec2/**instance-types/http://aws.amazon.com/ec2/instance-types/) and we were trying to figure out the most cost-effective approach.

We ran two subjects through FS 5.2-beta on M1 Large Instance (m1.large) and Cluster Compute Eight Extra Large Instance (cc2.8xlarge). (same subjects run on both instance). We expected cc2.8xlarge to be faster (but it is also more expensive: $2.4 per hour; 8 cores); The run-times we got:

instance-type subject start-time end-time run-time m1.large subject-1 01:05:44 UTC 2013 15:40:45 UTC 2013 ~14hr-35mins m1.large subject-2 01:06:06 UTC 2013 15:08:45 UTC 2013 ~14hr-02mins cc2.8xlarge subject-1 01:26:38 UTC 2013 12:30:23 UTC 2013 ~11hr-04mins cc2.8xlarge subject-2 01:27:28 UTC 2013 12:19:08 UTC 2013 ~10hr-52mins

Although m1.large is a few hours slower, it seems to be the more cost effective option since it is $0.24 per hour (2 cores). If you have run Freesurfer on AWS, do you have a similar experience ? Any suggestions to speed up the run-times on AWS would be very helpful.

Thanks Mehul

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/**compliancelinehttp://www.partners.org/complianceline. If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

I agree with Pedro. For running a large dataset on ec2, it is better to launch a lot of instances m1.medium.

Gen

From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer-bounces@nmr.mgh.harvard.edu] On Behalf Of Pedro Paulo de Magalhães Oliveira Junior Sent: Saturday, February 02, 2013 3:08 AM To: Mehul Sampat Cc: Freesurfer Mailing List Subject: Re: [Freesurfer] FS 5.2-beta run-times on Amazon Web services (AWS)

In the http://cerebralvol.com service we are having 23.5 hours average with 5.1 and 23.8 hours average with 5.2-Beta

Notice that in order to minimize the cost to the user we are running it in m1.medium

We have achieved a full recon-all in less than 4 hours (3.83 hours) with cg1.4xlarge

But for a large amount of data I believe our 1024-core m1.medium is the best cost benefit.

--------------------------------------------------------------------- Pedro Paulo de Magalhães Oliveira Junior Netfilter & SpeedComm Telecom -- www.netfilter.com.brhttp://www.netfilter.com.br -- For mobile: http://itunes.apple.com/br/artist/netfilter/id365306441

On Sat, Feb 2, 2013 at 2:11 AM, Mehul Sampat <mpsampat@gmail.commailto:mpsampat@gmail.com> wrote: Hi Bruce, No I did not specify the # of open mp threads on the recon-all cmd line. These run times were obtained by running one subject per core. for example cc2.8xlarge has 8 cores and so we ran 8 subjects at once; Thanks for the info about the # of open mp threads options; I will look into it.

One other note: Bruce, Nick did you improve the memory management in 5.2 ? On our local machine we noticed we can run 6 subjects simultaneously even though we only have 12gb of ram. I thought some of the them might crash since we only have 2gb per subject but no crashes so far over 30 subjects.. Mehul

On Fri, Feb 1, 2013 at 7:12 PM, Bruce Fischl <fischl@nmr.mgh.harvard.edumailto:fischl@nmr.mgh.harvard.edu> wrote: Hi Mehul

did you specify the # of open mp threads on the recon-all cmd line? cheers Bruce

On Fri, 1 Feb 2013, Mehul Sampat wrote: Hi Folks, Just wanted to share our experience with running FS 5.2-beta on Amazon Web Services (AWS). Basically, AWS has multiple instance types (http://aws.amazon.com/ec2/instance-types/) and we were trying to figure out the most cost-effective approach.

We ran two subjects through FS 5.2-beta on M1 Large Instance (m1.large) and Cluster Compute Eight Extra Large Instance (cc2.8xlarge). (same subjects run on both instance). We expected cc2.8xlarge to be faster (but it is also more expensive: $2.4 per hour; 8 cores); The run-times we got:

instance-type subject start-time end-time run-time m1.large subject-1 01:05:44 UTC 2013 15:40:45 UTC 2013 ~14hr-35mins m1.large subject-2 01:06:06 UTC 2013 15:08:45 UTC 2013 ~14hr-02mins cc2.8xlarge subject-1 01:26:38 UTC 2013 12:30:23 UTC 2013 ~11hr-04mins cc2.8xlarge subject-2 01:27:28 UTC 2013 12:19:08 UTC 2013 ~10hr-52mins

Although m1.large is a few hours slower, it seems to be the more cost effective option since it is $0.24 per hour (2 cores). If you have run Freesurfer on AWS, do you have a similar experience ? Any suggestions to speed up the run-times on AWS would be very helpful.

Thanks Mehul

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edumailto:Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

________________________________ This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator