Thanks again Pedro. I just ran 220 brains through in day or two using the cloud. Fantastic. The script people have been talking about would have been useful. One thing I didn't realize was that I should have moved the freesurfer subject directory from /usr/local/freesurfer/subjects to the larger disk that comes with the instance. I had disk-write errors because the the main partition ran out of space.
Anyway, ec2 rocks with Freesurfer. - Joshua Lee Graduate Student Center for Mind and Brain & Department of Psychology University of California, Davis 530.747.3805
2011/11/12 Nolan Nichols bnniii@uw.edu
If only running a small number of instances, I think it is reasonable to leave the ami w/o a .license file since it is easy enough to scp it up to the cloud. For running a cluster it would be nice to have a script, perhaps using Fabric, to upload a .license file to each instance in a cluster. As it stands, the starcluster FS AMI does not have a .license file and requires one to be uploaded manually.
Nolan
2011/11/12 Thomas Ballinger thomasballinger@gmail.com:
Perhaps the AMI image could be without a licence file and uploading one (acquired though normal means) would be required? (I suppose that's the basic solution we're looking for a clean way around)
2011/11/12 Pedro Paulo de Magalhães Oliveira Junior <
ppj@netfilter.com.br>
Maybe we can set up a web server in the Martinos center and the AMI
Image
can authenticate in it.
Pedro Paulo de Magalhães Oliveira Junior Netfilter & SpeedComm Telecom -- www.netfilter.com.br -- For mobile: http://itunes.apple.com/br/artist/netfilter/id365306441
On Sat, Nov 12, 2011 at 15:28, Nick Schmansky <
nicks@nmr.mgh.harvard.edu>
wrote:
I'm happy to see the work getting the cloud to be feasible. I still
need
to catch up on this thread but one issue that I'd like to sort out is tracking the license count. Its fine for a new user to use an existing image, but I think there will need to be a way to remind people to
request
our .license file even if they don't actually use the file so that
cloud
usages get counted (license count is critical for us for grant renewal purposes). Ideas on a clean way to do this? N.
On Nov 12, 2011, at 11:10 AM, Thomas Ballinger thomasballinger@gmail.com wrote:
Satra, the ipython example looks great. The scripts I posted also use fabric for running Freesurfer, but don't download Freesurfer due to it already being installed on Pedro's image. I agree with Satra that individually administered instances might work better for those
wishing to
run a few cases with Freesurfer, particularly due to the predictable
long
running time (lessening the benefits of job distribution) and ability
of a
recon-all job to efficiently use all of a ec2 small instance's cpu. I
think
starcluster is pretty awesome, and I would love to push people in that direction, but it won't be quite as push-button, and the benefits may
be
minimal for small use cases. That's not to say there aren't plenty of
folks
who would rather use Nolan's solution (me for one :)) Tom
On Sat, Nov 12, 2011 at 9:14 AM, Nolan Nichols bnniii@uw.edu wrote:
Hi Satra,
Fabric does look nice, thanks for pointing it out and for a link to
the
ipython_in_a_box example.
That framework might work for what I had in mind for freesurfer and starcluster, which uses the same idea to keep the initial AMI
footprint
smaller/flexible.
Cheers,
Nolan
On Nov 12, 2011 7:32 AM, "Satrajit Ghosh" satra@mit.edu wrote:
hi nolan, i think starcluster is a great way to create a cluster on aws. but
for
those wanting to create a single instance and install things on an
ami
without actually having a large ami, fabric might be a good option. fabric: http://docs.fabfile.org/en/1.3.2/index.html example: (another neurodebian ami)
https://github.com/wholeslide/ipython_in_a_box/blob/master/fabfile.py
cheers,
satra
On Sat, Nov 12, 2011 at 6:11 AM, Nolan Nichols bnniii@uw.edu
wrote:
> > Thanks, Pedro! > > I am glad to see that you posted a public AMI for freesurfer. And > thanks to Thomas for producin documentation on how to use it. > > This thread inspired me share a StarCluster AMI (ami-27f1384e in the > us-east-1 region) that I built for batch FS jobs, which includes the > initial setup for installing tools from the NeuroDebian repository. > I've also been playing around with using s3fs, which allows Amazon's > S3 to be mounted as a drive and accessed directly as the > "subjects_dir". > > The AMI can be launched here: > >
https://console.aws.amazon.com/ec2/home?region=us-east-1#launchAmi=ami-27f13...
> > I did not include the .license for FS as I wasn't sure about
licensing
> issues, but I would be happy to publish an AMI with a built-in
license
> if I get the go ahead. For now you will need to start a cluster and > your license to each instance. > > A few details: > > Ubuntu 10.04 x86 > built from StarClusters ami-8cf913e5 > us-east-1 region > See the StarCluster details at http://web.mit.edu/stardev/cluster/ > Customizing StarCluster > > (
http://web.mit.edu/stardev/cluster/docs/latest/manual/create_new_ami.html)
> s3fs (http://code.google.com/p/s3fs/) > Freesurfer 5.1 32-bit > NeuroDebian ready (so you can apt-get install mricron, fsl, nipype, > etc.) > > Thanks again for starting this thread, and let me know if I can help > in any way. It would be great to see "cloud" resources be readily > accessibly to the community with minimal effort to get a cluster
with
> freesurfer and tools available via neurodebian up and running. > > Cheers, > > Nolan > _______________________________________________ > Freesurfer mailing list > Freesurfer@nmr.mgh.harvard.edu > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > > > The information in this e-mail is intended only for the person to
whom
> it is > addressed. If you believe this e-mail was sent to you in error and
the
> e-mail > contains patient information, please contact the Partners Compliance > HelpLine at > http://www.partners.org/complianceline . If the e-mail was sent to
you
> in error > but does not contain patient information, please contact the sender > and properly > dispose of the e-mail. >
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to
you
in error but does not contain patient information, please contact the sender
and
properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom
it
is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to
you in
error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer