We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards, Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer- bounces@nmr.mgh.harvard.edu] On Behalf Of Jens Offenbach Sent: Wednesday, November 19, 2014 5:47 AM To: freesurfer@nmr.mgh.harvard.edu Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0- patch
We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
I have corresponding setup modules for auto-deployment and configuration in the cloud, but it still takes some time (even in the local cloud network) to transfer the archive from the fileserver to the virtual machine and especially to untar that archive. A manual untar took nearly 20 minutes, that's too slow for on-demand provisioning. I think the "subjects" folder can be removed. It seems to contain only example mri data, but I want to make sure not to remove anything that is actually required for analysis purposes. I need a minimum archive file which is able to perform the pipeline execution.
Yes, we have a cloud portal, where different consumers can request the freesurfer service after uploading their mri raw files. The virtual machines are created and configured on the fly starting from a raw OS image. When the freesurfer run terminates the virtual machines are destroyed and the whole procedures starts again if someone requests a new freesurfer service (Create and start a virtual machine, transfer freesurfer archive, untar that archive, set environmement variables ...). We only want to parallelize different freesurfer jobs and not the freesurfer pipeline itself. We do not have a medical background.
Regards, Jens
Gesendet: Mittwoch, 19. November 2014 um 13:14 Uhr Von: "Krieger, Donald N." kriegerd@upmc.edu An: "'Freesurfer support list'" freesurfer@nmr.mgh.harvard.edu Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards, Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer- bounces@nmr.mgh.harvard.edu] On Behalf Of Jens Offenbach Sent: Wednesday, November 19, 2014 5:47 AM To: freesurfer@nmr.mgh.harvard.edu Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0- patch
We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
hi jens,
if you search the archives, there is a post that lists the minimal set of files needed for running recon-all.
i used a tool called cde to create it: http://www.pgbovine.net/cde.html
this reduces the archive down to about 650MB comprising binaries and atlases.
this wouldn't be for general purpose freesurfer processing, just for running recon-all.
cheers,
satra
On Wed, Nov 19, 2014 at 8:40 AM, Jens Offenbach wolle5050@gmx.de wrote:
I have corresponding setup modules for auto-deployment and configuration in the cloud, but it still takes some time (even in the local cloud network) to transfer the archive from the fileserver to the virtual machine and especially to untar that archive. A manual untar took nearly 20 minutes, that's too slow for on-demand provisioning. I think the "subjects" folder can be removed. It seems to contain only example mri data, but I want to make sure not to remove anything that is actually required for analysis purposes. I need a minimum archive file which is able to perform the pipeline execution.
Yes, we have a cloud portal, where different consumers can request the freesurfer service after uploading their mri raw files. The virtual machines are created and configured on the fly starting from a raw OS image. When the freesurfer run terminates the virtual machines are destroyed and the whole procedures starts again if someone requests a new freesurfer service (Create and start a virtual machine, transfer freesurfer archive, untar that archive, set environmement variables ...). We only want to parallelize different freesurfer jobs and not the freesurfer pipeline itself. We do not have a medical background.
Regards, Jens
Gesendet: Mittwoch, 19. November 2014 um 13:14 Uhr Von: "Krieger, Donald N." kriegerd@upmc.edu An: "'Freesurfer support list'" freesurfer@nmr.mgh.harvard.edu Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards,
Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer- bounces@nmr.mgh.harvard.edu] On Behalf Of Jens Offenbach Sent: Wednesday, November 19, 2014 5:47 AM To: freesurfer@nmr.mgh.harvard.edu Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and
5.3.0-
patch
We are currently working on a solution that brings FreeSurfer in the
cloud.
Unfortunately, we are facing a lot of problems. Hopefully, someone can
help
me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is
required to
download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu
family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how
to apply
it or if I must apply it? The Readme says to replace “mri_segstats”.
What about
the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files
which are
necessary to execute the pipeline without any viewer or examples. Which
files
and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline
execution. I know
that the scripts folder contains some status files which offer important
data
helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately
20
hours, I have to say something about the progress within this current
step. Is
there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its
work
done so far which means that the overall pipeline progress is e.g. 63 %.
Is it
possible to say that a specific step takes a specific portion of the
overall
pipeline execution time. What influences this portion (format or quality
of input,
platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom
it is
addressed. If you believe this e-mail was sent to you in error and the
contains patient information, please contact the Partners Compliance
HelpLine
at
http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...] . If the e-mail was sent to you in
error but does not contain patient information, please contact the
sender and
properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Hi satra, I have tried to find some posts about the minimum files required to run recon-all, but I cannot find anything. Could you please try to find the post in the archives and provide me the link?
Thank you very much!
Gesendet: Mittwoch, 19. November 2014 um 15:23 Uhr Von: "Satrajit Ghosh" satra@mit.edu An: "Freesurfer support list" freesurfer@nmr.mgh.harvard.edu Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
hi jens, if you search the archives, there is a post that lists the minimal set of files needed for running recon-all. i used a tool called cde to create it: http://www.pgbovine.net/cde.html this reduces the archive down to about 650MB comprising binaries and atlases. this wouldn't be for general purpose freesurfer processing, just for running recon-all.
cheers,
satra On Wed, Nov 19, 2014 at 8:40 AM, Jens Offenbach <wolle5050@gmx.de[wolle5050@gmx.de]> wrote:I have corresponding setup modules for auto-deployment and configuration in the cloud, but it still takes some time (even in the local cloud network) to transfer the archive from the fileserver to the virtual machine and especially to untar that archive. A manual untar took nearly 20 minutes, that's too slow for on-demand provisioning. I think the "subjects" folder can be removed. It seems to contain only example mri data, but I want to make sure not to remove anything that is actually required for analysis purposes. I need a minimum archive file which is able to perform the pipeline execution.
Yes, we have a cloud portal, where different consumers can request the freesurfer service after uploading their mri raw files. The virtual machines are created and configured on the fly starting from a raw OS image. When the freesurfer run terminates the virtual machines are destroyed and the whole procedures starts again if someone requests a new freesurfer service (Create and start a virtual machine, transfer freesurfer archive, untar that archive, set environmement variables ...). We only want to parallelize different freesurfer jobs and not the freesurfer pipeline itself. We do not have a medical background.
Regards, Jens
Gesendet: Mittwoch, 19. November 2014 um 13:14 Uhr Von: "Krieger, Donald N." <kriegerd@upmc.edu[kriegerd@upmc.edu]> An: "'Freesurfer support list'" <freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu]> Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards, Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu[freesurfer-bounces@nmr.mgh.harvard.edu] [mailto:freesurfer-[freesurfer-] bounces@nmr.mgh.harvard.edu[bounces@nmr.mgh.harvard.edu]] On Behalf Of Jens Offenbach Sent: Wednesday, November 19, 2014 5:47 AM To: freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu] Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0- patch
We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine> at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...]] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer________________... Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....] The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
hi jens,
here is the post - the links were all broken, but i've updated them below. note that this run was from nov 2013. i would recommend doing a run with all the current patches installed.
---- an update on the files needed for a vanilla recon-all run (evaluation done using cde:http://www.pgbovine.net/cde.html):
freesurfer files: https://www.dropbox.com/s/ei3m9dcvq8zemps/fslist.txt?dl=0 all files (fs + system+ output): https://www.dropbox.com/s/1rjk7s0qyzn2hm5/allfiles.txt?dl=0 size of fs components (641M): https://www.dropbox.com/s/mmcadnp63ao4dv8/size.txt?dl=0
one thing that surprised me a bit was that no component of fsaverage appeared to be used (other than the directory being touched). this could be something the program missed, or is actually true. ----
cheers,
satra
On Wed, Nov 19, 2014 at 5:03 PM, Jens Offenbach wolle5050@gmx.de wrote:
Hi satra, I have tried to find some posts about the minimum files required to run recon-all, but I cannot find anything. Could you please try to find the post in the archives and provide me the link?
Thank you very much!
Gesendet: Mittwoch, 19. November 2014 um 15:23 Uhr Von: "Satrajit Ghosh" satra@mit.edu An: "Freesurfer support list" freesurfer@nmr.mgh.harvard.edu Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
hi jens,
if you search the archives, there is a post that lists the minimal set of files needed for running recon-all.
i used a tool called cde to create it: http://www.pgbovine.net/cde.html
this reduces the archive down to about 650MB comprising binaries and atlases.
this wouldn't be for general purpose freesurfer processing, just for running recon-all.
cheers,
satra
On Wed, Nov 19, 2014 at 8:40 AM, Jens Offenbach <wolle5050@gmx.de[ wolle5050@gmx.de]> wrote:I have corresponding setup modules for auto-deployment and configuration in the cloud, but it still takes some time (even in the local cloud network) to transfer the archive from the fileserver to the virtual machine and especially to untar that archive. A manual untar took nearly 20 minutes, that's too slow for on-demand provisioning. I think the "subjects" folder can be removed. It seems to contain only example mri data, but I want to make sure not to remove anything that is actually required for analysis purposes. I need a minimum archive file which is able to perform the pipeline execution.
Yes, we have a cloud portal, where different consumers can request the freesurfer service after uploading their mri raw files. The virtual machines are created and configured on the fly starting from a raw OS image. When the freesurfer run terminates the virtual machines are destroyed and the whole procedures starts again if someone requests a new freesurfer service (Create and start a virtual machine, transfer freesurfer archive, untar that archive, set environmement variables ...). We only want to parallelize different freesurfer jobs and not the freesurfer pipeline itself. We do not have a medical background.
Regards, Jens
Gesendet: Mittwoch, 19. November 2014 um 13:14 Uhr Von: "Krieger, Donald N." <kriegerd@upmc.edu[kriegerd@upmc.edu]> An: "'Freesurfer support list'" <freesurfer@nmr.mgh.harvard.edu[ freesurfer@nmr.mgh.harvard.edu]> Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards,
Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu[
freesurfer-bounces@nmr.mgh.harvard.edu] [mailto:freesurfer-[freesurfer-]
bounces@nmr.mgh.harvard.edu[bounces@nmr.mgh.harvard.edu]] On Behalf Of
Jens Offenbach
Sent: Wednesday, November 19, 2014 5:47 AM To: freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu] Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and
5.3.0-
patch
We are currently working on a solution that brings FreeSurfer in the
cloud.
Unfortunately, we are facing a lot of problems. Hopefully, someone can
help
me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is
required to
download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu
family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how
to apply
it or if I must apply it? The Readme says to replace “mri_segstats”.
What about
the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files
which are
necessary to execute the pipeline without any viewer or examples. Which
files
and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline
execution. I know
that the scripts folder contains some status files which offer important
data
helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately
20
hours, I have to say something about the progress within this current
step. Is
there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its
work
done so far which means that the overall pipeline progress is e.g. 63 %.
Is it
possible to say that a specific step takes a specific portion of the
overall
pipeline execution time. What influences this portion (format or quality
of input,
platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
The information in this e-mail is intended only for the person to whom
it is
addressed. If you believe this e-mail was sent to you in error and the
contains patient information, please contact the Partners Compliance
HelpLine> at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...]] . If the e-mail was sent to you in
error but does not contain patient information, please contact the
sender and
properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer________________... Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....] The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Hi Satra, that was really helpful... I patched 5.3.0, made a full run of recon-all with strace and wrote a little tool that checks which files must actually be part of the archive.
I can confirm your file list, but there is a problem with fsaverage (you have already expressed your concerns) because some files are missing. Have a look to my subsequent list. I post here the minimum content of the freesurfer archive for getting recon-all -all running. I am currently testing the version in my cloud using Ubuntu 14.04 (x86_64) and Ubuntu 14.10 (x86_64).
freesurfer_linux_x86_64_5.3.1_pipeline.tar:
freesurfer/ freesurfer/lib/ freesurfer/lib/bem/ freesurfer/lib/bem/ic4.tri freesurfer/lib/bem/ic5.tri freesurfer/lib/bem/ic7.tri freesurfer/SetUpFreeSurfer.sh freesurfer/FreeSurferEnv.csh freesurfer/bin/ freesurfer/bin/mri_and freesurfer/bin/compute_vox2vox freesurfer/bin/mris_curvature freesurfer/bin/mrisp_paint freesurfer/bin/mri_label2label freesurfer/bin/mri_add_xform_to_header freesurfer/bin/analyzeto4dfp freesurfer/bin/mri_info freesurfer/bin/mris_inflate freesurfer/bin/mris_ca_train freesurfer/bin/mri_convert freesurfer/bin/mri_pretess freesurfer/bin/talairach_afd freesurfer/bin/mri_normalize freesurfer/bin/mri_fill freesurfer/bin/mri_fuse_segmentations freesurfer/bin/mris_jacobian freesurfer/bin/mris_curvature_stats freesurfer/bin/mri_aparc2aseg freesurfer/bin/mri_cc freesurfer/bin/extract_talairach_avi_QA.awk freesurfer/bin/mri_robust_register freesurfer/bin/mris_ca_label freesurfer/bin/mri_watershed freesurfer/bin/mris_calc freesurfer/bin/mri_remove_neck freesurfer/bin/mris_make_surfaces freesurfer/bin/mri_ca_label freesurfer/bin/mri_vol2surf freesurfer/bin/mri_gcut freesurfer/bin/pctsurfcon freesurfer/bin/mris_remove_intersection freesurfer/bin/mri_nu_correct.mni freesurfer/bin/imgreg_4dfp freesurfer/bin/mri_motion_correct.fsl freesurfer/bin/mris_smooth freesurfer/bin/mris_sphere freesurfer/bin/mri_or freesurfer/bin/mri_binarize freesurfer/bin/fsr-getxopts freesurfer/bin/mris_euler_number freesurfer/bin/mpr2mni305 freesurfer/bin/mris_anatomical_stats freesurfer/bin/mri_segstats freesurfer/bin/mri_robust_template freesurfer/bin/mri_edit_wm_with_aseg freesurfer/bin/mri_ca_register freesurfer/bin/tkregister2_cmdl freesurfer/bin/mris_spherical_average freesurfer/bin/mri_concatenate_lta freesurfer/bin/recon-all freesurfer/bin/mri_make_uchar freesurfer/bin/mri_em_register freesurfer/bin/gauss_4dfp freesurfer/bin/mri_matrix_multiply freesurfer/bin/freesurfer freesurfer/bin/mris_volmask freesurfer/bin/talairach_avi freesurfer/bin/mris_extract_main_component freesurfer/bin/mris_label2annot freesurfer/bin/mri_segment freesurfer/bin/mris_fix_topology freesurfer/bin/mri_concat freesurfer/bin/mris_topo_fixer freesurfer/bin/avi2talxfm freesurfer/bin/ifh2hdr freesurfer/bin/flirt.fsl freesurfer/bin/mris_register freesurfer/bin/mri_normalize_tp2 freesurfer/bin/mri_mask freesurfer/bin/mri_tessellate freesurfer/bin/mri_ca_normalize freesurfer/fsafd/ freesurfer/fsafd/TalairachingProbas.adf freesurfer/fsafd/TalairachingCovariance.adf freesurfer/fsafd/TalairachingMean.adf freesurfer/fsfast/ freesurfer/FreeSurferEnv.sh freesurfer/WMParcStatsLUT.txt freesurfer/SetUpFreeSurfer.csh freesurfer/build-stamp.txt freesurfer/average/ freesurfer/average/RB_all_withskull_2008-03-26.gca freesurfer/average/rigidly_aligned_brain_template.tif freesurfer/average/colortable_BA.txt freesurfer/average/lh.average.curvature.filled.buckner40.tif freesurfer/average/711-2C_as_mni_average_305.4dfp.img freesurfer/average/lh.destrieux.simple.2009-07-29.gcs freesurfer/average/711-2C_as_mni_average_305.4dfp.ifh freesurfer/average/711-2B_as_mni_average_305_mask.4dfp.img freesurfer/average/rh.curvature.buckner40.filled.desikan_killiany.2010-03-25.gcs freesurfer/average/rh.destrieux.simple.2009-07-29.gcs freesurfer/average/RB_all_2008-03-26.gca freesurfer/average/rh.DKTatlas40.gcs freesurfer/average/lh.DKTatlas40.gcs freesurfer/average/mni305.cor.mgz freesurfer/average/lh.curvature.buckner40.filled.desikan_killiany.2010-03-25.gcs freesurfer/average/rh.average.curvature.filled.buckner40.tif freesurfer/average/711-2B_as_mni_average_305_mask.4dfp.ifh freesurfer/subjects/ freesurfer/subjects/rh.EC_average/ freesurfer/subjects/rh.EC_average/label/ freesurfer/subjects/rh.EC_average/label/rh.entorhinal.label freesurfer/subjects/rh.EC_average/surf/ freesurfer/subjects/rh.EC_average/surf/rh.white freesurfer/subjects/rh.EC_average/surf/rh.sphere.reg freesurfer/subjects/fsaverage/ freesurfer/subjects/fsaverage/label/ freesurfer/subjects/fsaverage/label/lh.BA3b.thresh.label freesurfer/subjects/fsaverage/label/rh.V1.label freesurfer/subjects/fsaverage/label/rh.BA4a.thresh.label freesurfer/subjects/fsaverage/label/lh.V2.thresh.label freesurfer/subjects/fsaverage/label/lh.BA45.thresh.label freesurfer/subjects/fsaverage/label/rh.BA1.thresh.label freesurfer/subjects/fsaverage/label/rh.BA3a.label freesurfer/subjects/fsaverage/label/rh.BA3b.label freesurfer/subjects/fsaverage/label/lh.BA3a.thresh.label freesurfer/subjects/fsaverage/label/lh.V1.thresh.label freesurfer/subjects/fsaverage/label/lh.V1.label freesurfer/subjects/fsaverage/label/rh.BA3a.thresh.label freesurfer/subjects/fsaverage/label/rh.BA44.label freesurfer/subjects/fsaverage/label/rh.V1.thresh.label freesurfer/subjects/fsaverage/label/rh.BA1.label freesurfer/subjects/fsaverage/label/rh.MT.thresh.label freesurfer/subjects/fsaverage/label/lh.BA44.thresh.label freesurfer/subjects/fsaverage/label/lh.BA2.label freesurfer/subjects/fsaverage/label/lh.BA4a.thresh.label freesurfer/subjects/fsaverage/label/lh.perirhinal.label freesurfer/subjects/fsaverage/label/lh.MT.thresh.label freesurfer/subjects/fsaverage/label/lh.BA6.label freesurfer/subjects/fsaverage/label/rh.BA2.label freesurfer/subjects/fsaverage/label/lh.BA1.label freesurfer/subjects/fsaverage/label/lh.BA4a.label freesurfer/subjects/fsaverage/label/rh.BA4a.label freesurfer/subjects/fsaverage/label/lh.V2.label freesurfer/subjects/fsaverage/label/rh.MT.label freesurfer/subjects/fsaverage/label/lh.BA45.label freesurfer/subjects/fsaverage/label/lh.BA4p.label freesurfer/subjects/fsaverage/label/rh.BA45.thresh.label freesurfer/subjects/fsaverage/label/lh.MT.label freesurfer/subjects/fsaverage/label/rh.BA3b.thresh.label freesurfer/subjects/fsaverage/label/rh.BA2.thresh.label freesurfer/subjects/fsaverage/label/rh.BA4p.thresh.label freesurfer/subjects/fsaverage/label/rh.BA4p.label freesurfer/subjects/fsaverage/label/lh.BA44.label freesurfer/subjects/fsaverage/label/rh.perirhinal.label freesurfer/subjects/fsaverage/label/lh.BA1.thresh.label freesurfer/subjects/fsaverage/label/rh.BA44.thresh.label freesurfer/subjects/fsaverage/label/lh.BA4p.thresh.label freesurfer/subjects/fsaverage/label/rh.V2.thresh.label freesurfer/subjects/fsaverage/label/rh.BA45.label freesurfer/subjects/fsaverage/label/rh.V2.label freesurfer/subjects/fsaverage/label/lh.BA3a.label freesurfer/subjects/fsaverage/label/lh.BA2.thresh.label freesurfer/subjects/fsaverage/label/rh.BA6.label freesurfer/subjects/fsaverage/label/lh.BA3b.label freesurfer/subjects/fsaverage/label/lh.BA6.thresh.label freesurfer/subjects/fsaverage/label/rh.BA6.thresh.label freesurfer/subjects/fsaverage/mri/ freesurfer/subjects/fsaverage/mri/transforms/ freesurfer/subjects/fsaverage/mri/transforms/talairach.xfm freesurfer/subjects/fsaverage/surf/ freesurfer/subjects/fsaverage/surf/lh.sphere.reg freesurfer/subjects/fsaverage/surf/rh.sphere.reg freesurfer/subjects/lh.EC_average/ freesurfer/subjects/lh.EC_average/label/ freesurfer/subjects/lh.EC_average/label/lh.entorhinal.label freesurfer/subjects/lh.EC_average/surf/ freesurfer/subjects/lh.EC_average/surf/lh.sphere.reg freesurfer/subjects/lh.EC_average/surf/lh.white freesurfer/ASegStatsLUT.txt freesurfer/mni/ freesurfer/mni/lib/ freesurfer/mni/lib/perl5/ freesurfer/mni/lib/perl5/5.8.5/ freesurfer/mni/lib/perl5/5.8.5/MNI/ freesurfer/mni/lib/perl5/5.8.5/MNI/Spawn.pm freesurfer/mni/lib/perl5/5.8.5/MNI/Startup.pm freesurfer/mni/lib/perl5/5.8.5/MNI/MiscUtilities.pm freesurfer/mni/lib/perl5/5.8.5/MNI/Batch.pm freesurfer/mni/lib/perl5/5.8.5/MNI/NumericUtilities.pm freesurfer/mni/lib/perl5/5.8.5/MNI/Spawn.pod freesurfer/mni/lib/perl5/5.8.5/MNI/TagPoint.pm freesurfer/mni/lib/perl5/5.8.5/MNI/PathUtilities.pm freesurfer/mni/lib/perl5/5.8.5/MNI/FileUtilities.pm freesurfer/mni/lib/perl5/5.8.5/MNI/MincUtilities.pm freesurfer/mni/lib/perl5/5.8.5/MNI/DataDir.pm freesurfer/mni/lib/perl5/5.8.5/MNI/TagSet.pm freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/ freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/perllocal.pod freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/ freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/mni_perllib/ freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/mni_perllib/.packlist freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/Getopt/ freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/Getopt/Tabular/ freesurfer/mni/lib/perl5/5.8.5/i386-linux-thread-multi/auto/Getopt/Tabular/.packlist freesurfer/mni/lib/perl5/5.8.5/Getopt/ freesurfer/mni/lib/perl5/5.8.5/Getopt/Tabular.pm freesurfer/mni/lib/perl5/5.8.5/Getopt/Tabular.pod freesurfer/mni/lib/perl5/5.8.5/mni_perllib.pod freesurfer/mni/lib/perl5/5.8.5/MNI.pm freesurfer/mni/lib/perl5/5.8.5/auto/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/update_history.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/get_history.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/percent_threshold.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/autosplit.ix freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/volume_params.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/volume_cog.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/put_history.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/volume_max.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/volume_min.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/get_dimension_order.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/compute_resample_args.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/compute_reshape_args.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/volume_minmax.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MincUtilities/auto_threshold.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/split_path.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/autosplit.ix freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/replace_dir.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/expand_path.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/merge_paths.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/replace_ext.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/PathUtilities/normalize_dirs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/NumericUtilities/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/NumericUtilities/autosplit.ix freesurfer/mni/lib/perl5/5.8.5/auto/MNI/NumericUtilities/labs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/NumericUtilities/in_range.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/NumericUtilities/round.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/statfs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/search_directories.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/check_output_dirs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/autosplit.ix freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/generate_numbered_filename.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/find_program.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/find_programs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/check_files.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/check_input_dirs.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/test_file.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/FileUtilities/check_output_path.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/ freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/timestamp.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/nlist_equal.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/make_banner.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/autosplit.ix freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/userstamp.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/shellquote.al freesurfer/mni/lib/perl5/5.8.5/auto/MNI/MiscUtilities/lcompare.al freesurfer/mni/bin/ freesurfer/mni/bin/mincmath freesurfer/mni/bin/mincinfo freesurfer/mni/bin/nu_correct freesurfer/mni/bin/mincresample freesurfer/mni/bin/evaluate_field freesurfer/mni/bin/volume_stats freesurfer/mni/bin/minc_modify_header freesurfer/mni/bin/make_template freesurfer/mni/bin/sharpen_volume freesurfer/mni/bin/minclookup freesurfer/mni/bin/resample_labels freesurfer/mni/bin/spline_smooth freesurfer/mni/bin/volume_hist freesurfer/mni/bin/nu_evaluate freesurfer/mni/bin/sharpen_hist freesurfer/mni/bin/nu_estimate_np_and_em freesurfer/mni/share/ freesurfer/mni/share/N3/ freesurfer/mni/share/N3/average_305_mask_1mm.mnc.gz
Regards, Christoph
Gesendet: Mittwoch, 19. November 2014 um 23:49 Uhr Von: "Satrajit Ghosh" satra@mit.edu An: "Freesurfer support list" freesurfer@nmr.mgh.harvard.edu Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
hi jens, here is the post - the links were all broken, but i've updated them below. note that this run was from nov 2013. i would recommend doing a run with all the current patches installed.
---- an update on the files needed for a vanilla recon-all run (evaluation done using cde:http://www.pgbovine.net/cde.html): freesurfer files: https://www.dropbox.com/s/ei3m9dcvq8zemps/fslist.txt?dl=0%5Bhttps://www.drop...] all files (fs + system+ output): https://www.dropbox.com/s/1rjk7s0qyzn2hm5/allfiles.txt?dl=0%5Bhttps://www.dr...] size of fs components (641M): https://www.dropbox.com/s/mmcadnp63ao4dv8/size.txt?dl=0%5Bhttps://www.dropbo...] one thing that surprised me a bit was that no component of fsaverage appeared to be used (other than the directory being touched). this could be something the program missed, or is actually true. ----
cheers,
satra On Wed, Nov 19, 2014 at 5:03 PM, Jens Offenbach <wolle5050@gmx.de[wolle5050@gmx.de]> wrote:Hi satra, I have tried to find some posts about the minimum files required to run recon-all, but I cannot find anything. Could you please try to find the post in the archives and provide me the link?
Thank you very much!
Gesendet: Mittwoch, 19. November 2014 um 15:23 Uhr Von: "Satrajit Ghosh" <satra@mit.edu[satra@mit.edu]> An: "Freesurfer support list" <freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu]> Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
hi jens, if you search the archives, there is a post that lists the minimal set of files needed for running recon-all. i used a tool called cde to create it: http://www.pgbovine.net/cde.html%5Bhttp://www.pgbovine.net/cde.html] this reduces the archive down to about 650MB comprising binaries and atlases. this wouldn't be for general purpose freesurfer processing, just for running recon-all.
cheers,
satra On Wed, Nov 19, 2014 at 8:40 AM, Jens Offenbach <wolle5050@gmx.de[wolle5050@gmx.de][wolle5050@gmx.de[wolle5050@gmx.de]]> wrote:I have corresponding setup modules for auto-deployment and configuration in the cloud, but it still takes some time (even in the local cloud network) to transfer the archive from the fileserver to the virtual machine and especially to untar that archive. A manual untar took nearly 20 minutes, that's too slow for on-demand provisioning. I think the "subjects" folder can be removed. It seems to contain only example mri data, but I want to make sure not to remove anything that is actually required for analysis purposes. I need a minimum archive file which is able to perform the pipeline execution.
Yes, we have a cloud portal, where different consumers can request the freesurfer service after uploading their mri raw files. The virtual machines are created and configured on the fly starting from a raw OS image. When the freesurfer run terminates the virtual machines are destroyed and the whole procedures starts again if someone requests a new freesurfer service (Create and start a virtual machine, transfer freesurfer archive, untar that archive, set environmement variables ...). We only want to parallelize different freesurfer jobs and not the freesurfer pipeline itself. We do not have a medical background.
Regards, Jens
Gesendet: Mittwoch, 19. November 2014 um 13:14 Uhr Von: "Krieger, Donald N." <kriegerd@upmc.edu[kriegerd@upmc.edu][kriegerd@upmc.edu[kriegerd@upmc.edu]]> An: "'Freesurfer support list'" <freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu][freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu]]> Betreff: Re: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0-patch
Dear Jens,
I can offer just a bit of information and I also have a question. We are using Fedora 11.0 on an x86_64 box and both the Centos 4 and Centos 6 compiles work. The package appears to be self-contained as I have seen no faults on failures to find shared libraries. We simply untar freesurfer to a convenient location and set the PATH and FREESURFERHOME environment variables accordingly. I see no reason why that shouldn't work in a cloud solution except that I do not know what you have in mind as a use for your solution. Will there be a group of users who are provided access so that they can run freesurfer on an image data set through a portal to your cloud? Are you using the cloud to distribute the calculation to many machines? If so, what is the "granularity" of this, e.g. does one freesurfer run get one machine?
Regards, Don
Don Krieger, Ph.D. Department of Neurological Surgery University of Pittsburgh (412)648-9654 Office (412)521-4431 Cell/Text
-----Original Message----- From: freesurfer-bounces@nmr.mgh.harvard.edu[freesurfer-bounces@nmr.mgh.harvard.edu][freesurfer-bounces@nmr.mgh.harvard.edu[freesurfer-bounces@nmr.mgh.harvard.edu]] [mailto:freesurfer-[freesurfer-][freesurfer-] bounces@nmr.mgh.harvard.edu[bounces@nmr.mgh.harvard.edu][bounces@nmr.mgh.harvard.edu[bounces@nmr.mgh.harvard.edu]]] On Behalf Of Jens Offenbach Sent: Wednesday, November 19, 2014 5:47 AM
To: freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu][freesurfer@nmr.mgh.harvard.edu[freesurfer@nmr.mgh.harvard.edu]] Subject: [Freesurfer] Pipeline/Step Progress Monitoring, Packaging and 5.3.0- patch
We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list> Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu][Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]]
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine> at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...]]] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu][Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....]]
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu][Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu]] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer________________...] Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....] The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...]] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu[Freesurfer@nmr.mgh.harvard.edu] https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer________________... Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer%5Bhttps://mail....] The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline%5Bhttp://www.partners.org/compliancel...] . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Hi Jens
you could estimate it, but it might not be exactly right. There's no way to know how long it will take to converge as it depends on the details of the anatomy. If the current output is not verbose enough for you, you can turn on more output by:
setenv DIAG 0x04040
if you run some recons with this on you'll be able to get an idea of how many integration steps each pipeline step takes
cheers Bruce
On Wed, 19 Nov 2014, Jens Offenbach wrote:
We are currently working on a solution that brings FreeSurfer in the cloud. Unfortunately, we are facing a lot of problems. Hopefully, someone can help me:
I want to run FreeSurfer on Ubuntu 14.04 or 14.10. What package is required to download (CentOS 4 x86_64 (64b))? Does it work anyway on the Ubuntu family?
On the FTP server I have found a patch for 5.3.0, but I am not sure how to apply it or if I must apply it? The Readme says to replace “mri_segstats”. What about the other files “recon-all, fsfast, tksurferfv, …”?
I want to reduce the archive to its bare minimum, otherwise the automated deployment process takes too long. It should contain only those files which are necessary to execute the pipeline without any viewer or examples. Which files and folders can be deleted safely?
The most important issue is to monitor the FreeSurfer pipeline execution. I know that the scripts folder contains some status files which offer important data helping me to monitor the pipeline execution progress of “recon-all”. Unfortunately, that is not enough… If one step takes e.g. approximately 20 hours, I have to say something about the progress within this current step. Is there any possibility to monitor the progress of a single step. We need something that allows us to say e.g. that Step “CA Reg” has 34 % of its work done so far which means that the overall pipeline progress is e.g. 63 %. Is it possible to say that a specific step takes a specific portion of the overall pipeline execution time. What influences this portion (format or quality of input, platform, …)?
Thank you very much for your help!
Regards Jens
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
freesurfer@nmr.mgh.harvard.edu
-
Bruce Fischl -
Jens Offenbach -
Krieger, Donald N. -
Satrajit Ghosh