Hi Darren, Nick and freesurfers,
Thanks very much for your replies! I've discussed this and have a couple of questions. I would be very glad if you could help me out.
I'm currently talking with my group to decide where to go. Like I said, we will get a research-only 3T scanner, which (if used as much as we hope) will deliver about 20 GB of raw data each week. We need to think of a solid processing and storage pipeline in order to be able to do something with the data ;)
We discussed this and are trying to come up with a good solution, both storage and analyses-wise. One solution for the enhancement of the analyses would be to buy a couple of solid workstations. The other would be a grid solution. The main problem isn't as much finance as it is manpower. No current member of the faculty has the time to invest in setting up and maintaining a grid if that will be a very difficult task to do. Hopefully we will get the money to hire an ICT person to do this. However, this is not certain. So I have a question concerning the difficulty of setting up the grid like proposed by Nick: semi-parallel in the sense that different commands of a batch are distributed to different nodes. We are wondering just how difficult this is.
I address this question to both of you. Darren: you have SUN grid and Nick you have something like SUN grid. Nick, you state:
<snip> You are on-track with the first scenario. If you have the technical skills and time to install the Sun Grid Engine, or PBS (which does nearly the same thing), then the batch interface will be easy for others to submit as many 'recon-all' instances as they want and will handle balancing. Once a batch system is setup, you can expect to have to spend some time maintaining the system (help-line, monitoring for stalled jobs, failed nodes, or failed disks, etc). You could also write your own scripts implementing a simple batch system, if you don't mind semi-manually tracking the resources, and if you will oversee it or use it yourself, and keep it small-scale. </snip>
The issue of course is this:
1. Just how much technical skills and time does the installation and maintenance of a Grid cost? It is my impression that the implementation of the SUN grid is not that hard. But I'm always an optimist if it comes to computers, although at times this has proven to be a great overestimation of my own knowledge and skills.
2. How 'intuitive' is the usage of a batch system like SUN grid and how stable is it? That is: given that we write guidelines, howtos, and set up a Wiki, how difficult will drag&drop type users will be able to work with such a system?
The final maintenance issues that Nick addresses concerns failure.
3. Hardware failure: freezing nodes and disk failures I expect to cost as much time as it will cost when we buy individual workstations, Maybe even less?
4. Software failure. Maybe I'm way off here, but it's my impression that the host running the batch system uploads the input data from a central server to a node, which analyses the data and then downloads the resulting output again, deleting the data on the node. If this is correct and if a single job fails, do you have to manually delete working data on a node?
I hope you can help me out with this.
Thank you!
Andries