External Email - Use Caution
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish. ________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
External Email - Use Caution
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1XVNJCSKr6Sjj4GWWO1Ofl3Pch5rTSqkz64DyXHUeZwMRDw...https://secure-web.cisco.com/1r-r_2QA3xAtLgXaT_9dtIg35Y8hSEE526FKCAaCPyWNLyadUJvRfaUG_xSdmhXxOrE7KQHt3U7Z5_2j30BpxJKgFJpYMzLC9upZNmMidSjQMF0QDCVohu8D6MQFZc8a-dqcC0yWQrScqMauQJhPJL2nDIGTDP8HC-Z-A_0ZSz78KuwuRn7AE-UrYRNKlAFW7bZggY0DK7fTVfwPP9sSVvIuicx46ETRUF7hHXaKaVyH2KvJqy-3YrUFoTD_VrwwHC0Nm5ZXp7Is4I-gUUwgrPqnNyttaRZmWQS_ckBSArDTkXPO7JaL3jA_Ukme0xSF5l03I919U3AMOqyoL1AHVgB0q53rMJzHJTgSnBTgbknY/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C4f2704fc4f23487cf9c408da45e7d678%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637899159291945719%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3D8XsWi5%252FNDgbyR%252FuGJuUsXl4T3fY7T3arbGqb5gw%252FG7w%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
Hi Ela - Which version of FSL are you using?
Anastasia. ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/165hEU3-piKRFhZX54xBGg8T03HRpf7Ke9yyilIyR36tHg_Pdr2d2wXQbsgHyqJ4zjtMdC74C9qNMXb_qsw_-LZvFAU4e3_DL0POgw7oxwAx3w7jF9VlEg8o2tXlot9QpUclZrOOcwOT1JPnBCBwZf1q8XOTaATmqthgfQpSVKA9WiDL8CZ3M4jPL1PxHc7fp8lYgMzLjul7EYO-OAF00C1tJ_tjuCqkQxun4ZhCvI_dCYkgeM-dhrDhTeA9JBloQz5e1jyTVy0gpp8a3IcFYCG5w-564r95-j2p1FOTVUh8_szszh-IxByQSiYMgVHU_wM7NwCY8BwFrQV-XJGVqfw/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C4f2704fc4f23487cf9c408da45e7d678%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637899159291945719%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3D8XsWi5%252FNDgbyR%252FuGJuUsXl4T3fY7T3arbGqb5gw%252FG7w%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1NZAFTC9tSMW2I9iPAXXy--QywZi9vvfgUSyPYs6BqIVlW-...https://secure-web.cisco.com/1TvoBA34REP11DUbNgGqeQ2FIgyVGb8N0u7uPJ6SzrfZbdOKsSZqWaOL-LwBkjuqEZcMHiXongCJQYIWr2TFBnpD8iPmHYMg1gYQeLputz7bm6DsBi8W9f4s_3AqGr1BtHz4LNNubelWpb4SiSkt4ZtlfSwYOLvnIhlDhYSxuQQKe_1fNjCItduWQ4pfCP0hEGG1vNx4NxNCN21e1Rw82N1Vu2xPafj7rDeM5kvN6JbPeh1_TSE8ULvHKtFCl8q7gKAhIx24t7BpBtBenZfNVAs1rKns1XJEqf856VujD16M1rbkbhA8sps6vw1Kk8RUoDBxi-hGdNisUKZw3lOfGNg/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C4f2704fc4f23487cf9c408da45e7d678%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637899159291945719%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3D8XsWi5%252FNDgbyR%252FuGJuUsXl4T3fY7T3arbGqb5gw%252FG7w%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C4f2704fc4f23487cf9c408da45e7d678%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637899159291945719%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3D8XsWi5%252FNDgbyR%252FuGJuUsXl4T3fY7T3arbGqb5gw%252FG7w%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1H8GW2Bn7Ll57Nw2fly82lB7NVqWFThpdmba4nDLP3FxEXH...https://secure-web.cisco.com/1-HKtlf4QazjpH-LlBXFyBrSaNFWpEjDRd7ii2lhaFzMbplWHrO6hyiBBeeJB8s4rEfNZAaFFMF7kXOtPc7GvZWbso6DEVrsZirTtcikTCmbkiAuldmluoG5oDtBDUrc5xZxC3Q9vvkjSj51hlQ9TJhoddZfYBw0lDlIpdqG0UPQmu7OrtmdNJnI1uMFjL-MciXggaQCTa6TCog37VpQL2peqWosaVcbDuSix_6_d1WRTyKJ0RjAfUcnnb7fsUJxae0o6firs-5BSxclEMc9Krr9SPT0uVJe87Yopdf_Yr5FLy_MKA2Fsh5h7z-WrAFXiOWPRINSeTern3StDuLCg2g/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1H8GW2Bn7Ll57Nw2fly82lB7NVqWFThpdmba4nDLP3FxEXH...https://secure-web.cisco.com/1nh0JccwwU-fXnZcSbdlvFkAlXBpFK9VZ_EvIj2_PnnF0FJbizHGXyvTSx4p0K5hORFvfZQ5lS-ubnBMmMvCI56wsk2WyhWvHWPUW2IOsbUgHqgdss1tLozq6kxtbeWpc2RY24enMG0OEf3FwOikpy8DcfJZCwH_4JCGxEbQG9NEGZuOy3yiThhs4sDsB-9wbzXef1N4TZ8JWVEvg6mk7ad9rsLgYxIyUuEXKB8vWJIqDfbqpDJgQ3-c9U48JdeUVEk-cULIOnf-Zn7q_OAPlSKSTMb7kLRky71N53dJgnFl7iCEMgAVag1fZA84K0kzK3aEPH0dXEISNnseSFQRF3w/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul ________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 12:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1-HKtlf4QazjpH-LlBXFyBrSaNFWpEjDRd7ii2lhaFzMbplWHrO6hyiBBeeJB8s4rEfNZAaFFMF7kXOtPc7GvZWbso6DEVrsZirTtcikTCmbkiAuldmluoG5oDtBDUrc5xZxC3Q9vvkjSj51hlQ9TJhoddZfYBw0lDlIpdqG0UPQmu7OrtmdNJnI1uMFjL-MciXggaQCTa6TCog37VpQL2peqWosaVcbDuSix_6_d1WRTyKJ0RjAfUcnnb7fsUJxae0o6firs-5BSxclEMc9Krr9SPT0uVJe87Yopdf_Yr5FLy_MKA2Fsh5h7z-WrAFXiOWPRINSeTern3StDuLCg2g/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1nh0JccwwU-fXnZcSbdlvFkAlXBpFK9VZ_EvIj2_PnnF0FJbizHGXyvTSx4p0K5hORFvfZQ5lS-ubnBMmMvCI56wsk2WyhWvHWPUW2IOsbUgHqgdss1tLozq6kxtbeWpc2RY24enMG0OEf3FwOikpy8DcfJZCwH_4JCGxEbQG9NEGZuOy3yiThhs4sDsB-9wbzXef1N4TZ8JWVEvg6mk7ad9rsLgYxIyUuEXKB8vWJIqDfbqpDJgQ3-c9U48JdeUVEk-cULIOnf-Zn7q_OAPlSKSTMb7kLRky71N53dJgnFl7iCEMgAVag1fZA84K0kzK3aEPH0dXEISNnseSFQRF3w/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
Thanks Paul.
bedpostx_preproc.sh is the first job that is I submit to my cluster. My input file is attached to this email.
Thanks,
Ela
From: Wighton, Paulhttps://secure-web.cisco.com/1-MLOjaBGRyL4wOMDnTeccG5o-fhI_ylmXJOAkTGFrKN55BeQa_k4Zr55CvM-5jLnycJfpwGMP4e5UqzHSuy6qu0ngHc3VzyL3xsglKVZU7w9jCFVTsQzHG4eL6l60yqwFQL5PNk7xXYOfcMtWPguM_y3prvGAtu_RQcaG68UO9xd-q7fD7ln6n2hWvFlIQMeGMptWgPHJPfkL97jShJakiIrHkCciiSORoTqRUfT16xoWdtq4G_OI6V79iQRAk_SsmL2qg06Fz_i7S9-_mwso273MWVQZn1n_5hL5A_IG5Ou8SOau30IGzDByaWoU50uTR1eEwkZW2nKdjF29OjoVQ/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Dfrom%3A%2522Wighton%252C%2BPaul%2522 Sent: Wed, 08 Jun 2022 10:06:21 -0700https://secure-web.cisco.com/1_nqHAI5dk776UEDmSxUc7z3ozXABfYTpirVqTr8xfvPVD-vGwbLy4K7BNGRzZOJl8kj_ycXtDNEmihBKZnb5CYlOf0KrZ5_QfHDoZUXa-GWvI6vvclOruifME6-P1uJqzFdv8n47QHAzPvzKhu7lvNlcCtyk_WstC8nG02cSXEHhnuOSyC6Y0wnPHgDXRA5iWw5HGTY7wfiVy8EF0TnDGvBv_nfDdFZD5lv-hvohvvfLBsGCvQvnTL3YcZHRBY7HA07nV_h5BKJVEve5jdZ25eKHZCUTZ6QtE6g3isUcdEcInVAFT0fdx3EkfLcMRAS2z4zDitsTL8Wb-1FYolOdJw/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Ddate%3A20220608
Subject:Re: TRACULA -bedp parallel processing
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 12:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1jICIsbscj-K-FrzIe9MP1e3VlJfI_yxKSrRofZHrV2L4Cl...https://secure-web.cisco.com/1qoa2Xgjw0UbYtNT1iufuZtljUlx8BH0hlguZ04ZKleVHbPLWQDOj3HUZWJsGUc08EbaAIZETzg1JzlLNFLIo51GjAFZ8At0fKfafoxAQLMLWw2rpQKzKUHG425faPlvgJ0PxPKh0dZ2LnPeKT2gWwDvGMCrwyLnu2bb-3gvhgmAwfrjuOxhjH4FRxnXtjgX134uBwAbVeUfJM-YpEulNLdoTlxa-7k0OMWX6huIZP4vG0_mXoZQiDgONPF6DyfPIelb56IFUw_AaXoXJ5VXme7tHH4deVMxxbKGRTI2u45KYj5lLIsLnImyn1z_pAAsA-agW4DmGUA90sPSZBiakYQ/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1jICIsbscj-K-FrzIe9MP1e3VlJfI_yxKSrRofZHrV2L4Cl...https://secure-web.cisco.com/1UQ1-eRC8pPMIS1Hz13jH4OZIczAe4J1yff6HnG9qRBI21gF62iGnooyfRn01ItGhBuoUhs_f-ljYIDbR9ZP8JZfJHvx6oZCPIAEys75_4hGYUDmR2x3ZbB4rpvxBUkwZhcOZGOlHkwBqaXe0DxaxbKFzByupP7l1uMOrUALkvp9-Hwk7L0b_6JiKEz-alIBcPUfqfWUuLnLNMZi9vHzweKl4-B7ODQ5l5z1qrYt1MWdgWnORCr3AtNf4RsWFW5WUeDJ1rywacPfZ75ivuzMsCRDPjuVRRmFNBymL3XTlD94i7ltr-n5P09VWjtFdwHETQ3ON_qXDS-neerUAkx3w7Q/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
I tried to troubleshoot this, but I couldn't. I just want to follow-up and make sure no other solutions come to your mind. If not, we will run it out of cluster which seems to work just fine.
Thanks and all the best, Ela ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 3:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks Paul.
bedpostx_preproc.sh is the first job that is I submit to my cluster. My input file is attached to this email.
Thanks,
Ela
From: Wighton, Paulhttps://secure-web.cisco.com/1NvethJ2okwtBXYYV_EyxsXJx5qGOow2E5QtYJaKI1BCp3t8jVc7IQ9rNW9tMpRV7L5vP3mrgesq1jcnJKE5f_4HVbjGFYV70G8S40lFftxXK7lxGCnq56Ox0Mu9_i7WwuRnk-7bEGf-7R5dadwyxjWvyMgzXEvQhgfKan92fgm9AL_h7CtjK-_N5jWdO_f-cmdqFGeCspKHVODcFE4-d20kekjIkZbYia8GoJDcu27_d-Srd8pTig-g2Hi5NunkLAfpIgE4Io3ruNC0zu8Zvn6VCWVq-hy4a2HZ9ZIw7dodjmDWPVKcHcEqYkVV6sxGCB2gOlbzBrL_eeK8QHBJI7gRy-z5aUx-9dp2Io1Ww55g/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Dfrom%3A%2522Wighton%252C%2BPaul%2522 Sent: Wed, 08 Jun 2022 10:06:21 -0700https://secure-web.cisco.com/15JpdRjHpUwrjYyCfG0RWTJDoX413v5LN2Sed5H1OslWtt5j0CeOmss-WcOppPt0oWLSB3h1PXVS_mcB3lalLOjHb6zPQlqIN5vKRfohF0Mj1EXgkfg0eUNX1yvy0h7osnt39YaN4kHZT7dfh8b7lz5n4RFrfT49bz3uqGnbqEs88Gh1DFJfa0LBN6nlNlEKYcoVvEErAAZFcteu_-Fs__cSYKIWeATxNwCnCosyRS6peTz-ZPAzsA-i_ouJNkB4PDeJ7ZZm-i4vAbOAQGsBIU2404dp98DCAambIEO3SOkLgFcCG2V6DbgqudzsWc-oSIdI2OS0Pz_xJc4IFm7JoPovTTYe4NOyIusf9MhmMMAY/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Ddate%3A20220608
Subject:Re: TRACULA -bedp parallel processing
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 12:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1CwONV7ixc-guuS1SEjIePqXbylshJr5x9zwwjHlmjQM9W3...https://secure-web.cisco.com/19QJtf4BEOpos_0ZsSOSsD2nDFEWNEF-H3VLQWZtX8wUqoT9_E_O5G-MxvKRdedrQr9sKUt4ABy646VSpKUgy6983VBL_nOk2cBV3MzgBGV6nptdwxyTOuA5rqWOHETf4J9G3wePJIu0RY3MrBSsvFIwQV86AzzSwZYcuQTHDCsMWojHK933LwqHJHVusuZlrXuaZNLu5ufSzQJaOnOgqIkJZrONymjNvC6Gipgp4SmBmEjbVNWVo8vq3m_HCF54Y6TvSC_EdMUm7tyKOjbegBaOT4wfIUx-WtzHgz-3aPNWa_O1i1dS-i2Q-WGyKfVoJr8SsSs1aaw3li1UMlxfZfSO5dLDsh_C7nXI31_OgCqU/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1CwONV7ixc-guuS1SEjIePqXbylshJr5x9zwwjHlmjQM9W3...https://secure-web.cisco.com/1Nk0C4nRMMSY5p9BNN8sDwcVQVBVJPSEaoEZFTp1xTPSd_7BWnWOvqpFDGXaE1_8vacLLbsMsW72SpX-rMIMkHG_ZOh8XgDtXsMPfWmHstYCHROAgkHUGfShteTVZWZkkRH7K3_RCBv4fsLU9ySbcCj51a8HQlDurkQ0eeegeevacneTECH5-MC_9LjU56wa33XC8q4AKIex5FqyH7FKla4T6Qh-f_nvIAVan9S74FksdKFKP7BjQSX7h4JGp3i-O2m8y2uXn1w4dI5ADF0Mh0ONOV1nMdj97MguFKnDvXEyhPeO0tUhXMQFaBlB-NDeOGnQBQpkdzdkxA1Gj9LLrnbBi1kQDcNFesKJ3t7O6uGk/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
Unfortunately it's not something we've been able to replicate here ? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 14, 2022 8:53 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
I tried to troubleshoot this, but I couldn't. I just want to follow-up and make sure no other solutions come to your mind. If not, we will run it out of cluster which seems to work just fine.
Thanks and all the best, Ela ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 3:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks Paul.
bedpostx_preproc.sh is the first job that is I submit to my cluster. My input file is attached to this email.
Thanks,
Ela
From: Wighton, Paulhttps://secure-web.cisco.com/1Lgm2lQ15rBdqvHSmFbKJJbHz7zZcmAkqybWbQWBCzCY1qarjD2v6gZI4NVjt-Vu2BWT2qyYERkfC45tR1QW1rkSaCpdFjTIK2HeIzXdhL1NNlBpSiUSo91zkOwpaG7oiglqzfmSk9XFe8cFWyC3o92S25nwX6izLuMEWnMKmDUSWhyeY4agE-buXyYu9esH7qChFM--S96OZMYOdPYM6AqklV32CJcExo4dfp0VjO5bngYx6z1sAgnaQFEVlc2agmTqRA4Rvq2zTW336C0W2gtNm6MAVO2lkq6httV8NrevlpWxF0VpRgLCCgyiYmaZvFGx2qsCglmZh4X-Z22UJ1A/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Dfrom%3A%2522Wighton%252C%2BPaul%2522 Sent: Wed, 08 Jun 2022 10:06:21 -0700https://secure-web.cisco.com/1naUOvqMDxKyoWgJ9sQEL4pv-pvAOdvo59etyEySpJDwoT8qouab_cvfg-HoizIzZpxYVbbELgcVwulFcViDOfAyyHst9so2tdOrIP6_CsIBSgXQN14UjOlXowghc3YgXg8KMAiW0dMBqHRF-tG3Ls-wL9pb1aQ8jx73KOlp_JVnu78TzxZ39cgJ7Pqj9yxUYLxBp-pxtVxBCmnq4DvhCu0VCkjB69klGVz_n0mYdWvWWHDcCXNU9Y4fyS3DAi_mxxJx97xfEuB5mvFpQuW5FIBb9UZvCHMvLY4Tgmhwx2XYn_K72ZYYbYSHI97N2hvu3Ocp5jaX78Xcz9a3BDRszvQ/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Ddate%3A20220608
Subject:Re: TRACULA -bedp parallel processing
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 12:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1MMIME8g7wo2IgDRh4f9wI5HrH2wI8970dCuT8aISGU5I-d1NmRPiOPew0plfOdu2gCt8a3IrHYdjPQyYOvp-lDxNKuMP3ciXyYpl_iyxfcGhKAIbYYQUsCnxt_WUK34O6d1qF6MJ7BecmM9fvc5QOvhJY90Fmr_f7BlA84ABLUxdTKml5yktEKmnR5Nr_dcQtHsYzuLrgFJao9DSR7e0btkE_ZGSraJeCTMi0SCDUssAj5d0ZLYNh_KyITwQoqfTtaK07L4B5eBrwCcMsXSRXux2YWVJlTXJIM8BPxOOEX3OSfN4uztNgHDsLY5o3AytnjVa7zT_cP7o9iHVdFuM7w/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1aGyKjKHBRsjnRPGqhhpH83wBIumArqkclzDlOgukhHgDmux1B1av2zBBxKZ8NrolbhtVawO2j37-hX11iFeltmO4n_KX-Aupw9UfwRcOwTIzSggjP5ScPWDN37_JRP3_XU8DFhSxbOl96eyydbsCd0VLedLmDu6lr5Zf9h6VaHTv2RxMl-hxfXkaUOKlgqXyY4lpsf3ey27NJOmhVjO-22RqUGukCtlkNWS4oIoi4gjXRNJCzcJ5IhpzN32gJRM5qLfIv3CYDCRvSDAMEK1hcErwHv3KM6JizUJgarRxvqiCtZi0T_5BTck5zHi6Sj6OlgNzcDvuP9dX-2C_Wz-cLA/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
Hi all,
A nice gentleman, a doctor working as program analyst at James J. Peters Dept. of Veterans Administration Medical Center (OOMH) has kindly emailed me about this with a solution, whose name I won't mention for privacy reasons. I'm sharing it here in case it is helpful to others one day:
1) edit pre.Parallel.docx bedpostx_preproc.sh takes 2 arguments. Add a 2nd argument 0 at the end
bedpostx_preproc.sh /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri 0 bedpostx_preproc.sh /home2/am2359/fs6_data/TRACULA/subjects_dir/58/dmri 0
2) create nodif_brain_mask.nii.gz link in dmri cd /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri ln -s ../dlabel/diff/aparc+aseg_mask.bbr.nii.gz nodif_brain_mask.nii.gz
3) create dirs dmri.bedpostX/diff_slices dmri.bedpostX/logs/monitor dmri.bedpostX/xfms in /home2/am2359/fs6_data/TRACULA/subjects_dir/57
same for 58
This worked like a charm.
Thank you the anonymous helper and all,
Regards,
Ela
________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 14, 2022 9:23 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Unfortunately it's not something we've been able to replicate here 😕 ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 14, 2022 8:53 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
I tried to troubleshoot this, but I couldn't. I just want to follow-up and make sure no other solutions come to your mind. If not, we will run it out of cluster which seems to work just fine.
Thanks and all the best, Ela ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 3:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks Paul.
bedpostx_preproc.sh is the first job that is I submit to my cluster. My input file is attached to this email.
Thanks,
Ela
From: Wighton, Paulhttps://secure-web.cisco.com/1TfSWMyMhY030JCikbaj1E8qcMQl6W12Di12y__6m0LkxLCc3bkbnaYvvhnAqQJm8NakWDzKQ7NGoW0ShzFw2vjIv_vfAHi5ViyiA-veFRkQBaJ0dk8qKUqxrUQlKohY7lSkrKNnXIw9lavBbudxnBN8xKr332BYEjOue3Vc_st_9PUrp_aScD5NI1dkkAWFdmGprsibL6_d_ajfUdMX81YAH8zA042HfCcEwFlDfzIwlyLPQscmusWm3tIQd6tIgbLfZxH2w_U_PmZ12Ng7HOa9WywzmWaljqsZg6LNNa4cgF37ZsPdrT0Y_58HUCv46U9Q5np3um4xfNkDNFl6V5yNhKH3cMwcCAIa5SN1_-Zw/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1Lgm2lQ15rBdqvHSmFbKJJbHz7zZcmAkqybWbQWBCzCY1qarjD2v6gZI4NVjt-Vu2BWT2qyYERkfC45tR1QW1rkSaCpdFjTIK2HeIzXdhL1NNlBpSiUSo91zkOwpaG7oiglqzfmSk9XFe8cFWyC3o92S25nwX6izLuMEWnMKmDUSWhyeY4agE-buXyYu9esH7qChFM--S96OZMYOdPYM6AqklV32CJcExo4dfp0VjO5bngYx6z1sAgnaQFEVlc2agmTqRA4Rvq2zTW336C0W2gtNm6MAVO2lkq6httV8NrevlpWxF0VpRgLCCgyiYmaZvFGx2qsCglmZh4X-Z22UJ1A%252Fhttps%25253A%25252F%25252Fwww.mail-archive.com%25252Fsearch%25253Fl%25253Dfreesurfer%252540nmr.mgh.harvard.edu%252526q%25253Dfrom%25253A%25252522Wighton%2525252C%25252BPaul%25252522%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C2162f3c40c7a461f81b908da4e6db112%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637908530281578746%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRKIRzGxpktIrwzDb93E6jvRjkKCXVDXl8SPIu5m79TI%253D%26reserved%3D0 Sent: Wed, 08 Jun 2022 10:06:21 -0700https://secure-web.cisco.com/1t3ipB1kv30Wbm4akKyHkQnBAZBrS71X38e465zzBEVOVbUGvbLZrmn2K-mZ1ensAKp6A8IZIv5kTaKlbh6SP8FXfgOSrxEvqKceBTNu7x2ZaTkyApXJXJoVxsgOxYWuJv4O2fmkK_7dxGsD2aJb2OY0t8jEuIYCb3Dc8AXiLXfE08cY9QTikN1Z8OEHCq8X3IeVpqqM6uw7lFlAoeVjQvyX6GwLbdgjqUG-Ys62KZFi4xayc0P6euv-z2FEtrcA9eTi2gN-J17ZOa9Z9duS_ON5atHUMAOO25xn2qZmhlp0oaubFaDOi7dG8zPSFVBDSoeW_fUzrIFD5nuFBFjUag_hcdJMMN9h1_gkkkTHLZyI/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1naUOvqMDxKyoWgJ9sQEL4pv-pvAOdvo59etyEySpJDwoT8qouab_cvfg-HoizIzZpxYVbbELgcVwulFcViDOfAyyHst9so2tdOrIP6_CsIBSgXQN14UjOlXowghc3YgXg8KMAiW0dMBqHRF-tG3Ls-wL9pb1aQ8jx73KOlp_JVnu78TzxZ39cgJ7Pqj9yxUYLxBp-pxtVxBCmnq4DvhCu0VCkjB69klGVz_n0mYdWvWWHDcCXNU9Y4fyS3DAi_mxxJx97xfEuB5mvFpQuW5FIBb9UZvCHMvLY4Tgmhwx2XYn_K72ZYYbYSHI97N2hvu3Ocp5jaX78Xcz9a3BDRszvQ%252Fhttps%25253A%25252F%25252Fwww.mail-archive.com%25252Fsearch%25253Fl%25253Dfreesurfer%252540nmr.mgh.harvard.edu%252526q%25253Ddate%25253A20220608%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C2162f3c40c7a461f81b908da4e6db112%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637908530281578746%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DICvA%252BKgPSsAsiD3NyqIOWXJzC0RmyKVPMFDsAdWSDL8%253D%26reserved%3D0
Subject:Re: TRACULA -bedp parallel processing
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Wednesday, June 8, 2022 12:32 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Tuesday, June 7, 2022 2:46 PM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)? ________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Tuesday, June 7, 2022 11:46 AM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
External Email - Use Caution
Hi Anastasia,
FSL version is 6.0.5.1
Thanks,
Ela
To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
Subject: Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
________________________________ From: Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Monday, June 6, 2022 5:26 PM To: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela ________________________________ From: Yendiki, Anastasia AYENDIKI@mgh.harvard.edu Sent: Saturday, June 4, 2022 1:05 AM To: Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
________________________________ From: freesurfer-bounces@nmr.mgh.harvard.edu freesurfer-bounces@nmr.mgh.harvard.edu on behalf of Salar Dini, Elaheh elaheh.salardini@yale.edu Sent: Friday, June 3, 2022 7:44 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use Caution
Thank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory
cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up:
/home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
________________________________ From: Salar Dini, Elaheh Sent: Friday, June 3, 2022 3:09 PM To: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Subject: TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1oNTFDfurk5qx9XlvsE90yLBrNnt954nxZP5YB-KS0ODkh5...https://secure-web.cisco.com/1OoCOocHtWccCcDvAAjj32OWeusOC1ItF3MiESY17R9DPa6MTywqIaXe0Rh96AONERGp03iJn1eA4TbajDgOHb8E7HZSMzy2E1jH8TeGmEr4qVpmBVeJvAaHHUmex8yE7jU_27QV83GyAcOeQjFrdc7qKn6tREwWdKH_6JgADVnR9jEY98AeZIRRymAiLK9cGPGE-2sk0IIxsLv06eu8LofeDpgjw2gtv61Z2fBdKnOvnvtlqZBSc5B38t5MPh5omBwQYbMtf_gaXYnLZy5K2Y6HbXr6pMXrzJPxVQ80WpA8/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MMIME8g7wo2IgDRh4f9wI5HrH2wI8970dCuT8aISGU5I-d1NmRPiOPew0plfOdu2gCt8a3IrHYdjPQyYOvp-lDxNKuMP3ciXyYpl_iyxfcGhKAIbYYQUsCnxt_WUK34O6d1qF6MJ7BecmM9fvc5QOvhJY90Fmr_f7BlA84ABLUxdTKml5yktEKmnR5Nr_dcQtHsYzuLrgFJao9DSR7e0btkE_ZGSraJeCTMi0SCDUssAj5d0ZLYNh_KyITwQoqfTtaK07L4B5eBrwCcMsXSRXux2YWVJlTXJIM8BPxOOEX3OSfN4uztNgHDsLY5o3AytnjVa7zT_cP7o9iHVdFuM7w%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fsecure-web.cisco.com%2525252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%2525252Fhttps%252525253A%252525252F%252525252Fnam12.safelinks.protection.outlook.com%252525252F%252525253Furl%252525253Dhttps%25252525253A%25252525252F%25252525252Fwww.massgeneralbrigham.org%25252525252Fcomplianceline%2525252526data%252525253D05%25252525257C01%25252525257Celaheh.salardini%252525252540yale.edu%25252525257C4f2704fc4f23487cf9c408da45e7d678%25252525257Cdd8cbebb21394df8b4114e3e87abeb5c%25252525257C0%25252525257C0%25252525257C637899159291945719%25252525257CUnknown%25252525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%25252525253D%25252525257C3000%25252525257C%25252525257C%25252525257C%2525252526sdata%252525253D8XsWi5%25252525252FNDgbyR%25252525252FuGJuUsXl4T3fY7T3arbGqb5gw%25252525252FG7w%25252525253D%2525252526reserved%252525253D0%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C64067fcf096a4c4c5ada08da48b617fd%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637902244183106888%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C2162f3c40c7a461f81b908da4e6db112%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637908530281734985%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DykXDhMqGje64XD8jI%252FpETM3Xph8n3C63E%252BWRtC5a14s%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1oNTFDfurk5qx9XlvsE90yLBrNnt954nxZP5YB-KS0ODkh5...https://secure-web.cisco.com/1vMGE9hbB7SeLACCp4oYFhPrQzAhgxO2aGfkFBtYdovWBDBekX9SIPfRxU8IZF02iFhnWb8ld7fsJ4p7EpklqIxvQvKCv-qItmLsfqwBVXkCED0g-JWotudkJkhIOMxda3BJDDrVptcXGIRqDXZbZ4VbGmIsogqh-Ig-2opZYXTl-qRHrNkRncpcRdscryghZvK_4pPwOjmUAe3M5MphJuasCUjn1qVyq_vX2t1lAmbkHvkdhdyCZItbG18P9uNsbxDCnZjYtxeK_hZchG5kLNlDiHTcdeIdKWZWqrQ03XnvlhkfkaKdxUE3pPf3xYk7nTSA4vjQF8Y5IyetSFyfhCZkFqANep_E90f0Qnpck8iA/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1aGyKjKHBRsjnRPGqhhpH83wBIumArqkclzDlOgukhHgDmux1B1av2zBBxKZ8NrolbhtVawO2j37-hX11iFeltmO4n_KX-Aupw9UfwRcOwTIzSggjP5ScPWDN37_JRP3_XU8DFhSxbOl96eyydbsCd0VLedLmDu6lr5Zf9h6VaHTv2RxMl-hxfXkaUOKlgqXyY4lpsf3ey27NJOmhVjO-22RqUGukCtlkNWS4oIoi4gjXRNJCzcJ5IhpzN32gJRM5qLfIv3CYDCRvSDAMEK1hcErwHv3KM6JizUJgarRxvqiCtZi0T_5BTck5zHi6Sj6OlgNzcDvuP9dX-2C_Wz-cLA%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C64067fcf096a4c4c5ada08da48b617fd%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637902244183106888%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253DxM5jPyAPzFJ0d3%2525252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C2162f3c40c7a461f81b908da4e6db112%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637908530281734985%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DllQNsqIj03jhnW2Ik2xvDonfUqiavYO1EszChBuzSWo%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1oNTFDfurk5qx9XlvsE90yLBrNnt954nxZP5YB-KS0ODkh5...https://secure-web.cisco.com/1w0BahcOHjMwwMdIvYh1BK2Ib9HC7PC3HgwWi8TVolsjqZPqe4kgI9FiTzS2T9NTMffC7-4QDbLpPfBwleuEf5W-ft_38sLZWMkn9Ik1t95rvfmIjPp6gCQKqAch3nCAvjhfR8traCGliaTMYc210dpuQ_Wh0xYN1LRcIbbA32sbadh-HQfuc22hK89Gb7hh5bhFLZ9KcaKr1ZfyB_PU7vY4LK64X75620DD3dRu6hhuGMLUoRuhxy_dJ2yTioZyK2O5LTU385R4jUBMEijrHXgXoHSveu4vBURA25Um2cBrkvpaGOWpmZJVKHEJHy5ACAOllHQWC_rlIe4CO_Hs3_sVXPXquWjwVZEqg2ZgsqBg/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C2162f3c40c7a461f81b908da4e6db112%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637908530281734985%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DU8x1ZalXQd7d3oNCvBEfxiCClL7nFY70wLmS3qMihGg%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
External Email - Use Caution
Hi Ela,
I ran into the exact same problem. Here is how I fix it.
1) edit pre.Parallel.docx bedpostx_preproc.sh takes 2 arguments. Add a 2nd argument 0 at the end
bedpostx_preproc.sh /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri 0 bedpostx_preproc.sh /home2/am2359/fs6_data/TRACULA/subjects_dir/58/dmri 0
2) create nodif_brain_mask.nii.gz link in dmri cd /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri ln -s ../dlabel/diff/aparc+aseg_mask.bbr.nii.gz nodif_brain_mask.nii.gz
3) create dirs dmri.bedpostX/diff_slices dmri.bedpostX/logs/monitor dmri.bedpostX/xfms in /home2/am2359/fs6_data/TRACULA/subjects_dir/57
same for 58
now you can run Parallel.docx
Hope I didn't miss anything.
Best,
King-Wai Chu
On Tue, Jun 14, 2022 at 8:55 PM Salar Dini, Elaheh < elaheh.salardini@yale.edu> wrote:
External Email - Use CautionI tried to troubleshoot this, but I couldn't. I just want to follow-up and make sure no other solutions come to your mind. If not, we will run it out of cluster which seems to work just fine.
Thanks and all the best, Ela
*From:* Salar Dini, Elaheh elaheh.salardini@yale.edu *Sent:* Wednesday, June 8, 2022 3:32 PM *To:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
Thanks Paul.
bedpostx_preproc.sh is the first job that is I submit to my cluster. My input file is attached to this email.
Thanks,
Ela
*From: *Wighton, Paul https://secure-web.cisco.com/1NvethJ2okwtBXYYV_EyxsXJx5qGOow2E5QtYJaKI1BCp3t8jVc7IQ9rNW9tMpRV7L5vP3mrgesq1jcnJKE5f_4HVbjGFYV70G8S40lFftxXK7lxGCnq56Ox0Mu9_i7WwuRnk-7bEGf-7R5dadwyxjWvyMgzXEvQhgfKan92fgm9AL_h7CtjK-_N5jWdO_f-cmdqFGeCspKHVODcFE4-d20kekjIkZbYia8GoJDcu27_d-Srd8pTig-g2Hi5NunkLAfpIgE4Io3ruNC0zu8Zvn6VCWVq-hy4a2HZ9ZIw7dodjmDWPVKcHcEqYkVV6sxGCB2gOlbzBrL_eeK8QHBJI7gRy-z5aUx-9dp2Io1Ww55g/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Dfrom%3A%2522Wighton%252C%2BPaul%2522 *Sent: *Wed, 08 Jun 2022 10:06:21 -0700 https://secure-web.cisco.com/15JpdRjHpUwrjYyCfG0RWTJDoX413v5LN2Sed5H1OslWtt5j0CeOmss-WcOppPt0oWLSB3h1PXVS_mcB3lalLOjHb6zPQlqIN5vKRfohF0Mj1EXgkfg0eUNX1yvy0h7osnt39YaN4kHZT7dfh8b7lz5n4RFrfT49bz3uqGnbqEs88Gh1DFJfa0LBN6nlNlEKYcoVvEErAAZFcteu_-Fs__cSYKIWeATxNwCnCosyRS6peTz-ZPAzsA-i_ouJNkB4PDeJ7ZZm-i4vAbOAQGsBIU2404dp98DCAambIEO3SOkLgFcCG2V6DbgqudzsWc-oSIdI2OS0Pz_xJc4IFm7JoPovTTYe4NOyIusf9MhmMMAY/https%3A%2F%2Fwww.mail-archive.com%2Fsearch%3Fl%3Dfreesurfer%40nmr.mgh.harvard.edu%26q%3Ddate%3A20220608
*Subject:*Re: TRACULA -bedp parallel processing
Hi Ela,
It looks to me like line 79 of bedpostx_preproc.sh is:
if [ ${gflag} -eq 1 ]; then
Which is consistent with the error:
-eq: unary operator expected
What this error is saying is that the variable `gflag` has not been set, so the script does not know how to perform the comparison.
`gflag` is set a few lines above to the second parameter passed when executing `bedpostx_preproc.sh`. I would investigate how `bedpostx_preproc.sh` is being invoked to chase down the cause of this error.
-Paul
*From:* Salar Dini, Elaheh elaheh.salardini@yale.edu *Sent:* Wednesday, June 8, 2022 12:32 PM *To:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
That actually helps but not solving all the problems. So, what happens is that if I create dmri.bedpostX folder, the first error goes away, a.k.a. (cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory). However, the second error still exists: a.k.a. (home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected". line 79 is "${FSLDIR}/bin/fslslice ${subjdir}/nodif_brain_mask). And when I try to run Parallel lines, I still get the error (dmri.bedpostX/logs/log0000: No such file or directory). So, my guess is, it still doesn't copy the whole needed package for some reason. So, what we end up having in dmri.bedpostX folder is a)bvals, b)bvecs, c) nodif_brain_mask.nii.gz.
I can always manually copy all needed file from fsl folder to our subjects folder, but I am curious why we get the error and why everything is not copied automatically. I also don't know what files we need to manually copy.
Thanks again, Ela
*From:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu *Sent:* Tuesday, June 7, 2022 2:46 PM *To:* Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
What if you create the directory (that shows up in the "not a directory" error)?
*From:* Salar Dini, Elaheh elaheh.salardini@yale.edu *Sent:* Tuesday, June 7, 2022 11:46 AM *To:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
External Email - Use Caution*Hi Anastasia,*
*FSL version is 6.0.5.1*
*Thanks,*
*Ela *
*To:* Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu
*Subject:* Re: TRACULA -bedp parallel processin
Hi Ela - Which version of FSL are you using?
Anastasia.
*From:* Salar Dini, Elaheh elaheh.salardini@yale.edu *Sent:* Monday, June 6, 2022 5:26 PM *To:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
Thanks for the reply. I'm not sure if you are looking for this: we use a small cluster with 40 nodes, 512 CPU cores. We are still using PBS.
Thanks, Ela
*From:* Yendiki, Anastasia AYENDIKI@mgh.harvard.edu *Sent:* Saturday, June 4, 2022 1:05 AM *To:* Salar Dini, Elaheh elaheh.salardini@yale.edu; Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: TRACULA -bedp parallel processing
What kind of cluster are you running the parallel processes on?
*From:* freesurfer-bounces@nmr.mgh.harvard.edu < freesurfer-bounces@nmr.mgh.harvard.edu> on behalf of Salar Dini, Elaheh < elaheh.salardini@yale.edu> *Sent:* Friday, June 3, 2022 7:44 PM *To:* Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* Re: [Freesurfer] TRACULA -bedp parallel processing
External Email - Use CautionThank you for the guidance.
When it tries to run pre.ParallelI , I'll get this:
Copying files to bedpost directory cp: target '/home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX' is not a directory /home1/es2263/fsl//bin/bedpostx_preproc.sh: line 79: [: -eq: unary operator expected Done
This happens for all subjects.
After this, when I check the subject 57 folder in subjects_dir, there is a file named "dmri.bedpostX.nii.gz".
However, when it tries to run Parallel lines, these errors coming up: /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0000: No such file or directory /home1/es2263/fsl//bin/bedpostx_single_slice.sh: line 77: /home2/am2359/fs6_data/TRACULA/subjects_dir/57/dmri.bedpostX/logs/log0001: No such file or directory
...
If I run trac-all -bedp -c <Cinfig File> , and not try to process parallelly, it runs without any errors to the end.
Do you have any thoughts what goes wrong?
Thanks again,
Ela
You run them in that order. First the pre, then the parallel, then the post. You can queue them that way, so that the parallel jobs wait for the pre to finish, and the post waits for all the parallel jobs to finish.
*From:* Salar Dini, Elaheh *Sent:* Friday, June 3, 2022 3:09 PM *To:* Freesurfer support list freesurfer@nmr.mgh.harvard.edu *Subject:* TRACULA -bedp parallel processing
Hello,
I have tried to create a text file with command lines that can be run in parallel by using trac-all -bedp -c <config file> -jobs Parallel. When I use this command three text files are produced named: pre.Parallel, post.Parallel, and Parallel. Should I just use Parallel as a source for command lines? What are pre.Parallel and post.Parallel for? I am attaching my Configuration file, pre.Parallel, post.Parallel, and Parallel to this message as well.
OS: Centos 7, Freesurfer version: 7.2.0
Thank you for your help. Kindly, Ela
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at *MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be* https://secure-web.cisco.com/1hSlezONEdVUOGhP7kVcpFuoXNaqDktT3NF7AfqEbmT7qoF... https://secure-web.cisco.com/19QJtf4BEOpos_0ZsSOSsD2nDFEWNEF-H3VLQWZtX8wUqoT9_E_O5G-MxvKRdedrQr9sKUt4ABy646VSpKUgy6983VBL_nOk2cBV3MzgBGV6nptdwxyTOuA5rqWOHETf4J9G3wePJIu0RY3MrBSsvFIwQV86AzzSwZYcuQTHDCsMWojHK933LwqHJHVusuZlrXuaZNLu5ufSzQJaOnOgqIkJZrONymjNvC6Gipgp4SmBmEjbVNWVo8vq3m_HCF54Y6TvSC_EdMUm7tyKOjbegBaOT4wfIUx-WtzHgz-3aPNWa_O1i1dS-i2Q-WGyKfVoJr8SsSs1aaw3li1UMlxfZfSO5dLDsh_C7nXI31_OgCqU/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fsecure-web.cisco.com%252F1MD2Bmxi2wsdgYaSDic2uMxA1tDMkEAh-D8EJJvZDJHFQghQNVFbD7a7lZ5uU6p4cg0xU4oPxdB5gLLfolaUg3h3qnyrfchLnq-0S46f6ClMf4dskw11fMtisF-4uUIX3il1EXcSvZIB4DgWl_nSTr8KH81nkJOJl23T8DE7yZFvd2jI2dKAbxKgoCx_ElQW46ChHK393IYDY6z5navAAWcV69dfZ5_HqRZ_XllJwkVyhOXrMgI6nYnUq3ID1WGbaaUynRmsyDbVtTqsbT2N-9yuBKSMaYQ_42TFgSsgho_0d-LkOkxIQAKEmLtwROmjlCLJrZX9ApKuyHR44VhMZjg%252Fhttps%25253A%25252F%25252Fnam12.safelinks.protection.outlook.com%25252F%25253Furl%25253Dhttps%2525253A%2525252F%2525252Fwww.massgeneralbrigham.org%2525252Fcomplianceline%252526data%25253D05%2525257C01%2525257Celaheh.salardini%25252540yale.edu%2525257C4f2704fc4f23487cf9c408da45e7d678%2525257Cdd8cbebb21394df8b4114e3e87abeb5c%2525257C0%2525257C0%2525257C637899159291945719%2525257CUnknown%2525257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%2525253D%2525257C3000%2525257C%2525257C%2525257C%252526sdata%25253D8XsWi5%2525252FNDgbyR%2525252FuGJuUsXl4T3fY7T3arbGqb5gw%2525252FG7w%2525253D%252526reserved%25253D0%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DRgirQesOnoQQft1vJKgLpz8APPq7Af3tQsUppu3aA2U%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at *MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be* https://secure-web.cisco.com/1hSlezONEdVUOGhP7kVcpFuoXNaqDktT3NF7AfqEbmT7qoF... https://secure-web.cisco.com/1Nk0C4nRMMSY5p9BNN8sDwcVQVBVJPSEaoEZFTp1xTPSd_7BWnWOvqpFDGXaE1_8vacLLbsMsW72SpX-rMIMkHG_ZOh8XgDtXsMPfWmHstYCHROAgkHUGfShteTVZWZkkRH7K3_RCBv4fsLU9ySbcCj51a8HQlDurkQ0eeegeevacneTECH5-MC_9LjU56wa33XC8q4AKIex5FqyH7FKla4T6Qh-f_nvIAVan9S74FksdKFKP7BjQSX7h4JGp3i-O2m8y2uXn1w4dI5ADF0Mh0ONOV1nMdj97MguFKnDvXEyhPeO0tUhXMQFaBlB-NDeOGnQBQpkdzdkxA1Gj9LLrnbBi1kQDcNFesKJ3t7O6uGk/https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fwww.massgeneralbrigham.org%252Fcomplianceline%26data%3D05%257C01%257Celaheh.salardini%2540yale.edu%257C64067fcf096a4c4c5ada08da48b617fd%257Cdd8cbebb21394df8b4114e3e87abeb5c%257C0%257C0%257C637902244183106888%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C3000%257C%257C%257C%26sdata%3DxM5jPyAPzFJ0d3%252BsATXVYi94aAgrJMnWZ7Q58Xt2OKk%253D%26reserved%3D0 .
Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
freesurfer@nmr.mgh.harvard.edu
-
King-Wai Chu -
Salar Dini, Elaheh -
Wighton, Paul -
Yendiki, Anastasia