Hi Giulio,
This is odd. I've run your scan on my computer two times using both a single thread and 8 threads with the default FS 7.2 and obtained the same results within the same number of threads.
Regarding the np.random functions, these are called only by the lesion version -- which, by the way, we have fixed in version 7.3 the seed to obtain reproducible results.
Are you sure you're running these commands on the same machine?
Stefano
________________________________ Da: giulio siracusano giuliosiracusano@gmail.com Inviato: sabato 27 agosto 2022 14:15 A: Cerri, Stefano SCERRI@mgh.harvard.edu Cc: giulio siracusano siracusanogiulio@gmail.com; freesurfer@nmr.mgh.harvard.edu freesurfer@nmr.mgh.harvard.edu Oggetto: Re: [Freesurfer] SAMSEG estimation error
External Email - Use Caution
Hi Stefano, thanks a lot for the reply. Unfortunately, I've realized that even when you run the same command with a single-thread (see attached results 1 and 3) you get slighly different results, which are also different against the multi-thread execution (result 2). I've checked the code in Python for samseg (in /usr/local/freesurfer/7.2.0/python/packages/samseg) and I can see there are several calls to np.random, without setting the pseudonumber generator (such as with np.random(1234)). This of course might have implications on the reproducibility of the results, although I think most of the code is in a compiled library. You can check yourself using the file I've shared with you on google drive. I agree with you that, based on the implementation, running SAMSEG with a multi-thread execution (i.e. using --threads flag) causes a non-deterministic behaviour and the results will not be reproducible. Of course, we are talking about very very small variations (<0.1%) among results, but it is something disappointing (at least when using default settings which imply a single-thread execution). Thanks for letting me know about this option when compiling the source code.
Regards
Giulio
On Sat, 27 Aug 2022 at 17:38, Cerri, Stefano <SCERRI@mgh.harvard.edumailto:SCERRI@mgh.harvard.edu> wrote: Hi Giulio,
Although SAMSEG is a deterministic algorithm the default version might give you different results when executed with a different number of threads. These are small differences when looking at seg.mgz (a few voxels with swapped labels), but they are more visible when looking at volumes computed from soft segmentations like the stats files.
Note that you will obtain the same exact results when running SAMSEG with the same number of threads.
If you want to have cross-thread reproducible results with SAMSEG you can download the source files of FS 7.2 and build FS with the flag (GEMS_CROSS_THREAD_REPRODUCIBLE ON when doing cmake). Note that this version of SAMSEG will run slower than the original version.
Source files can be downloaded here: https://github.com/freesurfer/freesurfer/tags?after=v7.3.1https://secure-web.cisco.com/1f6_OrrMtkirHO7dyqPm2qDT9b28pekhPVqQDcrIico18JMZJFze8NUMfEmUyZL_ET3c5k6hPg_ul_SYvt8iHOnbZGCiik3UoA9NCuBbQWCQ5eeJiNr7EcPuCHlQKNIoctCSq_92TxkdVOjBg2kGAnVK_1gEyHrJ_dAQ6rkNvKJac1gAZcq9xLpq42f2h2DP0yhhiE-pWBcUkQ-inW1ONYfFXcIs_PWVI4Am1yxVbV5O_cOh3b5XTaonmrlzgV52sWROlqI1brHtmhIEK5Y_KVdZp9Cf9YiN6jbG5o-8l4mFiySHc8unvHScbT5g6Kmjr/https%3A%2F%2Fgithub.com%2Ffreesurfer%2Ffreesurfer%2Ftags%3Fafter%3Dv7.3.1
and instructions on how to build FS are here: https://surfer.nmr.mgh.harvard.edu/fswiki/BuildGuidehttps://secure-web.cisco.com/1CaXd2_aFCyTRvuzpyKfKWpi8AQAZcUmyziYiFtuz41i6XVpkRWrd_FAre0ODDRkOUV5uzxaxka461mo5ZANOTULK21YPIagBLfcPWuRiWCTk-31wWC1V7PA8l_UT7TWEAaaeKX5QyiWMmqBEjXo923vWrxgAetrsjJ-ckAcy9_Vz98qo5j3-Ecf4lQlGjTCy7pL7adLkSE-65OwzZJx0QUu74gGhZsm_giG9eYGitfDU8vXaLKzWgdPCRuxJ-Xvpegne6rGL1L9uvNBuG3JD1Q7nLwvAMuJJ_vfz4N-Ea9rLnSVAQUvosNavo-YeUJ3d/https%3A%2F%2Fsurfer.nmr.mgh.harvard.edu%2Ffswiki%2FBuildGuide
Hope it helps, Stefano
________________________________ Da: giulio siracusano <siracusanogiulio@gmail.commailto:siracusanogiulio@gmail.com> Inviato: sabato 27 agosto 2022 04:22 A: freesurfer@nmr.mgh.harvard.edumailto:freesurfer@nmr.mgh.harvard.edu <freesurfer@nmr.mgh.harvard.edumailto:freesurfer@nmr.mgh.harvard.edu> Cc: Cerri, Stefano <SCERRI@mgh.harvard.edumailto:SCERRI@mgh.harvard.edu> Oggetto: Re: [Freesurfer] SAMSEG estimation error
External Email - Use Caution
Hi Stefano,
I've investigated the use of SAMSEG for AD/MCI patients. I'm using the 7.2 version of Freesurfer and I've realized that I get different results using the same data but when I run the SAMSEG multiple times.
Differently from using ASEG, SAMSEG provides also the chance to execute the code using multiple threads.
In the first run I've executed SAMSEG on ADNI patient id 002_S_0685 using the MP-RAGE series having ID I18211 (the link below provides the NIFTI version of the series which has been used for the processing).
https://drive.google.com/drive/folders/15oghna17Qz-OSuo9E3-rldOt935AEEuO?usp...https://secure-web.cisco.com/1kjcQ6V-ZSEdfK2uygax7NBB93ggYjLmfqbPpwSpeHhQevXMkepVFRYeckjpgVHV1Oh-uMLgjGFKI2PfVRNiGQTK5Dy4PIAShzp5i5bimEXKxeKoS713q15oMAwcQhSKATzEmPHGzN8asmgshA5ht7lmRmBLOjQ3Xym0hFW2JhMPHCuQwvRnE6QRghkwv-ESvMN0G3rFtnEvS5kCBMxXH9ucX5ChFJLkS9Q0fI5AKpZQ_LD0_QMiVFn5qsKGbb9aLlcFY9Qrd7s7tAZbnw_5hrpktiA4_6pudckEKwVW97JppQE6e78a0Z0Ay4m-uGLOx/https%3A%2F%2Fdrive.google.com%2Fdrive%2Ffolders%2F15oghna17Qz-OSuo9E3-rldOt935AEEuO%3Fusp%3Dsharing
The first execution of SAMSEG was with default settings (1 thread) and results are attached, the second was using --threads 6.
These results from the SAME series are similar but they are NOT the same.
How can we explain that?
Regards
Giulio
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://www.massgeneralbrigham.org/compliancelinehttps://secure-web.cisco.com/1F9iopsAcv81Y5jnzKuZoZPRhV3AbjEr6rKYyE8cQbZEG5FThfwmc4dYXy9KDFGLrsIVZVlqSY6shJ2Zb6FB8KpyyUq2y3tD_LYuGahEffbc-9TZ5qHDHDbU9yFqNUXQDw5GkdR2AV0axWYh2JoLXmWNIUFQzK8qo6ktjIEnSbxcULqwhVLQV6XAjDfrm76y2l3fkiHzlxQ_X43ZKoaovYNdBo2P-nO0_WoL23n-arwP7hh6lJvYIrz49dqmlcsZmO-ov8EycopgFUvcxlvqbUTYGoYZgmZv1sHgCjjoRKAxe0u9anN0OriAfeItHHUdo/https%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline .