External Email - Use Caution

Hi Stefano,

thanks a lot for the reply.

Unfortunately, I've realized that even when you run the same command with a single-thread (see attached results 1 and 3) you get slighly different results,
which are also different against the multi-thread execution (result 2).

I've checked the code in Python for samseg (in /usr/local/freesurfer/7.2.0/python/packages/samseg) and I can see there are several calls to np.random,
without setting the pseudonumber generator (such as with np.random(1234)).

This of course might have implications on the reproducibility of the results, although I think most of the code is in a compiled library.

You can check yourself using the file I've shared with you on google drive.

I agree with you that, based on the implementation, running SAMSEG with a multi-thread execution (i.e. using --threads flag) causes a non-deterministic behaviour and the results will not be reproducible.

Of course, we are talking about very very small variations (<0.1%) among results, but it is something disappointing (at least when using default settings which imply a single-thread execution).

Thanks for letting me know about this option when compiling the source code.

Regards

Giulio

On Sat, 27 Aug 2022 at 17:38, Cerri, Stefano <SCERRI@mgh.harvard.edu> wrote:

Hi Giulio,

Although SAMSEG is a deterministic algorithm the default version might give you different results when executed with a different number of threads. These are small differences when looking at seg.mgz (a few voxels with swapped labels), but they are more visible when looking at volumes computed from soft segmentations like the stats files.

Note that you will obtain the same exact results when running SAMSEG with the same number of threads.

If you want to have cross-thread reproducible results with SAMSEG you can download the source files of FS 7.2 and build FS with the flag (GEMS_CROSS_THREAD_REPRODUCIBLE ON when doing cmake). Note that this version of SAMSEG will run slower than the original version.

Source files can be downloaded here: MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://github.com/freesurfer/freesurfer/tags?after=v7.3.1

and instructions on how to build FS are here: MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://surfer.nmr.mgh.harvard.edu/fswiki/BuildGuide

Hope it helps,

Stefano

Da: giulio siracusano <siracusanogiulio@gmail.com>
Inviato: sabato 27 agosto 2022 04:22
A: freesurfer@nmr.mgh.harvard.edu <freesurfer@nmr.mgh.harvard.edu>
Cc: Cerri, Stefano <SCERRI@mgh.harvard.edu>
Oggetto: Re: [Freesurfer] SAMSEG estimation error

External Email - Use Caution

Hi Stefano,

I've investigated the use of SAMSEG for AD/MCI patients. I'm using the 7.2 version of Freesurfer and I've realized that I get different results using the same data but when I run the SAMSEG multiple times.

Differently from using ASEG, SAMSEG provides also the chance to execute the code using multiple threads.

In the first run I've executed SAMSEG on ADNI patient id 002_S_0685 using the MP-RAGE series having ID I18211 (the link below provides the NIFTI version of the series which has been used for the processing).

MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://drive.google.com/drive/folders/15oghna17Qz-OSuo9E3-rldOt935AEEuO?usp=sharing

The first execution of SAMSEG was with default settings (1 thread) and results are attached, the second was using --threads 6.

These results from the SAME series are similar but they are NOT the same.

How can we explain that?

Regards

Giulio

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at MailScanner has detected a possible fraud attempt from "secure-web.cisco.com" claiming to be https://www.massgeneralbrigham.org/complianceline .