External Email - Use Caution        

Dear FreeSurfer Developers,

I am trying to run the new “NextBrain" probabilistic atlas mri_histo_atlas_segment model, which internally calls mri_synthseg with additional parameters `--threads 8 --cpu --parc --robust` as a first step. 
My system: MacBook Pro 2021 with M1 pro chip, currently running macOS Monterey 12.6
I installed the most up-to-date developmental FS version: freesurfer-macOS-darwin_arm64-dev-20240211-4e530af

Currently, I am definitely getting stuck on the mri_synthseg step, so for now my issue is related to SynthSeg, not to the NextBrain package.

This is what I did so far, assuming I am in the SUBJECTS folder:

1. The example for mri_hist_atlas_segment,  `mri_histo_atlas_segment bert/mri/orig.mgz bert/mri/histo_atlas_segmentation/ 1 8` will return an error without displaying the error message.

2. I reproduced the error by running it’s analysis steps one by one: SynthSeg returns an error right after running it with any kind of configuration (e.g., `mri_synthseg --i bert/mri/orig.mgz --o 
mri/synthseg.mgz`): `module 'numpy' has no attribute ‘float128’. This is traceable to line 1595 of freesurfer/dev/python/scripts/mri_synthseg. The error generally seems to be a known issue on Apple chips. I fixed it with replacing `np.float128` with `np.longfloat`.

3. Now, SynthSeg starts to set itself up but quits with:
```
Node: 'model_1/unet_bn_down_0/FusedBatchNormV3'
Detected at node 'model_1/unet_bn_down_0/FusedBatchNormV3' defined at (most recent call last):
    File "/Applications/freesurfer/dev/python/scripts/mri_synthseg", line 2642, in <module>
      main()
    File "/Applications/freesurfer/dev/python/scripts/mri_synthseg", line 162, in main
      predict(
    File "/Applications/freesurfer/dev/python/scripts/mri_synthseg", line 323, in predict
      post_patch_segmentation = net.predict(image)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 2554, in predict
      tmp_batch_outputs = self.predict_function(iterator)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 2341, in predict_function
      return step_function(self, iterator)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 2327, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 2315, in run_step
      outputs = model.predict_step(data)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 2283, in predict_step
      return self(x, training=False)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/training.py", line 569, in __call__
      return super().__call__(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/base_layer.py", line 1150, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 96, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/functional.py", line 512, in call
      return self._run_internal_graph(inputs, training=training, mask=mask)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/functional.py", line 669, in _run_internal_graph
      outputs = node.layer(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/engine/base_layer.py", line 1150, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/traceback_utils.py", line 96, in error_handler
      return fn(*args, **kwargs)
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/layers/normalization/batch_normalization.py", line 597, in call
      outputs = self._fused_batch_norm(
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/layers/normalization/batch_normalization.py", line 990, in _fused_batch_norm
      output, mean, variance = control_flow_util.smart_cond(
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/utils/control_flow_util.py", line 108, in smart_cond
      return tf.__internal__.smart_cond.smart_cond(
    File "/Applications/freesurfer/dev/python/miniconda3_apple_dist/py3_8_13/lib/python3.8/site-packages/keras/src/layers/normalization/batch_normalization.py", line 979, in _fused_batch_norm_inference
      return tf.compat.v1.nn.fused_batch_norm(
Node: 'model_1/unet_bn_down_0/FusedBatchNormV3'
2 root error(s) found.
  (0) INVALID_ARGUMENT:  input must be 4-dimensional[1,256,256,256,24]
         [[{{node model_1/unet_bn_down_0/FusedBatchNormV3}}]]
         [[model_1/model/gaussian_blur/ExpandDims_27/_546]]
  (1) INVALID_ARGUMENT:  input must be 4-dimensional[1,256,256,256,24]
         [[{{node model_1/unet_bn_down_0/FusedBatchNormV3}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_predict_function_4927]

resuming program execution
```

4. After further messing around with the mri_synthseg script, I can get past that point by changing all lines that call the `unet` function from `net = unet(nb_features=24, input_shape=[None, None, None, 1], ...)` to `net = unet(nb_features=24, input_shape=[1, NoneNoneNone, 1], ...)` or similar (e.g. line 627, 645, 667, 684, …). However, then it crashes at a later point with other errors apparently related to data dimensionality.


I would really appreciate any suggestions! I know that running the analysis on a MacBook might not be the best choice anyways, but before I try to set it up on our institute HTC (which might be a whole other challenge), I though I try it that way. Also, I figured someone might be interested in the error report.

Thanks a lot!
Best wishes,
Leon

____________

Leon D. Lotter, MD

Clinician Scientist PhD candidate 
INM-7: Brain and Behaviour, Research Centre Juelich
Institute of Systems Neuroscience, University Hospital Duesseldorf
Max Planck School of Cognition, Leipzig