[Mne_analysis] ValueError: raw[1]._cals must match

[Warner, Graham Cary]

Shutil.copytree() uses shutil.copystat() to copy metadata, it's similar to `cp -p` in bash. Here is the documentation for shutil.copystat() https://docs.python.org/2/library/shutil.html#shutil.copystat

I don't do anything else for the metadata. The line that throws the error, `raw = Raw(f,preload=False,verbose=False)`, is the first line of my script involving mne except for `from mne.io import Raw` at the beginning. It would use mne for other things if it opened successfully but since it crashes trying to open it I don't think those things are relevant. Possibly of importance is that this run was automatically divided into 2 FIFF files because of it's size. However, the fact that I can open the original copy without incident makes me skeptical of how relevant that is. It turns out that the files do differ in someway that isn't detected by bash's diff command because they have different hashes. I'm looking into exactly how they differ now but am having trouble finding the proper tools to do so. It could be that the different file paths cause the hashses to differ, I'm not sure, I'm not very well versed in how hashes work. I'm using the md5 hash from python's hashlib module. I'm not entirely sure what other sections of my code would be useful for you.

Thank you,
Graham
________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu [mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Denis-Alexander Engemann [denis.engemann at gmail.com]
Sent: Friday, November 20, 2015 4:08 PM
To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] ValueError: raw[1]._cals must match

Hum this sounds mysterious.
Could you detail what you do when you copy meta data?
Could you maybe share some code that demonstrates what you do?

Cheers,
Denis

On Fri, Nov 20, 2015 at 5:48 PM, Warner, Graham Cary <GWARNER at mgh.harvard.edu<mailto:GWARNER at mgh.harvard.edu>> wrote:
Hi all,

I'm working on a python script that makes a copy of a directory of FIFF files using python's shutil.copytree() function and then removes some metadata from each of the files in that new copy using the MNE module. It works fine on the whole but there is one particular file which, for some reason, causes the script to break. Here is the code I use and resulting error message:

>>> f = '/space/jazz/1/users/gwarner/temp_dirs_for_fiff_registration/temp_meg_storage/pp010/009/pp010_arc_raw.fif'
>>> raw = Raw(f,preload=False,verbose=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2, in __init__
  File "/space/jazz/1/users/gwarner/anaconda/lib/python2.7/site-packages/mne/utils.py", line 538, in verbose
    return function(*args, **kwargs)
  File "/space/jazz/1/users/gwarner/anaconda/lib/python2.7/site-packages/mne/io/fiff/raw.py", line 114, in __init__
    _check_raw_compatibility(raws)
  File "/space/jazz/1/users/gwarner/anaconda/lib/python2.7/site-packages/mne/io/base.py", line 1995, in _check_raw_compatibility
    raise ValueError('raw[%d]._cals must match' % ri)
ValueError: raw[1]._cals must match

The strangest part is that I can open the original file without any problems:

>>> f = '/space/lilli/3/users/DARPA-TRANSFER/meg/pp010/009/pp010_arc_raw.fif'
>>> raw = Raw(f,preload=False,verbose=False)>>>

Stranger still is that the bash diff command tells me that both versions of the file are identical.

 [gwarner at jazz ~]$ diff /space/jazz/1/users/gwarner/temp_dirs_for_fiff_registration/temp_meg_storage/pp010/009/pp010_arc_raw.fif /space/lilli/3/users/DARPA-TRANSFER/meg/pp010/009/pp010_arc_raw.fif
[gwarner at jazz ~]$

The fact that it works fine for several hundred files before breaking here tells me that it isn't a problem with my implementation of the module, the fact that I can successfully open the original copy of the file tells me that it isn't just a bad/corrupted file, and the result of the diff test seems to indicate that it isn't a problem with the way it was copied either. I'm using the same version of MNE each time and have all the necessary permissions. Can anyone enlighten me as to what is going on here? Please let me know if there is any additional information I can provide to clear this up.

Thank you,
Graham

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20151120/2dee8c12/attachment-0001.html