[Mne_analysis] "Value too large to be stored in data type" when running mne_mark_bad_channels in tcsh

Matti Hamalainen msh at nmr.mgh.harvard.edu
Mon Nov 25 04:33:31 EST 2013
Search archives:

Hi Steve,

Just FYI, MNE-C does not do the splitting in mne_kit2fiff either.

- Matti


On Nov 25, 2013, at 3:39 AM, Alexandre Gramfort <alexandre.gramfort at telecom-paristech.fr> wrote:

> hi Steve,
> 
> looks like you've hit a problem in our python fiff writing code that does not
> splits .fif files to avoid large files.
> 
> We'll look into it asap
> 
> Thanks for the report.
> 
> Best,
> Alex
> 
> 
> On Mon, Nov 25, 2013 at 9:34 AM, Stephen Politzer-Ahles <spa268 at nyu.edu> wrote:
>> Hi Alexandre,
>> 
>> It started as Yokogawa .con data, I wrote it to fif using mne.gui.kit2fiff
>> in the mne-python tools.
>> 
>> Best,
>> Steve
>> 
>> 
>> 
>> Stephen Politzer-Ahles
>> New York University, Abu Dhabi
>> Neuroscience of Language Lab
>> http://www.nyu.edu/projects/politzer-ahles/
>> 
>> 
>> On Mon, Nov 25, 2013 at 12:32 PM, Alexandre Gramfort
>> <alexandre.gramfort at telecom-paristech.fr> wrote:
>>> 
>>> Steve,
>>> 
>>> how did you write this file?
>>> 
>>> standard .fif file should not exceed 2GB.
>>> 
>>> Alex
>>> 
>>> 
>>> On Mon, Nov 25, 2013 at 7:34 AM, Stephen Politzer-Ahles <spa268 at nyu.edu>
>>> wrote:
>>>> Hi Eric,
>>>> 
>>>> Thanks, I will keep trying various things out in the meantime. I was
>>>> also
>>>> concerned about file size limits, but it seems weird that other
>>>> functions
>>>> like mne_process_raw still work on my big files.
>>>> 
>>>> If I don't find a solution, is it possible instead to just run
>>>> mne_mark_bad_channels on the averaged data (and make sure to use the
>>>> averaged, rather than the raw, as the --meas file in later steps of the
>>>> pipeline)?
>>>> 
>>>> Best,
>>>> Steve
>>>> 
>>>> 
>>>> 
>>>> Stephen Politzer-Ahles
>>>> New York University, Abu Dhabi
>>>> Neuroscience of Language Lab
>>>> http://www.nyu.edu/projects/politzer-ahles/
>>>> 
>>>> 
>>>> On Mon, Nov 25, 2013 at 5:20 AM, Eric Larson <larson.eric.d at gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hey Steve,
>>>>> 
>>>>> It is possible that this has to do with file locations being stored
>>>>> using
>>>>> 32-bit integers. I thought FIFF had an effective file-size limit of 2GB
>>>>> for
>>>>> that reason (using 32-bit signed integers would make this the limit),
>>>>> but
>>>>> I'm not 100% sure. If nobody beats me to a more definitive answer and
>>>>> you
>>>>> can't find anything about this issue yourself by tomorrow or Tuesday, I
>>>>> can
>>>>> look into it more then.
>>>>> 
>>>>> Cheers,
>>>>> Eric
>>>>> 
>>>>> 
>>>>> On Sun, Nov 24, 2013 at 12:33 PM, Stephen Politzer-Ahles
>>>>> <spa268 at nyu.edu>
>>>>> wrote:
>>>>>> 
>>>>>> To whom it may concern:
>>>>>> 
>>>>>> When I try to run mne_mark_bad_channels in tcsh on a raw .fif file
>>>>>> (about
>>>>>> 3.25 GB), I get an error like the following:
>>>>>> 
>>>>>> "/Volumes/data/....../A0007.fif ... [failed] fseek : Value too large
>>>>>> to
>>>>>> be stored in data type"
>>>>>> 
>>>>>> As far as I can tell this seems to be some kind of Unix error, not
>>>>>> something specific to MNE...but this happens regardless of which drive
>>>>>> I
>>>>>> happen to be trying to modify data on, and also other functions that
>>>>>> also
>>>>>> need to open the data (such as mne_process_raw) work fine, so I don't
>>>>>> think
>>>>>> it's necessarily a permission or filesystem issue. Does anyone have
>>>>>> any idea
>>>>>> what might be causing this?
>>>>>> 
>>>>>> Thanks for your input,
>>>>>> Steve
>>>>>> 
>>>>>> 
>>>>>> Stephen Politzer-Ahles
>>>>>> New York University, Abu Dhabi
>>>>>> Neuroscience of Language Lab
>>>>>> http://www.nyu.edu/projects/politzer-ahles/
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Mne_analysis mailing list
>>>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>>> 
>>>>>> 
>>>>>> The information in this e-mail is intended only for the person to whom
>>>>>> it
>>>>>> is
>>>>>> addressed. If you believe this e-mail was sent to you in error and the
>>>>>> e-mail
>>>>>> contains patient information, please contact the Partners Compliance
>>>>>> HelpLine at
>>>>>> http://www.partners.org/complianceline . If the e-mail was sent to you
>>>>>> in
>>>>>> error
>>>>>> but does not contain patient information, please contact the sender
>>>>>> and
>>>>>> properly
>>>>>> dispose of the e-mail.
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Mne_analysis mailing list
>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>> 
>>>> 
>>>> The information in this e-mail is intended only for the person to whom
>>>> it is
>>>> addressed. If you believe this e-mail was sent to you in error and the
>>>> e-mail
>>>> contains patient information, please contact the Partners Compliance
>>>> HelpLine at
>>>> http://www.partners.org/complianceline . If the e-mail was sent to you
>>>> in
>>>> error
>>>> but does not contain patient information, please contact the sender and
>>>> properly
>>>> dispose of the e-mail.
>>>> 
>> 
>> 
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> 
> 





More information about the Mne_analysis mailing list