[Mne_analysis] IOError: 2GB file size limit reached. Support for larger raw files will be added in the future.

Denis A. Engemann denis.engemann at gmail.com
Wed Mar 19 17:02:15 EDT 2014
Search archives:

+1e15 I had the same thought.

> On Mar 19, 2014, at 8:45 PM, Alexandre Gramfort <alexandre.gramfort at telecom-paristech.fr> wrote:
> 
> I suggest the raw.save splits files automatically like the C code.
> 
> Alex
> 
> 
> On Wed, Mar 19, 2014 at 8:41 PM, Denis-Alexander Engemann
> <denis.engemann at gmail.com> wrote:
>> Hi folks,
>> 
>> it just occurs to me that we might want to think about adding a Raw.split
>> method.
>> 
>> As a first quick and dirty solution, Lau, you might want think about a
>> function that splits the raw object and saves each split to fif files.
>> 
>> Something like this:
>> 
>> def split_save_raw(raw, splits , fname_pattern):
>>    for i_split, (tmin, tmax) in enumerate(splits):
>>        raw.copy().crop(tmin, tmax).save(fname_pattern % i_split)
>> 
>> 
>> split_save_raw(raw, [(0, 1), (1, 3), (3, 4)], 'my-split-%i-raw.fif')
>> 
>> Re-construction should then be trivial.
>> 
>> Cheers,
>> Denis
>> 
>> 
>> 
>> On Wed, Mar 19, 2014 at 7:56 PM, Lau Møller Andersen <lau.andersen at cnru.dk>
>> wrote:
>>> 
>>> Dear Eric and Alex
>>> 
>>> Thanks for your answers. I now understand that it is not a simple issue.
>>> The files themselves are not greater than 2.0 GB, but because I have
>>> multiple recordings per subject, I read them in as a list. The only reason
>>> that I ask is that I would like to save the filtered file (i.e. the one that
>>> is greater than 2.0 GB, because it is a list of raw files).
>>> 
>>> The way I work around it now, is simply to save the epoched files and
>>> working from there. The only annoying thing is, if/when I realize that I
>>> need to do something different for creating the epochs, then I need to do
>>> the time-consuming process of filtering again. But I can live with that.
>>> 
>>> Thanks again for a wonderful python toolbox.
>>> 
>>> Best
>>> 
>>> Lau
>>> 
>>> 
>>> Den 19/03/2014 kl. 16.33 skrev Eric Larson <larson.eric.d at gmail.com>:
>>> 
>>> It is challenging because the FIFF file format is not designed to handle
>>> files larger than that. Adding support for it would require redesigning and
>>> rewriting the I/O code fundamental to data analysis, and would likely break
>>> backward compatibility with many tools. AFAIK nobody is actively working on
>>> the issue.
>>> 
>>> There are different ways of working around the issue. If you have a
>>> specific use case, you could open an issue on the `mne-python` Github site,
>>> and perhaps we can provide some ideas. Ideally we could document the
>>> workarounds somewhere in case people hit this issue in the future, too.
>>> 
>>> Eric
>>> 
>>> 
>>> 
>>> On Wed, Mar 19, 2014 at 1:52 AM, Lau Møller Andersen
>>> <lau.andersen at cnru.dk> wrote:
>>>> 
>>>> Dear MNE developers
>>>> 
>>>> Just out of curiosity, is this going to happen in the near future, or is
>>>> it a greater challenge than I imagine?
>>>> 
>>>> Best
>>>> 
>>>> Lau
>>>> _______________________________________________
>>>> Mne_analysis mailing list
>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>> 
>>>> 
>>>> The information in this e-mail is intended only for the person to whom it
>>>> is
>>>> addressed. If you believe this e-mail was sent to you in error and the
>>>> e-mail
>>>> contains patient information, please contact the Partners Compliance
>>>> HelpLine at
>>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>>> error
>>>> but does not contain patient information, please contact the sender and
>>>> properly
>>>> dispose of the e-mail.
>>> 
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>> 
>>> 
>>> The information in this e-mail is intended only for the person to whom it
>>> is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>> error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>> 
>>> 
>>> The information in this e-mail is intended only for the person to whom it
>>> is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>> error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>> 
>> 
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>> 
>> 
>> The information in this e-mail is intended only for the person to whom it is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>> error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
> 
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis




More information about the Mne_analysis mailing list