[Mne_analysis] IOError: 2GB file size limit reached. Support for larger raw files will be added in the future.

[Alexandre Gramfort]

I suggest the raw.save splits files automatically like the C code.

Alex


On Wed, Mar 19, 2014 at 8:41 PM, Denis-Alexander Engemann
<denis.engemann at gmail.com> wrote:
> Hi folks,
>
> it just occurs to me that we might want to think about adding a Raw.split
> method.
>
> As a first quick and dirty solution, Lau, you might want think about a
> function that splits the raw object and saves each split to fif files.
>
> Something like this:
>
> def split_save_raw(raw, splits , fname_pattern):
>     for i_split, (tmin, tmax) in enumerate(splits):
>         raw.copy().crop(tmin, tmax).save(fname_pattern % i_split)
>
>
> split_save_raw(raw, [(0, 1), (1, 3), (3, 4)], 'my-split-%i-raw.fif')
>
> Re-construction should then be trivial.
>
> Cheers,
> Denis
>
>
>
> On Wed, Mar 19, 2014 at 7:56 PM, Lau Møller Andersen <lau.andersen at cnru.dk>
> wrote:
>>
>> Dear Eric and Alex
>>
>> Thanks for your answers. I now understand that it is not a simple issue.
>> The files themselves are not greater than 2.0 GB, but because I have
>> multiple recordings per subject, I read them in as a list. The only reason
>> that I ask is that I would like to save the filtered file (i.e. the one that
>> is greater than 2.0 GB, because it is a list of raw files).
>>
>> The way I work around it now, is simply to save the epoched files and
>> working from there. The only annoying thing is, if/when I realize that I
>> need to do something different for creating the epochs, then I need to do
>> the time-consuming process of filtering again. But I can live with that.
>>
>> Thanks again for a wonderful python toolbox.
>>
>> Best
>>
>> Lau
>>
>>
>> Den 19/03/2014 kl. 16.33 skrev Eric Larson <larson.eric.d at gmail.com>:
>>
>> It is challenging because the FIFF file format is not designed to handle
>> files larger than that. Adding support for it would require redesigning and
>> rewriting the I/O code fundamental to data analysis, and would likely break
>> backward compatibility with many tools. AFAIK nobody is actively working on
>> the issue.
>>
>> There are different ways of working around the issue. If you have a
>> specific use case, you could open an issue on the `mne-python` Github site,
>> and perhaps we can provide some ideas. Ideally we could document the
>> workarounds somewhere in case people hit this issue in the future, too.
>>
>> Eric
>>
>>
>>
>> On Wed, Mar 19, 2014 at 1:52 AM, Lau Møller Andersen
>> <lau.andersen at cnru.dk> wrote:
>>>
>>> Dear MNE developers
>>>
>>> Just out of curiosity, is this going to happen in the near future, or is
>>> it a greater challenge than I imagine?
>>>
>>> Best
>>>
>>> Lau
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom it
>>> is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>> error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>>
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>> error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>>
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>> error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>