[Mne_analysis] IOError: 2GB file size limit reached. Support for larger raw files will be added in the future.

[Denis-Alexander Engemann]

Hi folks,

it just occurs to me that we might want to think about adding a Raw.split
method.

As a first quick and dirty solution, Lau, you might want think about a
function that splits the raw object and saves each split to fif files.

Something like this:

def split_save_raw(raw, splits , fname_pattern):
    for i_split, (tmin, tmax) in enumerate(splits):
        raw.copy().crop(tmin, tmax).save(fname_pattern % i_split)


split_save_raw(raw, [(0, 1), (1, 3), (3, 4)], 'my-split-%i-raw.fif')

Re-construction should then be trivial.

Cheers,
Denis



On Wed, Mar 19, 2014 at 7:56 PM, Lau Møller Andersen
<lau.andersen at cnru.dk>wrote:

> Dear Eric and Alex
>
> Thanks for your answers. I now understand that it is not a simple issue.
> The files themselves are not greater than 2.0 GB, but because I have
> multiple recordings per subject, I read them in as a list. The only reason
> that I ask is that I would like to save the filtered file (i.e. the one
> that is greater than 2.0 GB, because it is a list of raw files).
>
> The way I work around it now, is simply to save the epoched files and
> working from there. The only annoying thing is, if/when I realize that I
> need to do something different for creating the epochs, then I need to do
> the time-consuming process of filtering again. But I can live with that.
>
> Thanks again for a wonderful python toolbox.
>
> Best
>
> Lau
>
>
> Den 19/03/2014 kl. 16.33 skrev Eric Larson <larson.eric.d at gmail.com>:
>
> It is challenging because the FIFF file format is not designed to handle
> files larger than that. Adding support for it would require redesigning and
> rewriting the I/O code fundamental to data analysis, and would likely break
> backward compatibility with many tools. AFAIK nobody is actively working on
> the issue.
>
> There are different ways of working around the issue. If you have a
> specific use case, you could open an issue on the `mne-python` Github site,
> and perhaps we can provide some ideas. Ideally we could document the
> workarounds somewhere in case people hit this issue in the future, too.
>
> Eric
>
>
>
> On Wed, Mar 19, 2014 at 1:52 AM, Lau Møller Andersen <lau.andersen at cnru.dk
> > wrote:
>
>> Dear MNE developers
>>
>> Just out of curiosity, is this going to happen in the near future, or is
>> it a greater challenge than I imagine?
>>
>> Best
>>
>> Lau
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you
>> in error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20140319/ec9c6383/attachment.html