[Mne_analysis] IOError: 2GB file size limit reached. Support for larger raw files will be added in the future.
Denis A. Engemann
denis.engemann at gmail.com
Wed Mar 19 17:02:15 EDT 2014
+1e15 I had the same thought.
> On Mar 19, 2014, at 8:45 PM, Alexandre Gramfort <alexandre.gramfort at telecom-paristech.fr> wrote:
>
> I suggest the raw.save splits files automatically like the C code.
>
> Alex
>
>
> On Wed, Mar 19, 2014 at 8:41 PM, Denis-Alexander Engemann
> <denis.engemann at gmail.com> wrote:
>> Hi folks,
>>
>> it just occurs to me that we might want to think about adding a Raw.split
>> method.
>>
>> As a first quick and dirty solution, Lau, you might want think about a
>> function that splits the raw object and saves each split to fif files.
>>
>> Something like this:
>>
>> def split_save_raw(raw, splits , fname_pattern):
>> for i_split, (tmin, tmax) in enumerate(splits):
>> raw.copy().crop(tmin, tmax).save(fname_pattern % i_split)
>>
>>
>> split_save_raw(raw, [(0, 1), (1, 3), (3, 4)], 'my-split-%i-raw.fif')
>>
>> Re-construction should then be trivial.
>>
>> Cheers,
>> Denis
>>
>>
>>
>> On Wed, Mar 19, 2014 at 7:56 PM, Lau Møller Andersen <lau.andersen at cnru.dk>
>> wrote:
>>>
>>> Dear Eric and Alex
>>>
>>> Thanks for your answers. I now understand that it is not a simple issue.
>>> The files themselves are not greater than 2.0 GB, but because I have
>>> multiple recordings per subject, I read them in as a list. The only reason
>>> that I ask is that I would like to save the filtered file (i.e. the one that
>>> is greater than 2.0 GB, because it is a list of raw files).
>>>
>>> The way I work around it now, is simply to save the epoched files and
>>> working from there. The only annoying thing is, if/when I realize that I
>>> need to do something different for creating the epochs, then I need to do
>>> the time-consuming process of filtering again. But I can live with that.
>>>
>>> Thanks again for a wonderful python toolbox.
>>>
>>> Best
>>>
>>> Lau
>>>
>>>
>>> Den 19/03/2014 kl. 16.33 skrev Eric Larson <larson.eric.d at gmail.com>:
>>>
>>> It is challenging because the FIFF file format is not designed to handle
>>> files larger than that. Adding support for it would require redesigning and
>>> rewriting the I/O code fundamental to data analysis, and would likely break
>>> backward compatibility with many tools. AFAIK nobody is actively working on
>>> the issue.
>>>
>>> There are different ways of working around the issue. If you have a
>>> specific use case, you could open an issue on the `mne-python` Github site,
>>> and perhaps we can provide some ideas. Ideally we could document the
>>> workarounds somewhere in case people hit this issue in the future, too.
>>>
>>> Eric
>>>
>>>
>>>
>>> On Wed, Mar 19, 2014 at 1:52 AM, Lau Møller Andersen
>>> <lau.andersen at cnru.dk> wrote:
>>>>
>>>> Dear MNE developers
>>>>
>>>> Just out of curiosity, is this going to happen in the near future, or is
>>>> it a greater challenge than I imagine?
>>>>
>>>> Best
>>>>
>>>> Lau
>>>> _______________________________________________
>>>> Mne_analysis mailing list
>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>
>>>>
>>>> The information in this e-mail is intended only for the person to whom it
>>>> is
>>>> addressed. If you believe this e-mail was sent to you in error and the
>>>> e-mail
>>>> contains patient information, please contact the Partners Compliance
>>>> HelpLine at
>>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>>> error
>>>> but does not contain patient information, please contact the sender and
>>>> properly
>>>> dispose of the e-mail.
>>>
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom it
>>> is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>> error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>>
>>>
>>>
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom it
>>> is
>>> addressed. If you believe this e-mail was sent to you in error and the
>>> e-mail
>>> contains patient information, please contact the Partners Compliance
>>> HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>> error
>>> but does not contain patient information, please contact the sender and
>>> properly
>>> dispose of the e-mail.
>>
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> The information in this e-mail is intended only for the person to whom it is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>> error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
More information about the Mne_analysis
mailing list