[Mne_analysis] continuation (from Fieldtrip list) of writing fifs

[Eric Larson]

Hey MNE-ers,

We have now implemented raw saving for double and short formats in
mne-python. However, take note:

1) 'single' (32-bit floating point) is the standard format for raw files in
MNE C code and mne-python. This is the safest choice.
2) 'double' (64-bit floating point) format, while higher precision, will
not be read by mne command-line tools (MNE C code).
3) 'short' (16-bit) format, while it requires half the space of 'single',
results in lost precision. This option is added mostly for users who read
in a raw file already in short format, modify it in some minor way (e.g.,
correcting a digitization position), and then write the file out. It is not
recommended to use 'short' format to save data that has been processed
(e.g. filtered).

Cheers,
Eric





On Wed, Jan 30, 2013 at 9:31 PM, Matti Hamalainen
<msh at nmr.mgh.harvard.edu>wrote:

>
> Hey!
>
> The reason why, e.g., mne_browse_raw (and Matlab writing routines) write
> raw files as 32-bit floats is that there might have been some intervening
> operations and precision might be lost if the data are written back as
> 16-bit integers.
> MNE-Matlab does not write 64-bit (double precision floats) and I do not
> think MNE-Python does either. This data type is not supported by the fif
> raw data reading routines in MNE C code.
>
> - Matti
>
> On Jan 30, 2013, at 9:50 PM, Eric Larson wrote:
>
> Hey Andy,
>
> Yes, it should be possible to implement a precision argument in
> mne-python. I should be able to get to it this week.
>
> Cheers,
> Eric
>
>
>
> On Wed, Jan 30, 2013 at 6:38 AM, Dykstra, Andrew <
> Andrew.Dykstra at med.uni-heidelberg.de> wrote:
>
>> Hi Alexandre, Peter,
>>
>> While it's not nearly as problematic as going from 2 to 18 Gb, I also
>> experience file size increases (usually exact doubling minus the last
>> small buffer) when rewriting fif files, not only in MATLAB but also with
>> the Python tools.  In my case, I've always wondered whether it's just a
>> difference in the default writing precisions between the modern MNE
>> package and our system (Neuromag 122), in which case I'm content with
>> storing the data in the higher precision.  In any case, would it be
>> possible to include a precision argument in the Python writing tool?
>>
>> Cheers,
>> Andy
>>
>> --
>> Andrew R. Dykstra, PhD
>> Auditory Cognition Lab
>> Neurologie und Poliklinik
>> Universitätsklinikum Heidelberg
>> Im Neuenheimer Feld 400
>> 69120 Heidelberg
>>
>> "How small the cosmos.  How paltry and puny compared to human
>> consciousness . . . to a single individual recollection." - Vladimir Nabokov
>>
>>
>>
>> "hi Peter,
>>
>> > The size of the input file is ~2gb but when
>> > writing, the output is approx ~18gb. I suspect I'm writing a WHOLE
>> bunch of
>> > redundant data, but am unable to open the new file, even on a machine
>> > running 16gb RAM.
>>
>> what is likely to happen is that the matlab code writes data as double
>> precision
>> while the original neuromag data were in float 16. Note that a single fif
>> file
>> cannot exceed 2Go due to internal pointer arithmetic otherwise it will
>> be broken.
>> What you can do is hack the fif writing to make sure it's written back
>> in float16.
>>
>> Best,
>> Alex
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> The information in this e-mail is intended only for the person to whom it
>> is
>> addressed. If you believe this e-mail was sent to you in error and the
>> e-mail
>> contains patient information, please contact the Partners Compliance
>> HelpLine at
>> http://www.partners.org/complianceline . If the e-mail was sent to you
>> in error
>> but does not contain patient information, please contact the sender and
>> properly
>> dispose of the e-mail.
>>
>>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
>
>
>    ---------
>
> Matti Hamalainen, Ph.D.
> Athinoula A. Martinos Center for Biomedical Imaging
> Massachusetts General Hospital
>
> msh at nmr.mgh.harvard.edu
> mhamalainen at partners.org
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20130131/dfb455d9/attachment.html