[Mne_analysis] combining/splitting fif files

[Matti Hamalainen]

Hi Eliezer,

On Feb 15, 2009, at 1:07 PM, Eliezer Kanal wrote:

> Hello folks -
>
> I have an experimental paradigm which lasts approximately 40  
> minutes. I usually just let the MEG run straight, end up with a  
> single fif file, and then do all my analysis on this. We recently  
> switched from 16 bit to a 32 bit architecture (side point: does this  
> mean more accurate recordings? what DOES that mean for the end  
> user?), and I learned something new: the machine will only save  
> files less than 2 GB in size. As such, I now have two files,  
> myFile.fif (2 GB) and myFile_1.fif (~1.6 GB). I have no idea where  
> it split it, behavioral-wise.
>
> What's the best way to proceed from here? I can't analyze them as  
> they sit, since I can't specify multiple files in mne_process_raw if  
> the files contain different numbers of trials. Is there a program  
> that can concatenate them? Should I just try to use mne_read_raw in  
> matlab and hope matlab doesn't choke on almost 4 GB of data? Any  
> ideas? Thanks -

Lauri's and Dan's emails pretty much answer everything. Regarding  
Matlab, 16 vs 32 bits do not really make a difference there because  
the data are usually converted into doubles anyway and are actually 8  
bytes per reading. What this practically means is that for large data  
sets the brute force method of reading everything in at once and then  
processing it is not feasible but you need to process the data in  
smaller pieces. In fact, mne_browse_raw/mne_process_raw do exactly  
this by using a ring buffer data structure.

Since the split raw data files are really continuous but the data is  
just located in different files, I have dreamed of changes to  
mne_browse_raw/mne_process_raw to read the the files automatically as  
needed without the user knowing about the splitting. There are already  
tags in the fif files which facilitate this. However, this is  
something which can be only dreamed of at a the moment ;-) For now,  
Lauri's and Dan's suggestions using the --gave and --gcov options are  
the most feasible solution if you are happy with the standard  
processing.


- Matti




-------------

Matti Hamalainen, Ph.D.
Athinoula A. Martinos Center for Biomedical Imaging
Massachusetts General Hospital
Building 149, 13th Street, Mailcode 149-2301
Charlestown, MA 02129-2060
USA

e-mail          msh at nmr.mgh.harvard.edu
Tel             +1 617 726 0323
FAX             +1 617 726 7422



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20090215/a7b035ab/attachment.html