[Mne_analysis] Grand Averaging from Text Events Files

Brian Coffman bcoffman at mrn.org
Mon Jan 30 19:00:53 EST 2012
Search archives: 
Hello everyone,
I have tested many different things and have decided that the issue was
that I was using the UNIX command "sort" to recombine and sort event lists
for individual events. If I sort the text manually (using a text editing
program) and insert the events by copy and paste into a new text file, the
initial skip is read from the file without error. I still am not sure why
using sort in this way effects the ability of MNE to read the initial skip
at the head of the file, but this seems to be the issue. I have had issues
using this command in other tasks as well and I recommend others be
cautious in using this command in UNIX scripting.

Thanks for your help with this, everyone.

On Wed, Jan 25, 2012 at 2:49 PM, Dan Wakeman <dgwakeman at gmail.com> wrote:

> Hi,
>
> I think I can explain:
>
> On Wed, Jan 25, 2012 at 4:19 PM, Brian Coffman <bcoffman at mrn.org> wrote:
> > Hello again,
> >
> > On Wed, Jan 25, 2012 at 1:50 PM, Dan Wakeman <dgwakeman at gmail.com>
> wrote:
> >>
> >> On Wed, Jan 25, 2012 at 3:38 PM, Brian Coffman <bcoffman at mrn.org>
> wrote:
> >> > Hi Conrad,
> >> > Thanks for your help. This looks a bit complex for what we are doing.
> We
> >> > already get the events we are interested when averaging without
> >> > corrections
> >> > by specifying --eventsout within mne_process_raw. These files come out
> >> > with
> >> > an initial skip (offset) value in the form of "seconds milliseconds 0
> >> > 0." It
> >> > seems after a bit more testing that this initial skip is not accounted
> >> > for
> >> > when averaging from the events files that come out of MNE.
> >> How did you test this? We need to understand what tests you are doing
> >> to understand why you think it is "not accounted for."
> >
> > When averaging directly from the events file that MNE has generated
> (using
> > --eventsout) in a file with an initial skip of 4 seconds, I get a .fif
> and a
> > .log file that together suggest that MNE is averaging 4 seconds later
> than
> > it should, as if the first line of the events file is disregarded during
> > averaging.
>
> Well, I believe there is a logic to this which follows:
> time in an events file should be relative to the raw data, because
> that is what it applies to. Therefore the time in an events file is
> displayed primarily in samples as you would find in raw.first_samp
> (from [raw] = fiff_setup_read_raw(<name of raw file>)).
>
> time in a log file should be relative to the actual data points you
> have (as one cannot process non-existent data points). Therefore the
> time in a log file starts from the first sample, T, which corresponds
> to the sample # S in
> T = S - (raw.first_samp +1)
>
> This is a very small detail, but it does have a logic to it.
>
> >
> >> >
> >> > I have started to use the log file timing for the first run (log files
> >> > seem
> >> > to contain timing relative to the start of the .fif file, while the
> >> > events
> >> > files contain timing for the entire dataset, with an initial skip at
> the
> >> > beginning of the file) and this works fine. This doesn't work for the
> >> > 2nd
> >> > and 3rd file, though, as MNE looks for the dataset event time rather
> >> > than
> >> > the fif file event time. Using the events file timing for the second
> and
> >> > third runs seems to work just fine, with or without the initial skip.
> >> >
> >> > It seems very strange that MNE would write events in a way that can't
> be
> >> > read for averaging. Is this possibly related to the version of MNE I
> am
> >> > using (2.7.3)?
> >>
> >> How is MNE writing events in a way that can't be used for averaging?
> >> Can you explain what this means?
> >
> >
> > The event times in the --eventsout file are equal to the event times in
> the
> > log file + the initial skip. If the initial skip were taken into
> > consideration and subtracted from the event times that follow when
> > averaging, then one would get the correct event times, however, this does
> > not seem to be the case. The log file from an average using the event
> file
> > generated by MNE shows events are off by 4 seconds (the initial skip) and
> > are in fact the event times listed in the event file. The log file from
> an
> > average without specifying an event file shows the correct event times.
>
> Can you e-mail me (directly) the three/four files you are talking
> about, so I can see the values? To look into the error further.
>
> >>
> >>
> >> Dan
> >> >
> >> > On Wed, Jan 25, 2012 at 1:08 PM, <conrad at nmr.mgh.harvard.edu> wrote:
> >> >>
> >> >> Brian,
> >> >>
> >> >> I'm not sure why you need to change the events by an offset. I'll
> >> >> explain
> >> >> my approach to the same problem (rewriting new .eve files based on
> >> >> subject
> >> >> criteria eg. correctness). From your description, I am not sure what
> >> >> you
> >> >> mean by offsets, and I think if your triggers were at the right times
> >> >> you
> >> >> don't want to rewrite or readd any offsets just use the original
> event
> >> >> timing. (If you do want to change the timing of triggers that is
> >> >> another
> >> >> problem my lab has to deal with. We have a solution but haven't
> tested
> >> >> it
> >> >> out yet)
> >> >>
> >> >> I contain all of my subject (and study) variables in structures,
> >> >> hopefully
> >> >> their syntax is intuitive so I don't need to explain it. I do this in
> >> >> Matlab, but if you use a different programming environment I may be
> >> >> able
> >> >> to help you translate.
> >> >>
> >> >> Presume we have already extracted event files from each of the raw
> data
> >> >> blocks.
> >> >>
> >> >> > event_file = sprintf('%s/triggers/%s_%s.eve',...
> >> >> >   subject.meg.dir, subject.name, block);
> >> >> > events = load(event_file);
> >> >>
> >> >> Matlab's load command loads them perfectly. Now events is a N row by
> 4
> >> >> column matrix representing the data from the .eve file. Then my code
> >> >> goes
> >> >> line by line, finding events that correspond to triggers we sent and
> >> >> the
> >> >> next user input and evaluates their action. For events that we will
> use
> >> >> in
> >> >> the analysis I reassign by:
> >> >>
> >> >> > events(i, 4) = some_code;
> >> >>
> >> >> I do not delete events I will not use. I presume you still could by
> >> >> doing:
> >> >>
> >> >> > events(i, :) = [];
> >> >> or just make a new events matrix for ones you will include
> >> >> > new_events(end + 1, :) = events(i, :);
> >> >> just make sure to include the initial line before you do that
> >> >> > new_events(1, :) = events(1, :);
> >> >>
> >> >> Then I make a new event_file and reprint every line based on the
> events
> >> >> matrix.
> >> >>
> >> >> > event_file = sprintf('%s/triggers/%s_%s_applied.eve',...
> >> >> >   subject.meg.dir, subject.name, block);
> >> >> > fid = fopen(event_file);
> >> >> > for i = 1:size(events, 1)
> >> >> >   fprintf(fid, '%6d %3.3f %6d %3d\n',...
> >> >> >     events(i, 1), events(i, 2), events(i, 3), events(i, 4));
> >> >> > end
> >> >> > fclose(fid);
> >> >>
> >> >> I hope that makes some sense and could be useful. Good luck with it!
> >> >>
> >> >> - Conrad Nied
> >> >> Neuropsychology Lab @ MGH, David Gow
> >> >> 617 724 8846
> >> >> conrad at martinos.org
> >> >>
> >> >> > Hello MNE users,
> >> >> > I am having an odd issue that I am having trouble conceptualizing.
> >> >> >
> >> >> > I am averaging data that is split into 3 raw data files, then
> taking
> >> >> > the
> >> >> > events files generated from these averages and modifying these
> events
> >> >> > files
> >> >> > to include only events where the subject responded correctly. I am
> >> >> > then
> >> >> > taking these corrected events files and grand averaging the raw
> data
> >> >> > to
> >> >> > end
> >> >> > up with a single .fif average which is corrected for our behavioral
> >> >> > data.
> >> >> >
> >> >> > The issue I am having is related ot the initial offsets in the
> events
> >> >> > files. In all cases, there is an initial offset greater than zero
> for
> >> >> > the
> >> >> > 2nd and 3rd raw data files. In some cases, there is also an initial
> >> >> > offset
> >> >> > of a few seconds in the events file for the first of the three raw
> >> >> > data
> >> >> > files. It seems from the log files that the offset is being
> utilized
> >> >> > differently for the two cases (a few seconds offset vs a whole
> >> >> > dataset
> >> >> > offset). For the log from the first file, the offset is added to
> the
> >> >> > times,
> >> >> > while for the other files, the offset is not added. For example,
> with
> >> >> > an
> >> >> > initial offset of 4 seconds, the log for the uncorrected average
> >> >> > starts
> >> >> > at
> >> >> > 29.555 seconds, while for the corrected data, the log starts at
> >> >> > 33.555
> >> >> > seconds. The logs are identical for the second and third data
> files.
> >> >> > This
> >> >> > is also apparent in the averages, as there is no signal in the
> >> >> > corrected
> >> >> > average of the first raw file. It seems that the offsets are
> >> >> > necessary
> >> >> > for
> >> >> > the later files (data files 2 and 3), however, as I get no averages
> >> >> > without
> >> >> > them.
> >> >> >
> >> >> > My question is: Should I eliminate these offsets from the first
> text
> >> >> > events
> >> >> > file when averaging from events files? Is there something I have
> not
> >> >> > considered here?
> >> >> >
> >> >> >
> >> >> > Thanks in advance!!
> >> >> >
> >> >> > -Brian Coffman, M.S.
> >> >> > Research Associate, The Mind Research Network
> >> >> > Doctoral Candidate, The University of New Mexico
> >> >> >
> >> >> > Phone: (505) 336-0606
> >> >> > Fax: (505) 272-7011
> >> >> >
> >> >> > "I used to think that the brain was the most wonderful organ in my
> >> >> > body.
> >> >> > Then I realized who was telling me this."
> >> >> >   - Emo Phillips
> >> >> > _______________________________________________
> >> >> > Mne_analysis mailing list
> >> >> > Mne_analysis at nmr.mgh.harvard.edu
> >> >> > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> >> >>
> >> >>
> >> >>
> >> >> The information in this e-mail is intended only for the person to
> whom
> >> >> it
> >> >> is
> >> >> addressed. If you believe this e-mail was sent to you in error and
> the
> >> >> e-mail
> >> >> contains patient information, please contact the Partners Compliance
> >> >> HelpLine at
> >> >> http://www.partners.org/complianceline . If the e-mail was sent to
> you
> >> >> in
> >> >> error
> >> >> but does not contain patient information, please contact the sender
> and
> >> >> properly
> >> >> dispose of the e-mail.
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > -Brian Coffman, M.S.
> >> > Research Associate, The Mind Research Network
> >> > Doctoral Candidate, The University of New Mexico
> >> >
> >> > Phone: (505) 336-0606
> >> > Fax: (505) 272-7011
> >> >
> >> > "I used to think that the brain was the most wonderful organ in my
> body.
> >> > Then I realized who was telling me this."
> >> >   - Emo Phillips
> >> >
> >> >
> >> > _______________________________________________
> >> > Mne_analysis mailing list
> >> > Mne_analysis at nmr.mgh.harvard.edu
> >> > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> >> >
> >> >
> >> > The information in this e-mail is intended only for the person to whom
> >> > it is
> >> > addressed. If you believe this e-mail was sent to you in error and the
> >> > e-mail
> >> > contains patient information, please contact the Partners Compliance
> >> > HelpLine at
> >> > http://www.partners.org/complianceline . If the e-mail was sent to
> you
> >> > in
> >> > error
> >> > but does not contain patient information, please contact the sender
> and
> >> > properly
> >> > dispose of the e-mail.
> >> >
> >
> >
> >
> >
> > --
> > -Brian Coffman, M.S.
> > Research Associate, The Mind Research Network
> > Doctoral Candidate, The University of New Mexico
> >
> > Phone: (505) 336-0606
> > Fax: (505) 272-7011
> >
> > "I used to think that the brain was the most wonderful organ in my body.
> > Then I realized who was telling me this."
> >   - Emo Phillips
> >
>



-- 
-Brian Coffman, M.S.
Research Associate, The Mind Research Network
Doctoral Candidate, The University of New Mexico

Phone: (505) 336-0606
Fax: (505) 272-7011

"I used to think that the brain was the most wonderful organ in my body.
Then I realized who was telling me this."
  - Emo Phillips
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20120130/ab9d88c6/attachment.html

More information about the Mne_analysis mailing list