[Mne_analysis] Fwd: Re: many epochs dropped due to 'NO_DATA'
Jaakko Leppäkangas
jaeilepp at student.jyu.fi
Fri Apr 8 01:50:41 EDT 2016
I'm not sure if it is a good idea to set the first_samp property by hand.
It could have some side effects elsewhere. In general, you shouldn't have
to change the private attributes yourself. I don't know your analysis pipe
line, but the events should line up properly if you do mne.find_events to
the cropped data. This way the events outside the cropping window will be
excluded.
On 8 April 2016 at 02:56, Nakagawa Tristan T <nakagawa-t at ifrec.osaka-u.ac.jp
> wrote:
> That was it; when adding raw.first_samp to all my events I lose no epochs.
> I guess a cleaner way is to reset the first_samp property.
>
> For others with this problem, instead of:
> raw.first_samp=0
>
> do this (found by asking ipython for the definition with:
> raw.first_samp?? ):
> raw._first_samps[0]=0
>
> May I suggest a switch in the crop method (e.g.: keep_offset=True)?
> Thanks for the help!
>
> Tristan
>
>
> On 2016年04月07日 19:36, Alexandre Gramfort wrote:
> > have a look at raw.first_samp
> >
> > it can interfere. It is basically an offset in samples
> >
> > A
> >
> >
> >
> > On Thu, Apr 7, 2016 at 12:24 PM, Nakagawa Tristan T
> > <nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
> >> Hi,
> >>
> >> Thanks for the hint; It must have something to do with it.
> >>
> >> I must be overlooking something here, but I can't get it right:
> >> Somehow it looks as if the epochs method looks for timestamps from
> >> before the cropping of the data from somewhere (?), although I'm not
> >> sure how that is possible
> >>
> >> I first crop my data with raw.crop, save that data, and load it again
> >> from file.
> >> I previously crop the file at 232 seconds - it loses the first 44
> epochs.
> >> Other files I don't crop don't have this problem.
> >>
> >> I can't figure out what's wrong about the time indices:
> >>
> >> the resulting raw structure has the correct raw.time (from 0 to 618, at
> >> sfreq=1000) and time indices:
> >> raw[0][1] goes from 0 to 618;
> >> len(raw[0][1]): 618001
> >>
> >> My events array goes like this:
> >> In [37]: events
> >> Out[37]:
> >> array([[ 9279, 0, 1],
> >> [ 18677, 0, 2],
> >> [ 28475, 0, 3],
> >> [ 38491, 0, 3],
> >> [ 49491, 0, 1],
> >> [ 58705, 0, 3],
> >> [ 69505, 0, 1],
> >> [ 79905, 0, 1],
> >> [ 89736, 0, 1],
> >> [ 99134, 0, 2],
> >> [108933, 0, 1],
> >> [118948, 0, 1],
> >> [128146, 0, 2],
> >> [138361, 0, 3],
> >> [147759, 0, 2],
> >> [157558, 0, 3],
> >> [167774, 0, 1],
> >> ......
> >> [595400, 0, 1]])
> >>
> >>
> >>
> >> Notably, I lose the first epochs, not the last epochs:
> >>
> >>
> >> epochs = mne.Epochs(raw,events, event_id, tmin=tmin, tmax=tmax,
> >> proj=True, picks=picks_plan2,
> >> baseline=None, preload=True,
> >> reject=None)
> >>
> >> In [35]: 60 matching events found
> >> Applying baseline correction (mode: mean)
> >> 0 projection items activated
> >> Loading data for 60 events and 1001 original time points ...
> >> 44 bad epochs dropped
> >>
> >> In [36]: epochs.drop_log
> >> Out[36]:
> >> [['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> ['NO_DATA'],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> [],
> >> []]
> >>
> >> Any Ideas?
> >> Many Thanks,
> >>
> >> cheers Tristan
> >>
> >> On 2016年04月07日 17:02, Alexandre Gramfort wrote:
> >>> hi Tristan,
> >>>
> >>> this is telling you that your dropped epochs exceed the time window of
> >>> your raw data.
> >>>
> >>> check that the time indices (first column of events) are compatible
> >>> with the raw object.
> >>>
> >>> yes the raw.info['events'] are not used.
> >>>
> >>> HTH
> >>> Alex
> >>>
> >>> On Thu, Apr 7, 2016 at 9:33 AM, Nakagawa Tristan T
> >>> <nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
> >>>> Dear all,
> >>>>
> >>>> When trying to epoch my data, I get 44 out of 60 epochs dropped due to
> >>>> 'NO_DATA'.
> >>>>
> >>>> What does this error mean? it's not dropping epochs due to bad data,
> right?
> >>>>
> >>>> I thought it might be due to my having cropped the raw data, saved the
> >>>> events in an array, and now trying to do epochs with this 'events'
> array.
> >>>> After preprocessing, I have 360 channels and last time point at 618000
> >>>> The raw.info['events'] dict returned the old information:
> >>>> [{'channels': array([397], dtype=int32),
> >>>> 'list': array([ 36, 1, 0, ..., 1289236, 0,
> >>>> 1], dtype=int32)}]
> >>>>
> >>>> However, setting:
> >>>> raw.info['events'] =[]
> >>>> doesn't help.
> >>>>
> >>>> Here the epoching command I use:
> >>>> epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
> >>>> picks=picks,
> >>>> baseline=None, preload=True,
> >>>> reject=None)
> >>>>
> >>>> and the epochs array:
> >>>> array([[ 8819, 0, 2],
> >>>> [ 17833, 0, 3],
> >>>> [ 28433, 0, 3],
> >>>> [ 39049, 0, 1],
> >>>> [ 49849, 0, 3],
> >>>> [ 58846, 0, 3],
> >>>> ....]])
> >>>>
> >>>> thanks for any help,
> >>>> Tristan
> >>>>
> >>>>
> >>>> --
> >>>> Tristan T. Nakagawa, Ph.D.
> >>>> Laboratory for Brain-Immune Interaction,
> >>>> iFReC, Osaka University
> >>>> 3-1 Yamadaoka, Suita, Osaka, Japan
> >>>> Tel: 0668789710
> >>>> Office: CiNet R 2B6-2
> >>>> http://seymourlab.com
> >>>>
> >>>> _______________________________________________
> >>>> Mne_analysis mailing list
> >>>> Mne_analysis at nmr.mgh.harvard.edu
> >>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> >>>>
> >>>>
> >>>> The information in this e-mail is intended only for the person to
> whom it is
> >>>> addressed. If you believe this e-mail was sent to you in error and
> the e-mail
> >>>> contains patient information, please contact the Partners Compliance
> HelpLine at
> >>>> http://www.partners.org/complianceline . If the e-mail was sent to
> you in error
> >>>> but does not contain patient information, please contact the sender
> and properly
> >>>> dispose of the e-mail.
> >>>>
> >>> _______________________________________________
> >>> Mne_analysis mailing list
> >>> Mne_analysis at nmr.mgh.harvard.edu
> >>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> >>
> >>
> >> _______________________________________________
> >> Mne_analysis mailing list
> >> Mne_analysis at nmr.mgh.harvard.edu
> >> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> > _______________________________________________
> > Mne_analysis mailing list
> > Mne_analysis at nmr.mgh.harvard.edu
> > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> --
> Tristan T. Nakagawa, Ph.D.
> Laboratory for Brain-Immune Interaction,
> iFReC, Osaka University
> 3-1 Yamadaoka, Suita, Osaka, Japan
> Tel: 0668789710
> Office: CiNet R 2B6-2
> http://seymourlab.com
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20160408/6a449451/attachment-0001.html
More information about the Mne_analysis
mailing list