[Mne_analysis] Fwd: Re: many epochs dropped due to 'NO_DATA'
Nakagawa Tristan T
nakagawa-t at ifrec.osaka-u.ac.jp
Thu Apr 7 20:56:36 EDT 2016
That was it; when adding raw.first_samp to all my events I lose no epochs.
I guess a cleaner way is to reset the first_samp property.
For others with this problem, instead of:
raw.first_samp=0
do this (found by asking ipython for the definition with:
raw.first_samp?? ):
raw._first_samps[0]=0
May I suggest a switch in the crop method (e.g.: keep_offset=True)?
Thanks for the help!
Tristan
On 2016年04月07日 19:36, Alexandre Gramfort wrote:
> have a look at raw.first_samp
>
> it can interfere. It is basically an offset in samples
>
> A
>
>
>
> On Thu, Apr 7, 2016 at 12:24 PM, Nakagawa Tristan T
> <nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
>> Hi,
>>
>> Thanks for the hint; It must have something to do with it.
>>
>> I must be overlooking something here, but I can't get it right:
>> Somehow it looks as if the epochs method looks for timestamps from
>> before the cropping of the data from somewhere (?), although I'm not
>> sure how that is possible
>>
>> I first crop my data with raw.crop, save that data, and load it again
>> from file.
>> I previously crop the file at 232 seconds - it loses the first 44 epochs.
>> Other files I don't crop don't have this problem.
>>
>> I can't figure out what's wrong about the time indices:
>>
>> the resulting raw structure has the correct raw.time (from 0 to 618, at
>> sfreq=1000) and time indices:
>> raw[0][1] goes from 0 to 618;
>> len(raw[0][1]): 618001
>>
>> My events array goes like this:
>> In [37]: events
>> Out[37]:
>> array([[ 9279, 0, 1],
>> [ 18677, 0, 2],
>> [ 28475, 0, 3],
>> [ 38491, 0, 3],
>> [ 49491, 0, 1],
>> [ 58705, 0, 3],
>> [ 69505, 0, 1],
>> [ 79905, 0, 1],
>> [ 89736, 0, 1],
>> [ 99134, 0, 2],
>> [108933, 0, 1],
>> [118948, 0, 1],
>> [128146, 0, 2],
>> [138361, 0, 3],
>> [147759, 0, 2],
>> [157558, 0, 3],
>> [167774, 0, 1],
>> ......
>> [595400, 0, 1]])
>>
>>
>>
>> Notably, I lose the first epochs, not the last epochs:
>>
>>
>> epochs = mne.Epochs(raw,events, event_id, tmin=tmin, tmax=tmax,
>> proj=True, picks=picks_plan2,
>> baseline=None, preload=True,
>> reject=None)
>>
>> In [35]: 60 matching events found
>> Applying baseline correction (mode: mean)
>> 0 projection items activated
>> Loading data for 60 events and 1001 original time points ...
>> 44 bad epochs dropped
>>
>> In [36]: epochs.drop_log
>> Out[36]:
>> [['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> ['NO_DATA'],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> [],
>> []]
>>
>> Any Ideas?
>> Many Thanks,
>>
>> cheers Tristan
>>
>> On 2016年04月07日 17:02, Alexandre Gramfort wrote:
>>> hi Tristan,
>>>
>>> this is telling you that your dropped epochs exceed the time window of
>>> your raw data.
>>>
>>> check that the time indices (first column of events) are compatible
>>> with the raw object.
>>>
>>> yes the raw.info['events'] are not used.
>>>
>>> HTH
>>> Alex
>>>
>>> On Thu, Apr 7, 2016 at 9:33 AM, Nakagawa Tristan T
>>> <nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
>>>> Dear all,
>>>>
>>>> When trying to epoch my data, I get 44 out of 60 epochs dropped due to
>>>> 'NO_DATA'.
>>>>
>>>> What does this error mean? it's not dropping epochs due to bad data, right?
>>>>
>>>> I thought it might be due to my having cropped the raw data, saved the
>>>> events in an array, and now trying to do epochs with this 'events' array.
>>>> After preprocessing, I have 360 channels and last time point at 618000
>>>> The raw.info['events'] dict returned the old information:
>>>> [{'channels': array([397], dtype=int32),
>>>> 'list': array([ 36, 1, 0, ..., 1289236, 0,
>>>> 1], dtype=int32)}]
>>>>
>>>> However, setting:
>>>> raw.info['events'] =[]
>>>> doesn't help.
>>>>
>>>> Here the epoching command I use:
>>>> epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
>>>> picks=picks,
>>>> baseline=None, preload=True,
>>>> reject=None)
>>>>
>>>> and the epochs array:
>>>> array([[ 8819, 0, 2],
>>>> [ 17833, 0, 3],
>>>> [ 28433, 0, 3],
>>>> [ 39049, 0, 1],
>>>> [ 49849, 0, 3],
>>>> [ 58846, 0, 3],
>>>> ....]])
>>>>
>>>> thanks for any help,
>>>> Tristan
>>>>
>>>>
>>>> --
>>>> Tristan T. Nakagawa, Ph.D.
>>>> Laboratory for Brain-Immune Interaction,
>>>> iFReC, Osaka University
>>>> 3-1 Yamadaoka, Suita, Osaka, Japan
>>>> Tel: 0668789710
>>>> Office: CiNet R 2B6-2
>>>> http://seymourlab.com
>>>>
>>>> _______________________________________________
>>>> Mne_analysis mailing list
>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>
>>>>
>>>> The information in this e-mail is intended only for the person to whom it is
>>>> addressed. If you believe this e-mail was sent to you in error and the e-mail
>>>> contains patient information, please contact the Partners Compliance HelpLine at
>>>> http://www.partners.org/complianceline . If the e-mail was sent to you in error
>>>> but does not contain patient information, please contact the sender and properly
>>>> dispose of the e-mail.
>>>>
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
--
Tristan T. Nakagawa, Ph.D.
Laboratory for Brain-Immune Interaction,
iFReC, Osaka University
3-1 Yamadaoka, Suita, Osaka, Japan
Tel: 0668789710
Office: CiNet R 2B6-2
http://seymourlab.com
More information about the Mne_analysis
mailing list