[Mne_analysis] Fwd: Re: many epochs dropped due to 'NO_DATA'

[Alexandre Gramfort]

have a  look at raw.first_samp

it can interfere. It is basically an offset in samples

A



On Thu, Apr 7, 2016 at 12:24 PM, Nakagawa Tristan T
<nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
>
> Hi,
>
> Thanks for the hint; It must have something to do with it.
>
> I must be overlooking something here, but I can't get it right:
> Somehow it looks as if the epochs method looks for timestamps from
> before the cropping of the data from somewhere (?), although I'm not
> sure how that is possible
>
> I first crop my data with raw.crop, save that data, and load it again
> from file.
> I previously crop the file at 232 seconds - it loses the first 44 epochs.
> Other files I don't crop don't have this problem.
>
> I can't figure out what's wrong about the time indices:
>
> the resulting raw structure has the correct raw.time (from 0 to 618, at
> sfreq=1000) and time indices:
> raw[0][1] goes from 0 to 618;
> len(raw[0][1]): 618001
>
> My events array  goes like this:
> In [37]: events
> Out[37]:
> array([[  9279,      0,      1],
>        [ 18677,      0,      2],
>        [ 28475,      0,      3],
>        [ 38491,      0,      3],
>        [ 49491,      0,      1],
>        [ 58705,      0,      3],
>        [ 69505,      0,      1],
>        [ 79905,      0,      1],
>        [ 89736,      0,      1],
>        [ 99134,      0,      2],
>        [108933,      0,      1],
>        [118948,      0,      1],
>        [128146,      0,      2],
>        [138361,      0,      3],
>        [147759,      0,      2],
>        [157558,      0,      3],
>        [167774,      0,      1],
>        ......
>        [595400,      0,      1]])
>
>
>
> Notably, I lose the first epochs, not the last epochs:
>
>
> epochs = mne.Epochs(raw,events, event_id, tmin=tmin, tmax=tmax,
> proj=True, picks=picks_plan2,
>                     baseline=None, preload=True,
>                     reject=None)
>
> In [35]: 60 matching events found
> Applying baseline correction (mode: mean)
> 0 projection items activated
> Loading data for 60 events and 1001 original time points ...
> 44 bad epochs dropped
>
> In [36]: epochs.drop_log
> Out[36]:
> [['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  ['NO_DATA'],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  [],
>  []]
>
> Any Ideas?
> Many Thanks,
>
> cheers Tristan
>
> On 2016年04月07日 17:02, Alexandre Gramfort wrote:
>> hi Tristan,
>>
>> this is telling you that your dropped epochs exceed the time window of
>> your raw data.
>>
>> check that the time indices (first column of events) are compatible
>> with the raw object.
>>
>> yes the raw.info['events'] are not used.
>>
>> HTH
>> Alex
>>
>> On Thu, Apr 7, 2016 at 9:33 AM, Nakagawa Tristan T
>> <nakagawa-t at ifrec.osaka-u.ac.jp> wrote:
>>> Dear all,
>>>
>>> When trying to epoch my data, I get 44 out of 60 epochs dropped due to
>>> 'NO_DATA'.
>>>
>>> What does this error mean? it's not dropping epochs due to bad data, right?
>>>
>>> I thought it might be due to my having cropped the raw data, saved the
>>> events in an array, and now trying to do epochs with this 'events' array.
>>> After preprocessing, I have 360 channels and last time point at 618000
>>> The raw.info['events'] dict returned the old information:
>>> [{'channels': array([397], dtype=int32),
>>>   'list': array([     36,       1,       0, ..., 1289236,       0,
>>> 1], dtype=int32)}]
>>>
>>> However, setting:
>>> raw.info['events'] =[]
>>> doesn't help.
>>>
>>> Here the epoching command I use:
>>> epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
>>> picks=picks,
>>>                     baseline=None, preload=True,
>>>                     reject=None)
>>>
>>> and the epochs array:
>>> array([[  8819,      0,      2],
>>>        [ 17833,      0,      3],
>>>        [ 28433,      0,      3],
>>>        [ 39049,      0,      1],
>>>        [ 49849,      0,      3],
>>>        [ 58846,      0,      3],
>>>        ....]])
>>>
>>> thanks for any help,
>>> Tristan
>>>
>>>
>>> --
>>> Tristan T. Nakagawa, Ph.D.
>>> Laboratory for Brain-Immune Interaction,
>>> iFReC, Osaka University
>>> 3-1 Yamadaoka, Suita, Osaka, Japan
>>> Tel: 0668789710
>>> Office: CiNet R 2B6-2
>>> http://seymourlab.com
>>>
>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>
>>>
>>> The information in this e-mail is intended only for the person to whom it is
>>> addressed. If you believe this e-mail was sent to you in error and the e-mail
>>> contains patient information, please contact the Partners Compliance HelpLine at
>>> http://www.partners.org/complianceline . If the e-mail was sent to you in error
>>> but does not contain patient information, please contact the sender and properly
>>> dispose of the e-mail.
>>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis