[Mne_analysis] Source Space Decoding Classification Timecourse

[JR KING]

Hi Cody,

Overall, your baseline doesn't look too bad - you would need to do a
statistical test to check whether it is just noise variation or
above-chance decoding scores.

Still there could be multiple reasons behind a significant accuracy before
t0 here:
- accuracy is biased for imbalanced datasets. You can either
use epochs.equalize_event_counts before your cross validation, or better,
use a 'roc_auc' scoring metrics
- filtering the data can spread information over time. Try changing your
filtering parameters
- IIRC, the 'sample' protocol is actually not randomized, and it is
possible to predict the simulus category in advance.

If you're using the MNE master branch, then I would recommend simplfy using
this instead of your big loop (see
https://martinos.org/mne/dev/auto_tutorials/plot_sensors_decoding.html#temporal-decoding
for more details):

clf = make_pipeline(StandardScaler(), SelectKBest(f_classif, k=500),
SVC(kernel='linear'))
time_decod = SlidingEstimator(clf, scoring='roc_auc')
scores = cross_val_multiscore(clf, X, y, cv=5)
plt.plot(times, scores.mean(0))

(Note that I would personnally recommend clf =
make_pipeline(StandardScaler(), LogisticRegression(C=1)) which should be
better)

Else, I believe we will be releasing the next version of MNE this month, so
you'll just have to update MNE.

Hope that helps,

Jean-Rémi





On 4 August 2017 at 17:19, Ghuman, Avniel <ghumana at upmc.edu> wrote:

> Hi Cody,
>
> Do you have the same number of trials in each condition after any trial
> rejection you do? If not, then the issue might be that 50% is not the
> correct chance level to think about, rather the correct chance level is the
> proportion of trials that is in your more frequent condition (eyeballing,
> maybe like 55%?). There are unbiased classifiers you can use, but I am not
> sure if they are built into MNE python...
>
> Best wishes,
> Avniel
>
> ________________________________
> From: mne_analysis-bounces at nmr.mgh.harvard.edu [
> mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Cushing, Cody [
> CCUSHING1 at mgh.harvard.edu]
> Sent: Friday, August 04, 2017 5:11 PM
> To: mne_analysis at nmr.mgh.harvard.edu
> Subject: [Mne_analysis] Source Space Decoding Classification Timecourse
>
> Hi,
>
> I've been trying to modify the following example:
>
> http://martinos.org/mne/dev/auto_examples/decoding/plot_
> decoding_spatio_temporal_source.html
>
> to yield a time resolved classification accuracy.  I'm new to decoding so
> I've done it in a fairly brute way (just iterating this script over every
> time point), which yields a fairly convincing classification accuracy
> timecourse.  However, I'm a bit concerned at how high the accuracy is
> during the baseline, pre-stim period.  See attached for the modified script
> using the sample data and an example of the output.  I'm new to decoding,
> but the best answer I've been able to find for abnormally high pre-stim
> accuracy is failing to cross validate, but that shouldn't be the case as
> cross validation is being performed (but perhaps I'm doing it wrong) .  Is
> there something improper about my strategy here?  Thanks for any input.
>
> Cheers,
> Cody
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170805/083cedc4/attachment.html