[Mne_analysis] Cluster-based Permutation T-test for Decoders

Maryam Zolfaghar Maryam.Zolfaghar at colorado.edu
Thu Oct 10 14:22:59 EDT 2019
Search archives:

        External Email - Use Caution        

Thanks for the response.
Does AUC take care of the multiple comparison issues? How I will be sure
that the accuracy is significant (above chance)?

-Mary

On Thu, Oct 10, 2019 at 4:04 AM JR KING <jeanremi.king at gmail.com> wrote:

> You can use a one-versus-all classifier and compute the average AUC
> across  categories
>
> HTH
> JR
>
> On Thu, 10 Oct 2019 at 05:24, Maryam Zolfaghar <
> Maryam.Zolfaghar at colorado.edu> wrote:
>
>>         External Email - Use Caution
>>
>> Hi all,
>>
>> I am trying to
>>
>>    - use decoders to decode whether ERP or time-frequency signals have
>>    any meaninful information of four classes (location of the target on the
>>    screen) in my experiment *over time *(according to this example
>>    <https://mne.tools/stable/auto_tutorials/machine-learning/plot_sensors_decoding.html#decoding-over-time>
>>    ).
>>    -  and then test whether the output of the decoder is significantly
>>    above the chance (in my case: 1/4=0.25) using a permutation t-test with
>>    cluster-based correction.
>>
>> My question is:
>>
>>    - In the example
>>    <https://mne.tools/stable/auto_tutorials/machine-learning/plot_sensors_decoding.html#decoding-over-time>
>>    there are only two classes, so AUC was used. However, what if there are
>>    more than two classes? How I can analyze the significance of the decoder's
>>    output with the cluster-based correction?
>>
>>
>> Thanks,
>> -Mary
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20191010/3d80a4f3/attachment.html 


More information about the Mne_analysis mailing list