[Mne_analysis] Stats on decoding scores

[Neeraj Sharma]

        External Email - Use Caution        

Hi List,

I am also trying to understand mne.stats.permutation_cluster_1samp_test().
I used it as follows.

t_obs, clusters, clusters_pv, H0 =
mne.stats.permutation_cluster_1samp_test(X,threshold=.01,out_type='indices')

Here, X is 12x512 containing MMN temporal traces of 12 subjects. The t_obs
is as shown here (http://https://imgur.com/pQDQMAY). I am not able to find
the cluster locations in the cluster variable (in output). The cluster_pv
is 12x1, indicating it has found 12 clusters of varying p-values. Any
suggestions if I am missing some interpretation or passing some variable
incorrectly.

Best regards,
Neeraj

On Mon, Feb 17, 2020 at 10:31 AM JR KING <jeanremi.king at gmail.com> wrote:

>         External Email - Use Caution
>
> Hi Maryam,
>
> you can use
> https://mne.tools/stable/generated/mne.stats.spatio_temporal_cluster_1samp_test.html
>
> where X is the accuracy array of shape n_subjects x n_times x 1,
>
> hope that helps,
>
> Kindest regards
>
> JR
>
> On Tue, 11 Feb 2020 at 21:24, Maryam Zolfaghar <
> Maryam.Zolfaghar at colorado.edu> wrote:
>
>>         External Email - Use Caution
>>
>> Hi Phillip,
>>
>> Thank you for the response.
>>
>> More specifically, I am trying to use the "STATISTICAL ANALYSIS OF
>> DECODING ACCURACY" in this paper
>> <https://www.jneurosci.org/content/38/2/409.long>. They used MATLAB and
>> I am trying to use MNE Python.
>>
>> They did the following steps and report clusters of time points in which
>> the decoding was significantly greater than chance after correction for
>> multiple comparisons (e.g. Figure 3)
>> In Step 1, they tested whether the obtained decoding accuracy at each
>> individual time point during the delay interval was greater than chance
>> using one-sample t-tests comparing the mean accuracy across participants
>> to chance.
>> In Step 2, they constructed a Monte Carlo null distribution of
>> cluster-level t mass values.
>> In Step 3, they obtained a null distribution for the cluster mass.
>>
>> P.s. I am doing this analysis on my own project and data but I also want
>> to present MNE in a neuroscience department who are only using MATLAB to
>> show using python and MNE could be another great option and save their
>> time. That is why I am trying to see if I can use MNE for all steps instead
>> of implementing them by myself in python).
>>
>> Thank you,
>> -Maryam
>>
>> On Tue, Feb 11, 2020 at 8:55 AM Phillip Alday <phillip.alday at mpi.nl>
>> wrote:
>>
>>> Hi Maryam,
>>>
>>> First: cluster-based permutation tests won't tell you whether any
>>> particular times/clusters are actually significant (see
>>> http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test/).
>>> The null-hypotheses of these tests is exchangeability of conditions, at
>>> least when they're defined over the two-sample t-test; there is some debate
>>> over on the FieldTrip mailing list as to whether they make any sense for
>>> the one-sample t-tests. I do think you can construct a meaningful
>>> cluster-based permutation test using one-sample t-tests in some situations,
>>> including in decoding situations, but you have to be careful.
>>>
>>> Second: You have to transform your decoding scores to be on an unbounded
>>> scale before using the t-test or use a different test to construct your
>>> permutation test. This follows directly from the assumptions of the t-test
>>> (unboundedness and equal variance) and will be especially problematic when
>>> your decoding scores in some temporal regions are close to one, but close
>>> to 0.5 in other temporal regions, because these cannot have equal variance
>>> (the variance of the binomial distribution is a function of its mean). This
>>> is discussed in decoding analyzes of fMRI in Allefeld et al. 2015 (
>>> https://doi.org/10.1016/j.neuroimage.2016.07.040). For a simpler way
>>> around this, you could use the Fisher transformation (for correlations) or
>>> the logistic function to get decoding scores on an unbounded scale.
>>>
>>> If you still think you want to try to use a cluster-based permutation
>>> test, let me know and I'll see if I can extract the relevant code from a
>>> study I'm currently working on.
>>>
>>> Best,
>>>
>>> Phillip
>>>
>>>
>>>
>>> On 09/02/2020 03:01, Maryam Zolfaghar wrote:
>>>
>>>         External Email - Use Caution
>>> Hi all,
>>>
>>> I'm trying to analyze whether my decoding scores over time (
>>>
>>> https://mne.tools/stable/auto_tutorials/machine-learning/plot_sensors_decoding.html#decoding-over-time)
>>> are "significant" or not, doing permutation testing and cluster-based
>>> correction. Does anyone have any idea how to do it in MNE?
>>>
>>> Thank you,
>>> -Maryam
>>>
>>>
>>> _______________________________________________
>>> Mne_analysis mailing listMne_analysis at nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>
>>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200217/358de9ff/attachment.html