[Mne_analysis] Stats on decoding scores

[JR KING]

        External Email - Use Caution        

The clusters locations are given in the second output variable, see the
documentation

HTH
JR

On Tue, 18 Feb 2020 at 04:32, Neeraj Sharma <neerajww at gmail.com> wrote:

>         External Email - Use Caution
>
> Hi List,
>
> I am also trying to understand mne.stats.permutation_cluster_1samp_test().
> I used it as follows.
>
> t_obs, clusters, clusters_pv, H0 =
> mne.stats.permutation_cluster_1samp_test(X,threshold=.01,out_type='indices')
>
> Here, X is 12x512 containing MMN temporal traces of 12 subjects. The t_obs
> is as shown here (http://https://imgur.com/pQDQMAY). I am not able to
> find the cluster locations in the cluster variable (in output). The
> cluster_pv is 12x1, indicating it has found 12 clusters of varying
> p-values. Any suggestions if I am missing some interpretation or passing
> some variable incorrectly.
>
> Best regards,
> Neeraj
>
> On Mon, Feb 17, 2020 at 10:31 AM JR KING <jeanremi.king at gmail.com> wrote:
>
>>         External Email - Use Caution
>>
>> Hi Maryam,
>>
>> you can use
>> https://mne.tools/stable/generated/mne.stats.spatio_temporal_cluster_1samp_test.html
>>
>> where X is the accuracy array of shape n_subjects x n_times x 1,
>>
>> hope that helps,
>>
>> Kindest regards
>>
>> JR
>>
>> On Tue, 11 Feb 2020 at 21:24, Maryam Zolfaghar <
>> Maryam.Zolfaghar at colorado.edu> wrote:
>>
>>>         External Email - Use Caution
>>>
>>> Hi Phillip,
>>>
>>> Thank you for the response.
>>>
>>> More specifically, I am trying to use the "STATISTICAL ANALYSIS OF
>>> DECODING ACCURACY" in this paper
>>> <https://www.jneurosci.org/content/38/2/409.long>. They used MATLAB and
>>> I am trying to use MNE Python.
>>>
>>> They did the following steps and report clusters of time points in
>>> which the decoding was significantly greater than chance after correction
>>> for multiple comparisons (e.g. Figure 3)
>>> In Step 1, they tested whether the obtained decoding accuracy at each
>>> individual time point during the delay interval was greater than chance
>>> using one-sample t-tests comparing the mean accuracy across
>>> participants to chance.
>>> In Step 2, they constructed a Monte Carlo null distribution of
>>> cluster-level t mass values.
>>> In Step 3, they obtained a null distribution for the cluster mass.
>>>
>>> P.s. I am doing this analysis on my own project and data but I also
>>> want to present MNE in a neuroscience department who are only using MATLAB
>>> to show using python and MNE could be another great option and save their
>>> time. That is why I am trying to see if I can use MNE for all steps instead
>>> of implementing them by myself in python).
>>>
>>> Thank you,
>>> -Maryam
>>>
>>> On Tue, Feb 11, 2020 at 8:55 AM Phillip Alday <phillip.alday at mpi.nl>
>>> wrote:
>>>
>>>> Hi Maryam,
>>>>
>>>> First: cluster-based permutation tests won't tell you whether any
>>>> particular times/clusters are actually significant (see
>>>> http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test/).
>>>> The null-hypotheses of these tests is exchangeability of conditions, at
>>>> least when they're defined over the two-sample t-test; there is some debate
>>>> over on the FieldTrip mailing list as to whether they make any sense for
>>>> the one-sample t-tests. I do think you can construct a meaningful
>>>> cluster-based permutation test using one-sample t-tests in some situations,
>>>> including in decoding situations, but you have to be careful.
>>>>
>>>> Second: You have to transform your decoding scores to be on an
>>>> unbounded scale before using the t-test or use a different test to
>>>> construct your permutation test. This follows directly from the assumptions
>>>> of the t-test (unboundedness and equal variance) and will be especially
>>>> problematic when your decoding scores in some temporal regions are close to
>>>> one, but close to 0.5 in other temporal regions, because these cannot have
>>>> equal variance (the variance of the binomial distribution is a function of
>>>> its mean). This is discussed in decoding analyzes of fMRI in Allefeld et
>>>> al. 2015 (https://doi.org/10.1016/j.neuroimage.2016.07.040). For a
>>>> simpler way around this, you could use the Fisher transformation (for
>>>> correlations) or the logistic function to get decoding scores on an
>>>> unbounded scale.
>>>>
>>>> If you still think you want to try to use a cluster-based permutation
>>>> test, let me know and I'll see if I can extract the relevant code from a
>>>> study I'm currently working on.
>>>>
>>>> Best,
>>>>
>>>> Phillip
>>>>
>>>>
>>>>
>>>> On 09/02/2020 03:01, Maryam Zolfaghar wrote:
>>>>
>>>>         External Email - Use Caution
>>>> Hi all,
>>>>
>>>> I'm trying to analyze whether my decoding scores over time (
>>>>
>>>> https://mne.tools/stable/auto_tutorials/machine-learning/plot_sensors_decoding.html#decoding-over-time)
>>>> are "significant" or not, doing permutation testing and cluster-based
>>>> correction. Does anyone have any idea how to do it in MNE?
>>>>
>>>> Thank you,
>>>> -Maryam
>>>>
>>>>
>>>> _______________________________________________
>>>> Mne_analysis mailing listMne_analysis at nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>
>>>> _______________________________________________
>>> Mne_analysis mailing list
>>> Mne_analysis at nmr.mgh.harvard.edu
>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200218/e4bdf3f3/attachment-0001.html