[Mne_analysis] Question about permutation testing

[Eric Larson]

        External Email - Use Caution        

>
> If I then inspect the results, I get significant clusters during stimulus
> presentation (which is nice. However, there are  no significant
> anticipatory clusters, even though numerically the conditions clearly
> differ.
> Interestingly, I am able to make these anticipatory clusters significant
> if instead of analysing the whole time course, I limit the permutation test
> to the anticipatory window. I assume that this is the case because the
> anticipatory cluster is smaller than the reactive cluster????
>

The `step_down_p` is meant to deal with this case. Basically it runs the
clustering, and if it finds any clusters below the step_down_p threshold
(usually 0.05 is a good choice), it cuts those data points out of the
analysis and redoes the permutation test. This prevents the first cluster
from dominating the H0 values, increasing sensitivity to smaller but still
valid effects.

My question is whether this is a valid approach, and if not whether there
> is another alternative way to still show that the conditions already differ
> before stimulus processing.
>

It seems reasonable, but keep in mind that interpreting
cluster spatio-temporal locations from tests is considered problematic (see
the bottom of this section in our stats tutorial
<https://mne.tools/dev/auto_tutorials/discussions/plot_background_statistics.html#clustering>,
and the linked Fieldtrip tutorial
<http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test/>).
A potential workaround is to use TFCE (see the stats tutorial), though it
will take longer to run.
<http://www.fieldtriptoolbox.org/faq/how_not_to_interpret_results_from_a_cluster-based_permutation_test/>

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200410/398d4058/attachment.html