Hi Lea, I'm cc'ing the list so that others can benefit.
On 06/13/2018 12:28 PM, Backhausen, Lea wrote:
Hey Doug,
Thanks for your helpful suggestions on Monte Carlo simulation and permutation you gave in the Mailing List!
I have some follow-up questions, also regarding your recently published paper about false positive rates (https://www.sciencedirect.com/science/article/pii/S1053811917310960).
- Concerning Monte Carlo simulation where you said even 1000
iterations would be okay: What will the difference be between 1000 and 10,000 iterations? I read in the tutorial that >5000 are recommended?
This has to do with the precision of the cluster p-value Since you are doing a simulation, the p-value is subject to sampling error. If you look in the output file, you will see that I give 95% confidence intervals for the p-value, so as long as the high range is less than .05, then it should be ok. I think 1000 is probably fine.
- In your paper you write that permutation requires that data is
exchangeable across subjects and that age effects may be a problem. In our study we want to compare adolescent patients with ADHD and conduct disorder with typically developing adolescents. In this context, data would not be exchangeable across subjects, right? Could we still use permutation?
Exchangeability is a very complicated subject and not very intuitive. It is too complicated to get into here. I suggest reading up on it a little. It does not mean that you can swap groups without there being any effect.
- When using the CFT thresholds with Monte Carlo simulation you
recommend in your paper, do you still only report clusters with CWP<.05 or is the CWP less important when using stricter thresholds?
Generally, CWP must be less than 0.05 for publication. Sometimes people will show results with more liberal p-values (eg, 0.1) to show "trends". It all depends on what you can get past your reviewers:). doug
Thank you so much for your input!
Best
Lea Backhausen
Lea Backhausen
Research Assistant
Department of Child and Adolescent Psychiatry, Faculty of Medicine of the TU Dresden, Germany http://www.uniklinikum-dresden.de http://www.uniklinikum-dresden.de/