Who says FDR is more conservative?  It all depends on the thresholds you choose.

Note that the FDR threshold is different from the cluster-based p threshold, and it isn't necessarily appropriate to use the same for both.

The cluster-based p threshold controls the overall probability that you will get 1 or more false positives.  With p<0.05, the idea is that if you repeated your study 20 times, 19 times out of 20, you would have NO false positives.

With an FDR threshold of 0.05, you are guaranteed that 5% of your "significant" vertices are false positives.  A more conservative threshold would be 0.01.

By the way, for cluster thresholding, there are two p thresholds that are important.  The first is the p threshold applied to the z-stats or t-stats before finding clusters.  0.05 would usually be too liberal.  0.01 or 0.001 would be better.  The second is the multiple comparison corrected p value, for which you would usually use 0.05.

Date: Fri, 29 Jan 2010 10:02:45 -0500
From: stefan.brauns@googlemail.com
To: Freesurfer@nmr.mgh.harvard.edu
Subject: [Freesurfer] thickness maps: FDR versus Monte Carlo - different results

Hi there,

we are struggling with one question and haven't yet heard back. We would be very grateful for any advice or recommendation.

We were running an analysis (95 subjects) examining a group effect (two groups, controlling for the effects of a covariate and two cofactors, DODS model) on cortical thickness with mri_glmfit. We then corrected our results for multiple comparisons using two methods, FDR and Monte Carlo simulation with 4000 repeats. The threshold was set at p=0.05 respectively. Although examining the same population we got results in very different regions.

command for the simulation:

mri_glmfit-sim --glmdir xxxx.glmdir --sim mc-z 4000 1.301 mc-z abs.1.301 --sim-sign abs

the fdr results were obtained setting the fdr-threshold at p=0.05 with the tksurfer script command (sclv_set current_threshold_using_fdr 0.05 0)

With the fdr method we found a spot in the left supramarginal gyrus and with the clusterwise correction we got a cluster in the right middle temporal lobe (see pictures of corrected and uncorrected results attached). No other clusters/findings survived one of the correction methods.

We are well aware of the fact that FDR and Monte Carlo simulation are very different statistical methods and that FDR is more conservative. Does that explain the discrepant results? Would you expect a highly robust FDR finding to not show up at all when using Monte Carlo? What additional information could be used to decide which method to use for the final models?  
 
Thank you,

Stefan





Hotmail: Free, trusted and rich email service. Get it now.