Here's Tom Nichols explanation. He points out that you use the maximum i, not the minimum, so you don't have to get past i=1. Thanks Tom!
---------- Forwarded message ---------- Date: Fri, 16 Oct 2009 23:11:09 +0100 From: Thomas Nichols t.e.nichols@warwick.ac.uk To: Doug Greve greve@nmr.mgh.harvard.edu Cc: Donna Dierker donna@brainvis.wustl.edu Subject: Re: [Freesurfer] FDR correction (fwd)
Dear Doug,
Great discussion! But I think it misses the point. Can you forward the following to the list?
The Benjamini-Hochberg (BH) FDR procedure uses the inequality P(i) <= i / V * q for the ordered P-values P(1)<=P(2)<=...<=P(V) and desired level-q FDR control, which suggests that one will never get significance unless P(1)<q/V, just like Bonferroni. HOWEVER, the key element of BH-FDR is that you get to search over i and find the *largest* P-value that satisfies the inequality and use that P-value as a threshold.
Hence when considering changing resolutions (i.e. increasing V, but increasing resolution/smoothness and roughly keeping the information content of the images the same) the magic of BH-FDR is that if the 5-th %ile of uncorrected P-values is FDR-significant at one resolution, i.e. i' = 0.05*V satisfies P( i' / V ) <= i' / V * q then one would expect that the 5-th %ile of P-values after up-sampling would have a similar value, and thus would also satisfy the inequality even though V has changed.
That is the motivation that FDR is resilient to changes in resolution. However, I should note that in my own experience, leaving V fixed but changing smoothness, often changes the distribution of P-values dramatically, and thus changes the FDR result. But that should be a different beast that up-interpolation.
Hope this helps!
-Tom
On Fri, Oct 16, 2009 at 5:43 PM, Doug Greve greve@nmr.mgh.harvard.eduwrote:
Tom, here's an FDR question for you. It appears that the FDR correction is dependent on the number of voxels (need p < fdr/N just to get past i=1). Meaning that as N grows, the min p-value must also shrink to get past i=1. Any way to get around this?
thanks
doug
---------- Forwarded message ---------- Date: Fri, 16 Oct 2009 11:30:59 -0500 From: Donna Dierker donna@brainvis.wustl.edu To: Michael Harms mharms@conte.wustl.edu Cc: freesurfer@nmr.mgh.harvard.edu, Yulia WORBE yulia.worbe@upmc.fr Subject: Re: [Freesurfer] FDR correction
Regardless: FDR's sensitivity appears resolution-dependent to me.
On 10/16/2009 10:39 AM, Michael Harms wrote:
Interesting post Donna, but my understanding of FDR is that it sets the p-value threshold based on the LARGEST p-value that satisfies the FDR relationship.
That is, steps 3 and 4 in Genovese et al. (2002) are: 3) Let r be the largest i for which p <= i/V*q (assuming c=1) 4) Threshold the image at the p-value p(r).
So, it isn't the case that you require the most significant p-value to satisfy p <= 0.05/V "just to get past i=1" as you put it in your post.
Rather, you pick the largest p-value that satisfies the relationship, meaning that lower (more-significant) p-values may not have necessarily satisfied p <= i/V*q for their particular position in the sorted list of p-values.
cheers, Mike H.
On Fri, 2009-10-16 at 10:13 -0500, Donna Dierker wrote:
I never heard anything on my post here, but it might just be high
surface resolution:
http://www.mail-archive.com/neuro-mult-comp@brainvis.wustl.edu/msg00026.html
On 10/16/2009 09:58 AM, Michael Harms wrote:
Your FDR analysis sounds correct. You probably have a rather small
number of "marginally" significant vertices, which is why none survive FDR. You could try increasing the "q" value from say 0.05 to 0.1, in which case 10% of the surviving vertices would be expected to be false positives.
cheers, Mike H.
On Fri, 2009-10-16 at 12:03 +0200, Yulia WORBE wrote:
Dear Freesurfer team,
We are currently doing a cortical thickness studies between a group of psychiatric patients (n=60) and controls (n=30). We tested several smoothing levels (15mm, 20mm, 25mm)
When setting an uncorrected threshold (such as p<0.005), we obtained several regions of decreased thickness, which are consistent with the pathology.
However, when trying to correct for multiple comparisons using FDR ("Set Using FDR" button in qdec), the computed threshold is very high (e.g. 4.3 for 20mm smoothing) and, obviously, no significant regions are left.
Did we do anything wrong in the analysis ?
Thank you very much for your help, Yulia
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer