Interesting post Donna, but my understanding of FDR is that it sets the p-value threshold based on the LARGEST p-value that satisfies the FDR relationship.

That is, steps 3 and 4 in Genovese et al. (2002) are: 3) Let r be the largest i for which p <= i/V*q (assuming c=1) 4) Threshold the image at the p-value p(r).

So, it isn't the case that you require the most significant p-value to satisfy p <= 0.05/V "just to get past i=1" as you put it in your post.

Rather, you pick the largest p-value that satisfies the relationship, meaning that lower (more-significant) p-values may not have necessarily satisfied p <= i/V*q for their particular position in the sorted list of p-values.

cheers, Mike H.

On Fri, 2009-10-16 at 10:13 -0500, Donna Dierker wrote:

I never heard anything on my post here, but it might just be high surface resolution:

http://www.mail-archive.com/neuro-mult-comp@brainvis.wustl.edu/msg00026.html

On 10/16/2009 09:58 AM, Michael Harms wrote:

...

Your FDR analysis sounds correct. You probably have a rather small number of "marginally" significant vertices, which is why none survive FDR. You could try increasing the "q" value from say 0.05 to 0.1, in which case 10% of the surviving vertices would be expected to be false positives.

cheers, Mike H.

On Fri, 2009-10-16 at 12:03 +0200, Yulia WORBE wrote:

...

Dear Freesurfer team,

We are currently doing a cortical thickness studies between a group of psychiatric patients (n=60) and controls (n=30). We tested several smoothing levels (15mm, 20mm, 25mm)

When setting an uncorrected threshold (such as p<0.005), we obtained several regions of decreased thickness, which are consistent with the pathology.

However, when trying to correct for multiple comparisons using FDR ("Set Using FDR" button in qdec), the computed threshold is very high (e.g. 4.3 for 20mm smoothing) and, obviously, no significant regions are left.

Did we do anything wrong in the analysis ?

Thank you very much for your help, Yulia

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Good point (meaning that I don't know the answer). I'll see if I can find out.

doug

On Fri, 16 Oct 2009, Donna Dierker wrote:

Regardless: FDR's sensitivity appears resolution-dependent to me.

On 10/16/2009 10:39 AM, Michael Harms wrote:

...

Interesting post Donna, but my understanding of FDR is that it sets the p-value threshold based on the LARGEST p-value that satisfies the FDR relationship.

That is, steps 3 and 4 in Genovese et al. (2002) are: 3) Let r be the largest i for which p <= i/V*q (assuming c=1) 4) Threshold the image at the p-value p(r).

So, it isn't the case that you require the most significant p-value to satisfy p <= 0.05/V "just to get past i=1" as you put it in your post.

Rather, you pick the largest p-value that satisfies the relationship, meaning that lower (more-significant) p-values may not have necessarily satisfied p <= i/V*q for their particular position in the sorted list of p-values.

cheers, Mike H.

On Fri, 2009-10-16 at 10:13 -0500, Donna Dierker wrote:

...

I never heard anything on my post here, but it might just be high surface resolution:

http://www.mail-archive.com/neuro-mult-comp@brainvis.wustl.edu/msg00026.html

On 10/16/2009 09:58 AM, Michael Harms wrote:

...

Your FDR analysis sounds correct. You probably have a rather small number of "marginally" significant vertices, which is why none survive FDR. You could try increasing the "q" value from say 0.05 to 0.1, in which case 10% of the surviving vertices would be expected to be false positives.

cheers, Mike H.

On Fri, 2009-10-16 at 12:03 +0200, Yulia WORBE wrote:

...

Dear Freesurfer team,

We are currently doing a cortical thickness studies between a group of psychiatric patients (n=60) and controls (n=30). We tested several smoothing levels (15mm, 20mm, 25mm)

When setting an uncorrected threshold (such as p<0.005), we obtained several regions of decreased thickness, which are consistent with the pathology.

However, when trying to correct for multiple comparisons using FDR ("Set Using FDR" button in qdec), the computed threshold is very high (e.g. 4.3 for 20mm smoothing) and, obviously, no significant regions are left.

Did we do anything wrong in the analysis ?

Thank you very much for your help, Yulia

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

---

Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

Dear all,

The B&H procedure is dependent on the number of tests and, for neuroimaging methods, more tests is often equivalent to higher resolution, either in voxel-based or surface-based representations of the brain. The question that arises is whether this would influence sensitivity and/or power. As long as changes in resolution are uniform in space (i.e. uniform lattice for voxel-based methods or homogeneous density of vertices for surface-based methods), then the resolution, by itself, should not interfere in the sensitivity or power. The proportions of errors type I and II should remain the same. There are just more points to look at. Note that the maximum p-value to obtain at least one rejection might change, but 1 may be 1% of 100 tests or 0.001% of 100000 tests, so the absolute number of rejections would not be a valid way to control the proportion of false discoveries.

However, this is not the end of the story. The noise and the size of the effect (in terms of area/volume) influence the p-values, hence the resulting threshold. In other words: while the *uniformity* of the resampling is important to ensure that the threshold would remain the same in terms of p-values, the noise and size of the "activation" also influence. Changes in resolution have the same overall effect of filtering in space. Gaussian filters, for instance, are known for highlighting signal areas which size matches the width of the filter, while burying into noise spatially small regions of signal. Reductions in resolution have an effect similar to average (mean) filtering for voxel-based methods, and to whatever equivalent for convolution would be defined for surface-based methods.

Therefore, I would conclude that it is not an FDR issue, but instead related to the ability to discriminate signal from noise in neighbouring voxels/vertices in different resolutions. While mean filtering (equivalent to lowering the resolution) may cancel out noise, increasing the value of the statistic, it may also dilute a small effect, causing the opposite result. For methods where the intensity of the voxel/vertex is the variable of interest, the "best" resolution should be no higher than what can be afforded by the device (unless taken into account somehow). For voxel-based methods (say, fMRI, PET), this is easy to achieve. For surface-based fMRI, for instance, it is certainly more complex and related to how the information from a certain voxel from MRI can be split or projected into more than one vertex. For surface-based cortical thickness, there might not be a "best" resolution, but to what concerns FDR and other multiple testing procedures, a roughly homogeneous density of vertices on space might be a desirable feature.

With respect to binning, although subsampling the vertices uniformly in space or selecting them after sorting should produce different thresholds, if the distribution of p-values is well behaved (i.e. no discrete distribution, uniform under null, etc) and if the bins are not too wide, then this difference should be negligible for practical purposes.

Hope this helps!

Anderson