External Email - Use Caution

Thank you! Now I see how permutation works. So given about 100-200 subjects in the normal group, doing 1000 permutations is of no help.

In fact, in my cortical thickness abnormality test, no matter how many times of permutations I take, the original non-permutated data is the only one arrangement I can observe that has a "big" cluster area - all permutated data has no cluster of big continuous area (given the vertex-wise threshold). So, permutations cannot give me a reasonable probabulity distribution of the sampling data, which means I cannot rely on it to reduce False Positives. Very frustrated here. If you have more comments here, please let me know. Thanks.

Best Regards, Xiao

Douglas N. Greve https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=from:%22Douglas+N.+Greve%22 Tue, 04 Aug 2020 08:27:47 -0700 https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=date:20200804

There are actually only N-1 permutations that you can make:

Real matrix: 1 0 0 1 0 1 0 1 0 1

Permutation 1:

0 1 1 0 0 1 0 1 0 1

Permutation 2:

0 1 0 1 1 0 0 1 0 1

On 8/4/2020 10:41 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug,

I am still uncertain how to permutate the data for multiple comparisons. Please help me with a little more details on this. For example, could you give me 2-3 examples of the permutations for the matrix you gave to me?

In addition, if you can explain to me in a non-design-matrix way, that would be great. For example, suppose I have 1 subject to be compared to n subjects in a normal group. The number of comparisons is m. So I have m x (1+n) data in total:

V_10 V_11 , V_12 , V_13 , … V_1n

V_20 V_21 , V_22 , V_23 , … V_2n

…… …… …… ……

V_m0 V_m1 , V_m2 , V_m3 , … V_mn

_

_Could you please explain to me how data should be permutated? Or give me 2-3 examples of the permutations of the above data that could be?

_Thank you a lot!

Xiao

Douglas N. Greve < https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=f...

Thu, 23 Jul 2020 10:54:17 -0700 <

https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=d...

You would have a design matrix with two columns and rows equal to n+1, eg 1 0 0 1 0 1 0 1 0 1 ... n times you would then permute the design matrix

On 7/23/2020 12:06 PM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug,

For the first question, you answered "It is unusual, though it should work". Could you please briefly describe the way FS used to permute (based on my notation v0, v1, ... vn)? Or, the usual way to permute?The way I described seems to be the only way I can think of. Looking forward to your help here. Thanks a lot!

Xiao

Douglas N. Greve

https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=from:%22Douglas+N.+Greve%22Thu, 23 Jul 2020 07:37:13 -0700

https://www.mail-archive.com/search?l=freesurfer@nmr.mgh.harvard.edu&q=date:20200723

On 7/21/2020 11:45 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Hi Doug, For your questions:

1. "Not sure what you mean by in the ROI. Are you trying to permute across space? That generally does not work because the points are not exchangeable across space."

By "In the ROI", I mean for all vertices listed in the label file. Here, I am talking about a test subject and n control subjects, they are all considered in the same reference subject space - fsaverage. A ROI is defined by a label file for the subject fsaverage. So, I am comparing the test subject and control group on the same ROI region, vertex by vertex. I want to permute points in the test subject and points in all subjects in control groups. I am not talking about permuting vertex locations; I am talking about permuting values (thickness in my case) from subjects on each vertex. For example, for vertex i, I have one value (v0) from test subject, and n values (v1, v2,...vn) from control subjects: test subject control subjects v0 v1, v2, ...... vn One way of permutation would be: test subject control subjects v1 v0, v2, ...... vn Is this a reasonable way to do permutation?

It is unusual, though it should work.

2. "Not sure. You cannot discriminate between the groups when you are doing permutation"

By doing the permutation many (say 1000) times, I want to get the probability distribution of the sampling (observed or test) data inside the ROI area, so that I can decide if I should reject or accept null hypothesis based on cluster-wise significance level. What problems do you think I have in this idea?

That should work. I'm not sure what my original concern was

Thanks a lot! On 7/21/2020 1:12 AM, Xiaojiang Yang wrote:

External Email - Use Caution

Dear FS experts,

Instead of using mri_glmfit-sim, I am trying to implement a customized multiple comparison correction algorithm using permutation. Before I implement my own, I want to make sure my permutation idea is correct. So I was looking at how mri_glmfit-sim does the permutation. The link here

http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm has < http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm%C2%A0has

<

http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm%C2%A0has

http://freesurfer.net/fswiki/FsTutorial/MultipleComparisonsV6.0Perm%C2%A0has

a simple description for how to do permutation, but I don't quite understand the 1st step: Permute the design matrix. To me, permute the design matrix means permute the matrix rows here, but still hard to understand why permuting matrix rows does the trick.

This is pretty standard in permutation. I think Tom Nichols has some basic tutorials on how permutations work.

Anyway, I will not use any design matrix in my customized implementation, so it does not matter for now. My problem can be described as follows: if I have a ROI (a label file) on fsaverage, and I have a test subject and a group of control subjects whose thickness values on every vertex in this label are all known. (The test subject and control subjects are all using fsaverage reference space). I want to compare this test subject's thickness within the ROI to a control group of subjects (within the same ROI). This is a multiple-comparison problem, so I want to use permutation to get less FP rate. My question is: How do I permute the test subject's points and control subjects' points in ROI?

Not sure what you mean by in the ROI. Are you trying to permute across space? That generally does not work because the points are not exchangeable across space.

My understanding is that: for each point in the label, I randomly re-assign all (1+n) values from (1+n) subjects to these (n+1) subjects (where n is the number of subjects in control group). And when all points in the label are done, this is only 1 permutation. I will need at least 1000 times of permutation to get the comparison statistics.

Is my understanding right?

Not sure. You cannot discriminate between the groups when you are doing permutation

Thank you! Xiao

_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu <mailto:Freesurfer@nmr.mgh.harvard.edu Freesurfer@nmr.mgh.harvard.edu> <mailto:Freesurfer@nmr.mgh.harvard.edu Freesurfer@nmr.mgh.harvard.edu> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu <mailto:Freesurfer@nmr.mgh.harvard.edu Freesurfer@nmr.mgh.harvard.edu> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

_______________________________________________ Freesurfer mailing listFreesurfer@nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer

_______________________________________________ Freesurfer mailing listFreesurfer@nmr.mgh.harvard.eduhttps://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer