Dear FreeSurfers and LME experts,
I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps:
1) Following the wiki, I've created a pre-design matrix, M, with: First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition). Second column: Group factor binary coded. Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don't know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models.
From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions).
2) I then use lme_mass_fit_vw, but I have doubts about some inputs Zcols: column 1 in X ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition) I kept the others as default.
3) lme_mass_F for statistics of fixed effects. I check main effects and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest.
4) lme_mass_FDR2 for correcting the F stats for multiple comparisons. I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion?
Could you please let me know if this procedure is sound? Also, if I'm interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X?
I hope I haven't overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide.
Best regards,
Laura Rueda
Hi Laura,
1) you would need to model the 3 level variable differently, as 1,2,3 will be understood as continuous and that is not what you want. Instead you have two columns: one column for group 2, where all instances are binary (1 if group is 2, else 0), and one for group 3. Then the intercept and slope will be for group 1 and these columns contain the offsets for group 2 and 3 respectively. About interactions, make sure you really want to model all interactions. Some may be not very meaningful and keeping the model simple is usually a good idea. But if you want them all, add them all.
2) Zcols: Vector with the indices of the colums of X that will be considered as random effects. Usually the intercept is a random effect and maybe other variables (e.g. the time, if you want to allow slopes to be different across subjects). For just the intercept column use [ 1] ni should be correct
3) yes, but with the design above, since you model a global intercept and then the offset of each group (or whatever that three condition variable is) you need to make sure you interpret things correctly. E.g. a 1 in such a column for group 2 or three indicates that there is a difference from the first group. This is not the group 2 effect.
4) Sorry, I don't know. Maybe someone else has a suggest.
Since you have all interactions, you should be able to specify contrast according to your test in this model and don't need to create a new one. Your setup is a little complicated, so it would be wise to involve a local statistician to make sure you are interpreting things correctly.
Best, Martin
On 04/07/2016 10:57 AM, Laura Rueda Delgado wrote:
Dear FreeSurfers and LME experts,
I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps:
- Following the wiki, I've created a pre-design matrix, M, with:
First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition).
Second column: Group factor binary coded.
Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don’t know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models.
From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions).
- I then use lme_mass_fit_vw, but I have doubts about some inputs
Zcols: column 1 in X
ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition)
I kept the others as default.
- lme_mass_F for statistics of fixed effects. I check main effects
and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest.
- lme_mass_FDR2 for correcting the F stats for multiple comparisons.
I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion?
Could you please let me know if this procedure is sound? Also, if I’m interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X?
I hope I haven’t overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide.
Best regards,
*Laura Rueda*
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
Dear Martin,
Thank you for your quick response.
My research question is focused on the group and day effect, so I could simplify the model. From your response, I thought of including in the design matrix the binary code for Condition 2, and for Condition 3 in two columns, like you suggested; however, adding only the interaction effect of group and day. This way, the model takes into account the repeated measures of Condition while remaining relatively simple.
Now, to add more complexity and be more specific, I'm interested on neural correlates of learning, which is (un)fortunately tangled with performance. Therefore, I was thinking to add the learning gains (e.g. [performance_day_2 - performance_day_1]/performance_day_1) as a fixed effect. Checking the example datasets that come with the LME toolbox, I see that other measures (e.g. age, gender) are repeated for all the measures at different days of the same subject. And, when the age of participants at the first measurement is used as a fixed effect, "baseline age", the value is repeated for the baseline day and the other days. I was wondering if this could be applied for the Learning Gain as well (which is a measure of change of performance over the training days). I have my doubts because I would understand that this type of coding would describe two mixed aspects: a "prediction" of Gain from the brain data at Day 1, along with the appropriate relationship of Gain with brain data at Day 2. I would think that adding 0s for Day 1 in the variable Gain would be more appropriate, and this way the fixed effect of Day would be coded in the variable Gain. But of course, I recur to the experts to settle this issue.
Additionally, I see that I could compare two models with different number of random effects with lme_mass_LR. However, I would like to compare models with different covariates (other covariates of no interest, like gender and years of education). Is this possible within this toolbox?
Best regards,
Laura Rueda
From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer-bounces@nmr.mgh.harvard.edu] On Behalf Of Martin Reuter Sent: donderdag 7 april 2016 18:20 To: freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] LME for functional data, 3 factors
Hi Laura,
1) you would need to model the 3 level variable differently, as 1,2,3 will be understood as continuous and that is not what you want. Instead you have two columns: one column for group 2, where all instances are binary (1 if group is 2, else 0), and one for group 3. Then the intercept and slope will be for group 1 and these columns contain the offsets for group 2 and 3 respectively. About interactions, make sure you really want to model all interactions. Some may be not very meaningful and keeping the model simple is usually a good idea. But if you want them all, add them all.
2) Zcols: Vector with the indices of the colums of X that will be considered as random effects. Usually the intercept is a random effect and maybe other variables (e.g. the time, if you want to allow slopes to be different across subjects). For just the intercept column use [ 1] ni should be correct
3) yes, but with the design above, since you model a global intercept and then the offset of each group (or whatever that three condition variable is) you need to make sure you interpret things correctly. E.g. a 1 in such a column for group 2 or three indicates that there is a difference from the first group. This is not the group 2 effect.
4) Sorry, I don't know. Maybe someone else has a suggest.
Since you have all interactions, you should be able to specify contrast according to your test in this model and don't need to create a new one. Your setup is a little complicated, so it would be wise to involve a local statistician to make sure you are interpreting things correctly.
Best, Martin
On 04/07/2016 10:57 AM, Laura Rueda Delgado wrote: Dear FreeSurfers and LME experts,
I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps:
1) Following the wiki, I've created a pre-design matrix, M, with: First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition). Second column: Group factor binary coded. Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don't know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models.
From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions).
2) I then use lme_mass_fit_vw, but I have doubts about some inputs Zcols: column 1 in X ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition) I kept the others as default.
3) lme_mass_F for statistics of fixed effects. I check main effects and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest.
4) lme_mass_FDR2 for correcting the F stats for multiple comparisons. I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion?
Could you please let me know if this procedure is sound? Also, if I'm interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X?
I hope I haven't overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide.
Best regards,
Laura Rueda
_______________________________________________
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edumailto:Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
--
Martin Reuter, PhD
Assistant Professor of Radiology, Harvard Medical School
Assistant Professor of Neurology, Harvard Medical School
A.A.Martinos Center for Biomedical Imaging
Massachusetts General Hospital
Research Affiliate, CSAIL, MIT
Phone: +1-617-724-5652
Web : http://reuter.mit.edu
Hi Laura In the wikihttps://surfer.nmr.mgh.harvard.edu/fswiki/LinearMixedEffectsModelswe recommended to order the columns of the design matrix in thefollowing way: Column 1: theintercept term (which is a column of ones) Colum 2: the timecovariate if it varies across subjects (eg. time from baseline) Column 3-q: anytime-varying covariates (eg. training: 0 before training, 1 aftertraining) Column q+1-r: thegroup covariates of interest (eg. a binary variable indicatingwhether or not the subject is a patient), for n groups you will haven-1 binary covariates Column r+1-s:interactions between group covariates with the time-varyingcovariates (only the interesting interactions) Column s+1-p: anyother nuisance time-invariant covariates (eg. age-at-baseline, gender,etc...)
LME is a type oflinear regression model that integrates both a model for the mean anda model for the covariance into a single statistical model. So ifyou want to compare different models for the mean (with differentcovariates) then you need to start with a “maximal model for themean” that includes all the possible covariates of interest. Then you selectthe random effects for that "maximal model" using the model selectionprocedure with lme_mass_LR. Random effects can only be time-varyingcovariates (i.e a subset of the columns from 1 to q above, thosecomprise the Zcols parameter in lme_mass_fit_vw) After you choosewhich time-varying covariates are going to be considered as randomeffects then you can test if any single fixed effects covariate in your designmatrix has a significant contribution to the model in the same wayyou would for a traditional GLM. You will use F-tests for that.Covariates that do not significantly contribute to the model can beruled out of the model and a new model with less covariates but the same random effects can thenbe fitted. Keep in mind thatfitting lme models is computationally expensive and the computationoverhead quickly increases with the number of random effects and fixedeffects in your model. Also considering one or two random effects including the intercept term (or atmost three) is usually enough but this really depends on the natureof the data.
Best-Jorge
De: Laura Rueda Delgado laura.ruedadelgado@kuleuven.be Para: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Enviado: Lunes 11 de abril de 2016 13:05 Asunto: Re: [Freesurfer] LME for functional data, 3 factors
#yiv9346679664 #yiv9346679664 -- _filtered #yiv9346679664 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv9346679664 {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;} _filtered #yiv9346679664 {font-family:Consolas;panose-1:2 11 6 9 2 2 4 3 2 4;}#yiv9346679664 #yiv9346679664 p.yiv9346679664MsoNormal, #yiv9346679664 li.yiv9346679664MsoNormal, #yiv9346679664 div.yiv9346679664MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;color:black;}#yiv9346679664 a:link, #yiv9346679664 span.yiv9346679664MsoHyperlink {color:blue;text-decoration:underline;}#yiv9346679664 a:visited, #yiv9346679664 span.yiv9346679664MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv9346679664 pre {margin:0cm;margin-bottom:.0001pt;font-size:10.0pt;color:black;}#yiv9346679664 span.yiv9346679664EmailStyle17 {color:windowtext;}#yiv9346679664 span.yiv9346679664HTMLPreformattedChar {font-family:Consolas;color:black;}#yiv9346679664 span.yiv9346679664EmailStyle20 {color:#1F497D;}#yiv9346679664 .yiv9346679664MsoChpDefault {font-size:10.0pt;} _filtered #yiv9346679664 {margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv9346679664 div.yiv9346679664WordSection1 {}#yiv9346679664 Dear Martin, Thank you for your quick response. My research question is focused on the group and day effect, so I could simplify the model. From your response, I thought of including in the design matrix the binary code for Condition 2, and for Condition 3 in two columns, like you suggested; however, adding only the interaction effect of group and day. This way, the model takes into account the repeated measures of Condition while remaining relatively simple. Now, to add more complexity and be more specific, I'm interested on neural correlates of learning, which is (un)fortunately tangled with performance. Therefore, I was thinking to add the learning gains (e.g. [performance_day_2 - performance_day_1]/performance_day_1) as a fixed effect. Checking the example datasets that come with the LME toolbox, I see that other measures (e.g. age, gender) are repeated for all the measures at different days of the same subject. And, when the age of participants at the first measurement is used as a fixed effect, "baseline age", the value is repeated for the baseline day and the other days. I was wondering if this could be applied for the Learning Gain as well (which is a measure of change of performance over the training days). I have my doubts because I would understand that this type of coding would describe two mixed aspects: a “prediction” of Gain from the brain data at Day 1, along with the appropriate relationship of Gain with brain data at Day 2. I would think that adding 0s for Day 1 in the variable Gain would be more appropriate, and this way the fixed effect of Day would be coded in the variable Gain. But of course, I recur to the experts to settle this issue. Additionally, I see that I could compare two models with different number of random effects with lme_mass_LR. However, I would like to compare models with different covariates (other covariates of no interest, like gender and years of education). Is this possible within this toolbox? Best regards, Laura Rueda From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer-bounces@nmr.mgh.harvard.edu]On Behalf Of Martin Reuter Sent: donderdag 7 april 2016 18:20 To: freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] LME for functional data, 3 factors Hi Laura,
1) you would need to model the 3 level variable differently, as 1,2,3 will be understood as continuous and that is not what you want. Instead you have two columns: one column for group 2, where all instances are binary (1 if group is 2, else 0), and one for group 3. Then the intercept and slope will be for group 1 and these columns contain the offsets for group 2 and 3 respectively. About interactions, make sure you really want to model all interactions. Some may be not very meaningful and keeping the model simple is usually a good idea. But if you want them all, add them all.
2) Zcols: Vector with the indices of the colums of X that will be considered as random effects. Usually the intercept is a random effect and maybe other variables (e.g. the time, if you want to allow slopes to be different across subjects). For just the intercept column use [ 1] ni should be correct
3) yes, but with the design above, since you model a global intercept and then the offset of each group (or whatever that three condition variable is) you need to make sure you interpret things correctly. E.g. a 1 in such a column for group 2 or three indicates that there is a difference from the first group. This is not the group 2 effect.
4) Sorry, I don't know. Maybe someone else has a suggest.
Since you have all interactions, you should be able to specify contrast according to your test in this model and don't need to create a new one. Your setup is a little complicated, so it would be wise to involve a local statistician to make sure you are interpreting things correctly.
Best, Martin
On 04/07/2016 10:57 AM, Laura Rueda Delgado wrote: Dear FreeSurfers and LME experts, I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps: 1) Following the wiki, I've created a pre-design matrix, M, with: First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition). Second column: Group factor binary coded. Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don’t know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models. From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions). 2) I then use lme_mass_fit_vw, but I have doubts about some inputs Zcols: column 1 in X ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition) I kept the others as default. 3) lme_mass_F for statistics of fixed effects. I check main effects and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest. 4) lme_mass_FDR2 for correcting the F stats for multiple comparisons. I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion? Could you please let me know if this procedure is sound? Also, if I’m interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X? I hope I haven’t overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide. Best regards, Laura Rueda
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
-- Martin Reuter, PhD Assistant Professor of Radiology, Harvard Medical School Assistant Professor of Neurology, Harvard Medical School A.A.Martinos Center for Biomedical Imaging Massachusetts General Hospital Research Affiliate, CSAIL, MIT Phone: +1-617-724-5652 Web : http://reuter.mit.edu _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Dear Jorge,
Thank you for your explanation. I’ve done what you suggested, starting with the maximal model for the mean including all covariates of interest and no interest. And then used F-tests for checking the significance of the covariates of no interest. Now I have three questions:
- Since I’m fitting a mass univariate model, these F tests of covariates of no interest can result in a few voxels being significant (e.g. 100) or in many voxels (>1.000) (with a significance threshold of 0.05). Is the final decision regarding those covariates with low number of significant voxels discretionary?
- I’ve also started exploring different models with some modification of a covariate which could answer my question more directly. For choosing between two different models with equal random effects, on a previous thread it was recommended “to choose the one with higher maximum likelihood after model fitting” (https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2014-September/040464...https://mail.nmr.mgh.harvard.edu/pipermail/freesurfer/2014-September/040464.html). Is the maximum likelihood the ‘lreml’ field in the model structure (output of lme_mass_fit_vw)? Does this penalize for number of covariates?
- Related to the previous question, how do I check if the model is over-fitting the data? My sample size is not very big, so I’m reconsidering the inclusion of covariates of no interest.
Thank you in advance for any suggestion!
Laura Rueda Delgado Doctoral researcher Department of Kinesiology- Motor Control and Neural Plasticity Research Group KU Leuven Tervuursevest 101 bus 1501 3001 Leuven, Belgium tel. +32 16 37 64 78
From: jorge luis [mailto:jbernal0019@yahoo.es] Sent: maandag 11 april 2016 20:34 To: Freesurfer support list Cc: Laura Rueda Delgado Subject: Re: [Freesurfer] LME for functional data, 3 factors
Hi Laura In the wiki https://surfer.nmr.mgh.harvard.edu/fswiki/LinearMixedEffectsModels we recommended to order the columns of the design matrix in the following way:
Column 1: the intercept term (which is a column of ones)
Colum 2: the time covariate if it varies across subjects (eg. time from baseline)
Column 3-q: any time-varying covariates (eg. training: 0 before training, 1 after training)
Column q+1-r: the group covariates of interest (eg. a binary variable indicating whether or not the subject is a patient), for n groups you will have n-1 binary covariates
Column r+1-s: interactions between group covariates with the time-varying covariates (only the interesting interactions)
Column s+1-p: any other nuisance time-invariant covariates (eg. age-at-baseline, gender, etc...)
LME is a type of linear regression model that integrates both a model for the mean and a model for the covariance into a single statistical model. So if you want to compare different models for the mean (with different covariates) then you need to start with a “maximal model for the mean” that includes all the possible covariates of interest. Then you select the random effects for that "maximal model" using the model selection procedure with lme_mass_LR. Random effects can only be time-varying covariates (i.e a subset of the columns from 1 to q above, those comprise the Zcols parameter in lme_mass_fit_vw)
After you choose which time-varying covariates are going to be considered as random effects then you can test if any single fixed effects covariate in your design matrix has a significant contribution to the model in the same way you would for a traditional GLM. You will use F-tests for that. Covariates that do not significantly contribute to the model can be ruled out of the model and a new model with less covariates but the same random effects can then be fitted.
Keep in mind that fitting lme models is computationally expensive and the computation overhead quickly increases with the number of random effects and fixed effects in your model. Also considering one or two random effects including the intercept term (or at most three) is usually enough but this really depends on the nature of the data.
Best -Jorge
________________________________ De: Laura Rueda Delgado laura.ruedadelgado@kuleuven.be Para: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Enviado: Lunes 11 de abril de 2016 13:05 Asunto: Re: [Freesurfer] LME for functional data, 3 factors
Dear Martin,
Thank you for your quick response.
My research question is focused on the group and day effect, so I could simplify the model. From your response, I thought of including in the design matrix the binary code for Condition 2, and for Condition 3 in two columns, like you suggested; however, adding only the interaction effect of group and day. This way, the model takes into account the repeated measures of Condition while remaining relatively simple.
Now, to add more complexity and be more specific, I'm interested on neural correlates of learning, which is (un)fortunately tangled with performance. Therefore, I was thinking to add the learning gains (e.g. [performance_day_2 - performance_day_1]/performance_day_1) as a fixed effect. Checking the example datasets that come with the LME toolbox, I see that other measures (e.g. age, gender) are repeated for all the measures at different days of the same subject. And, when the age of participants at the first measurement is used as a fixed effect, "baseline age", the value is repeated for the baseline day and the other days. I was wondering if this could be applied for the Learning Gain as well (which is a measure of change of performance over the training days). I have my doubts because I would understand that this type of coding would describe two mixed aspects: a “prediction” of Gain from the brain data at Day 1, along with the appropriate relationship of Gain with brain data at Day 2. I would think that adding 0s for Day 1 in the variable Gain would be more appropriate, and this way the fixed effect of Day would be coded in the variable Gain. But of course, I recur to the experts to settle this issue.
Additionally, I see that I could compare two models with different number of random effects with lme_mass_LR. However, I would like to compare models with different covariates (other covariates of no interest, like gender and years of education). Is this possible within this toolbox?
Best regards,
Laura Rueda
From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer-bounces@nmr.mgh.harvard.edu] On Behalf Of Martin Reuter Sent: donderdag 7 april 2016 18:20 To: freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] LME for functional data, 3 factors
Hi Laura,
1) you would need to model the 3 level variable differently, as 1,2,3 will be understood as continuous and that is not what you want. Instead you have two columns: one column for group 2, where all instances are binary (1 if group is 2, else 0), and one for group 3. Then the intercept and slope will be for group 1 and these columns contain the offsets for group 2 and 3 respectively. About interactions, make sure you really want to model all interactions. Some may be not very meaningful and keeping the model simple is usually a good idea. But if you want them all, add them all.
2) Zcols: Vector with the indices of the colums of X that will be considered as random effects. Usually the intercept is a random effect and maybe other variables (e.g. the time, if you want to allow slopes to be different across subjects). For just the intercept column use [ 1] ni should be correct
3) yes, but with the design above, since you model a global intercept and then the offset of each group (or whatever that three condition variable is) you need to make sure you interpret things correctly. E.g. a 1 in such a column for group 2 or three indicates that there is a difference from the first group. This is not the group 2 effect.
4) Sorry, I don't know. Maybe someone else has a suggest.
Since you have all interactions, you should be able to specify contrast according to your test in this model and don't need to create a new one. Your setup is a little complicated, so it would be wise to involve a local statistician to make sure you are interpreting things correctly.
Best, Martin
On 04/07/2016 10:57 AM, Laura Rueda Delgado wrote: Dear FreeSurfers and LME experts,
I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps:
1) Following the wiki, I've created a pre-design matrix, M, with: First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition). Second column: Group factor binary coded. Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don’t know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models. From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions).
2) I then use lme_mass_fit_vw, but I have doubts about some inputs Zcols: column 1 in X ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition) I kept the others as default.
3) lme_mass_F for statistics of fixed effects. I check main effects and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest.
4) lme_mass_FDR2 for correcting the F stats for multiple comparisons. I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion?
Could you please let me know if this procedure is sound? Also, if I’m interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X?
I hope I haven’t overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide.
Best regards,
Laura Rueda
_______________________________________________
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edumailto:Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
--
Martin Reuter, PhD
Assistant Professor of Radiology, Harvard Medical School
Assistant Professor of Neurology, Harvard Medical School
A.A.Martinos Center for Biomedical Imaging
Massachusetts General Hospital
Research Affiliate, CSAIL, MIT
Phone: +1-617-724-5652
Web : http://reuter.mit.eduhttp://reuter.mit.edu/
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edumailto:Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Hi Laura
-Before making anydecision please make sure you correct for multiple comparisons usinglme_mass_FDR2. That would reduce the likelihood of observing falsepositives. If very few voxels survived in random unexpected regionsafter the correction then you can decide to drop that covariate fromthe model. -Yes the "lreml" fieldis the maximum likelihood estimate but you can only use it to comparemodels with exactly the same covariates and the same number butdifferent subsets of random effects. If you modify the values of onecovariate then you can not compare the modified model with theprevious model just on the basis of their maximum likelihoodestimates. -Well that isdifficult to know but the more statistical tests you do the morechances you have of observing false positives. Most of the timeresearchers have a-priori hypotheses about covariates and brainregions of interests.
Best-Jorge
De: Laura Rueda Delgado laura.ruedadelgado@kuleuven.be Para: jorge luis jbernal0019@yahoo.es; Freesurfer support list freesurfer@nmr.mgh.harvard.edu Enviado: Jueves 14 de abril de 2016 11:23 Asunto: Re: [Freesurfer] LME for functional data, 3 factors
#yiv6954010121 #yiv6954010121 -- _filtered #yiv6954010121 {font-family:Helvetica;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv6954010121 {font-family:Wingdings;panose-1:5 0 0 0 0 0 0 0 0 0;} _filtered #yiv6954010121 {font-family:Wingdings;panose-1:5 0 0 0 0 0 0 0 0 0;} _filtered #yiv6954010121 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv6954010121 {font-family:Tahoma;panose-1:2 11 6 4 3 5 4 4 2 4;} _filtered #yiv6954010121 {font-family:Consolas;panose-1:2 11 6 9 2 2 4 3 2 4;}#yiv6954010121 #yiv6954010121 p.yiv6954010121MsoNormal, #yiv6954010121 li.yiv6954010121MsoNormal, #yiv6954010121 div.yiv6954010121MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:12.0pt;}#yiv6954010121 a:link, #yiv6954010121 span.yiv6954010121MsoHyperlink {color:blue;text-decoration:underline;}#yiv6954010121 a:visited, #yiv6954010121 span.yiv6954010121MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv6954010121 pre {margin:0cm;margin-bottom:.0001pt;font-size:10.0pt;}#yiv6954010121 tt {}#yiv6954010121 p.yiv6954010121MsoAcetate, #yiv6954010121 li.yiv6954010121MsoAcetate, #yiv6954010121 div.yiv6954010121MsoAcetate {margin:0cm;margin-bottom:.0001pt;font-size:8.0pt;}#yiv6954010121 p.yiv6954010121MsoListParagraph, #yiv6954010121 li.yiv6954010121MsoListParagraph, #yiv6954010121 div.yiv6954010121MsoListParagraph {margin-top:0cm;margin-right:0cm;margin-bottom:0cm;margin-left:36.0pt;margin-bottom:.0001pt;font-size:12.0pt;}#yiv6954010121 span.yiv6954010121HTMLPreformattedChar {font-family:Consolas;}#yiv6954010121 p.yiv6954010121msonormal, #yiv6954010121 li.yiv6954010121msonormal, #yiv6954010121 div.yiv6954010121msonormal {margin-right:0cm;margin-left:0cm;font-size:12.0pt;}#yiv6954010121 p.yiv6954010121msochpdefault, #yiv6954010121 li.yiv6954010121msochpdefault, #yiv6954010121 div.yiv6954010121msochpdefault {margin-right:0cm;margin-left:0cm;font-size:12.0pt;}#yiv6954010121 span.yiv6954010121msohyperlink {}#yiv6954010121 span.yiv6954010121msohyperlinkfollowed {}#yiv6954010121 span.yiv6954010121emailstyle17 {}#yiv6954010121 span.yiv6954010121htmlpreformattedchar {}#yiv6954010121 span.yiv6954010121emailstyle20 {}#yiv6954010121 p.yiv6954010121msonormal1, #yiv6954010121 li.yiv6954010121msonormal1, #yiv6954010121 div.yiv6954010121msonormal1 {margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;color:black;}#yiv6954010121 span.yiv6954010121msohyperlink1 {color:blue;text-decoration:underline;}#yiv6954010121 span.yiv6954010121msohyperlinkfollowed1 {color:purple;text-decoration:underline;}#yiv6954010121 span.yiv6954010121emailstyle171 {color:windowtext;}#yiv6954010121 span.yiv6954010121htmlpreformattedchar1 {font-family:Consolas;color:black;}#yiv6954010121 span.yiv6954010121emailstyle201 {color:#1F497D;}#yiv6954010121 p.yiv6954010121msochpdefault1, #yiv6954010121 li.yiv6954010121msochpdefault1, #yiv6954010121 div.yiv6954010121msochpdefault1 {margin-right:0cm;margin-left:0cm;font-size:10.0pt;}#yiv6954010121 span.yiv6954010121EmailStyle34 {color:#1F497D;}#yiv6954010121 span.yiv6954010121BalloonTextChar {}#yiv6954010121 .yiv6954010121MsoChpDefault {font-size:10.0pt;} _filtered #yiv6954010121 {margin:70.85pt 70.85pt 70.85pt 70.85pt;}#yiv6954010121 div.yiv6954010121WordSection1 {}#yiv6954010121 _filtered #yiv6954010121 {} _filtered #yiv6954010121 {} _filtered #yiv6954010121 {} _filtered #yiv6954010121 {font-family:Wingdings;} _filtered #yiv6954010121 {font-family:Symbol;} _filtered #yiv6954010121 {} _filtered #yiv6954010121 {font-family:Wingdings;} _filtered #yiv6954010121 {font-family:Symbol;} _filtered #yiv6954010121 {} _filtered #yiv6954010121 {font-family:Wingdings;}#yiv6954010121 ol {margin-bottom:0cm;}#yiv6954010121 ul {margin-bottom:0cm;}#yiv6954010121 Dear Jorge, Thank you for your explanation. I’ve done what you suggested, starting with the maximal model for the mean including all covariates of interest and no interest. And then used F-tests for checking the significance of the covariates of no interest. Now I have three questions: - Since I’m fitting a mass univariate model, these F tests of covariates of no interest can result in a few voxels being significant (e.g. 100) or in many voxels (>1.000) (with a significance threshold of 0.05). Is the final decision regarding those covariates with low number of significant voxels discretionary? - I’ve also started exploring different models with some modification of a covariate which could answer my question more directly. For choosing between two different models with equal random effects, on a previous thread it was recommended “to choose the one with higher maximum likelihood after model fitting” (https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2014-September/040464...). Is the maximum likelihood the ‘lreml’ field in the model structure (output of lme_mass_fit_vw)? Does this penalize for number of covariates? - Related to the previous question, how do I check if the model is over-fitting the data? My sample size is not very big, so I’m reconsidering the inclusion of covariates of no interest. Thank you in advance for any suggestion! Laura Rueda Delgado Doctoral researcher Department of Kinesiology- Motor Control and Neural Plasticity Research Group KU Leuven Tervuursevest 101 bus 1501 3001 Leuven, Belgium tel. +32 16 37 64 78
From: jorge luis [mailto:jbernal0019@yahoo.es] Sent: maandag 11 april 2016 20:34 To: Freesurfer support list Cc: Laura Rueda Delgado Subject: Re: [Freesurfer] LME for functional data, 3 factors Hi Laura In the wikihttps://surfer.nmr.mgh.harvard.edu/fswiki/LinearMixedEffectsModels we recommended to order the columns of the design matrix in the following way: Column 1: the intercept term (which is a column of ones) Colum 2: the time covariate if it varies across subjects (eg. time from baseline) Column 3-q: any time-varying covariates (eg. training: 0 before training, 1 after training) Column q+1-r: the group covariates of interest (eg. a binary variable indicating whether or not the subject is a patient), for n groups you will have n-1 binary covariates Column r+1-s: interactions between group covariates with the time-varying covariates (only the interesting interactions) Column s+1-p: any other nuisance time-invariant covariates (eg. age-at-baseline, gender, etc...) LME is a type of linear regression model that integrates both a model for the mean and a model for the covariance into a single statistical model. So if you want to compare different models for the mean (with different covariates) then you need to start with a “maximal model for the mean” that includes all the possible covariates of interest. Then you select the random effects for that "maximal model" using the model selection procedure with lme_mass_LR. Random effects can only be time-varying covariates (i.e a subset of the columns from 1 to q above, those comprise the Zcols parameter inlme_mass_fit_vw) After you choose which time-varying covariates are going to be considered as random effects then you can test if any single fixed effects covariate in your design matrix has a significant contribution to the model in the same way you would for a traditional GLM. You will use F-tests for that. Covariates that do not significantly contribute to the model can be ruled out of the model and a new model with less covariates but the same random effects can then be fitted. Keep in mind that fitting lme models is computationally expensive and the computation overhead quickly increases with the number of random effects and fixed effects in your model. Also considering one or two random effects including the intercept term (or at most three) is usually enough but this really depends on the nature of the data. Best -Jorge De: Laura Rueda Delgado laura.ruedadelgado@kuleuven.be Para: Freesurfer support list freesurfer@nmr.mgh.harvard.edu Enviado: Lunes 11 de abril de 2016 13:05 Asunto: Re: [Freesurfer] LME for functional data, 3 factors Dear Martin, Thank you for your quick response. My research question is focused on the group and day effect, so I could simplify the model. From your response, I thought of including in the design matrix the binary code for Condition 2, and for Condition 3 in two columns, like you suggested; however, adding only the interaction effect of group and day. This way, the model takes into account the repeated measures of Condition while remaining relatively simple. Now, to add more complexity and be more specific, I'm interested on neural correlates of learning, which is (un)fortunately tangled with performance. Therefore, I was thinking to add the learning gains (e.g. [performance_day_2 - performance_day_1]/performance_day_1) as a fixed effect. Checking the example datasets that come with the LME toolbox, I see that other measures (e.g. age, gender) are repeated for all the measures at different days of the same subject. And, when the age of participants at the first measurement is used as a fixed effect, "baseline age", the value is repeated for the baseline day and the other days. I was wondering if this could be applied for the Learning Gain as well (which is a measure of change of performance over the training days). I have my doubts because I would understand that this type of coding would describe two mixed aspects: a “prediction” of Gain from the brain data at Day 1, along with the appropriate relationship of Gain with brain data at Day 2. I would think that adding 0s for Day 1 in the variable Gain would be more appropriate, and this way the fixed effect of Day would be coded in the variable Gain. But of course, I recur to the experts to settle this issue. Additionally, I see that I could compare two models with different number of random effects with lme_mass_LR. However, I would like to compare models with different covariates (other covariates of no interest, like gender and years of education). Is this possible within this toolbox? Best regards, Laura Rueda From: freesurfer-bounces@nmr.mgh.harvard.edu [mailto:freesurfer-bounces@nmr.mgh.harvard.edu] On Behalf Of Martin Reuter Sent: donderdag 7 april 2016 18:20 To: freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] LME for functional data, 3 factors Hi Laura,
1) you would need to model the 3 level variable differently, as 1,2,3 will be understood as continuous and that is not what you want. Instead you have two columns: one column for group 2, where all instances are binary (1 if group is 2, else 0), and one for group 3. Then the intercept and slope will be for group 1 and these columns contain the offsets for group 2 and 3 respectively. About interactions, make sure you really want to model all interactions. Some may be not very meaningful and keeping the model simple is usually a good idea. But if you want them all, add them all.
2) Zcols: Vector with the indices of the colums of X that will be considered as random effects. Usually the intercept is a random effect and maybe other variables (e.g. the time, if you want to allow slopes to be different across subjects). For just the intercept column use [ 1] ni should be correct
3) yes, but with the design above, since you model a global intercept and then the offset of each group (or whatever that three condition variable is) you need to make sure you interpret things correctly. E.g. a 1 in such a column for group 2 or three indicates that there is a difference from the first group. This is not the group 2 effect.
4) Sorry, I don't know. Maybe someone else has a suggest.
Since you have all interactions, you should be able to specify contrast according to your test in this model and don't need to create a new one. Your setup is a little complicated, so it would be wise to involve a local statistician to make sure you are interpreting things correctly.
Best, Martin
On 04/07/2016 10:57 AM, Laura Rueda Delgado wrote: Dear FreeSurfers and LME experts, I've just started using the LME toolbox by Bernal-Rusiel et al (2012, 2013) in Matlab, apart from FreeSurfer. My experimental design includes one between-subjects factor (group with two levels, 24 vs 22 subjects), and two within-subjects (WS) factors (day with two levels, and condition with three levels). As far as I understand, the LME toolbox can be used for longitudinal data and for investigating modulations of neural activity with behavioral measures. However, it's been difficult for me to set up both the design matrix and the input of the LME functions given three fixed factors in my design (I haven't included behavior yet). So I have a few questions that I hope you can help me answer. I follow these steps: 1) Following the wiki, I've created a pre-design matrix, M, with: First column: Day factor coded as 0 (first day) and 7 (7 days later, as during acquisition). Second column: Group factor binary coded. Third column: Condition factor coded with dummy variables 1 to 3 (three conditions in total). I don’t know if this is correct; I have failed to find in the mailing list any reference to an additional repeated measure besides time in LME models. From here, I've created the design matrix X adding a column of 1's for the intercept, adding the pre design matrix M, and adding columns for every possible interaction by multiplying element-wise the columns of M (including two-way and three-way interactions). 2) I then use lme_mass_fit_vw, but I have doubts about some inputs Zcols: column 1 in X ni: created a vector with 6 for all subjects (in total there are 6 repeated measures, 2 of day and 3 of condition) I kept the others as default. 3) lme_mass_F for statistics of fixed effects. I check main effects and interactions with contrasts based on the design matrix X: placing a 1 in the column position corresponding to each effect of interest. 4) lme_mass_FDR2 for correcting the F stats for multiple comparisons. I would like to compare the results using AlphaSim's Monte-Carlo simulations, however I'm not sure what image to use to estimate the smoothness. Would you have a suggestion? Could you please let me know if this procedure is sound? Also, if I’m interested in specific comparisons or post-hoc tests, e.g. Group 1$Day 1$Condition 3 vs. Group 1$Day 2$Condition 3, would I need a separate model for that? And lastly, for the analysis including the behavioral measures, should I just include them as a forth column in the pre-design matrix M and add the relevant effects in X? I hope I haven’t overwhelmed you with so many questions. I would greatly appreciate any suggestion you can provide. Best regards, Laura Rueda
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer -- Martin Reuter, PhD Assistant Professor of Radiology, Harvard Medical School Assistant Professor of Neurology, Harvard Medical School A.A.Martinos Center for Biomedical Imaging Massachusetts General Hospital Research Affiliate, CSAIL, MIT Phone: +1-617-724-5652 Web : http://reuter.mit.edu _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline. If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
freesurfer@nmr.mgh.harvard.edu
-
jorge luis -
Laura Rueda Delgado -
Martin Reuter