External Email - Use Caution
Hi,
I decided to eliminate de Female vs Male distintion leaving only three groups and the program is giving the same error. I don't know what is happening because I did the analysis without differianciating any groups and it works. So I imagine the problem comes form dividing the population into groups but i don't really understand why.
Thanks, Vicente Ferrer Predoctoral Researcher BCBL http://secure-web.cisco.com/1MeTtiU5upbosDZLvO5efqUkYhjil-_OOdO3ShPfbbWnXYMt...
Legal disclaimer/Aviso legal/Lege-oharra: http://secure-web.cisco.com/11W5WltBGFEvDsbaNKH94aEzdb5XB9OGMZWxLtvJbB4ZtRPs...
----- Original Message ----- From: "Vicente Ferrer" v.ferrer@bcbl.eu To: "Freesurfer support list" freesurfer@nmr.mgh.harvard.edu Sent: Tuesday, November 29, 2022 10:16:53 AM Subject: Re: [Freesurfer] ERROR: matrix is ill-conditioned or badly scaled, condno = 1e+08
External Email - Use Caution
Hi profesor Greve,
I believe I have located the problem, as you will see in the code below it seems one of the groups (FemalePDMCI) has only 7 subjects and (MALEPDMCI) has 13 which means the regression won't have enough DOF. They amount of DOF needed is NregressorsDODS = Nclasses*(Nvariables+1) = 4*(4+1) = 20. Therefore I believe I would have to renounce to divide my subjects in Males and Females. Here is the output, let me know if that was the main issue. Thanks. Vicente, mri_glmfit --y $PRJDIR/FREESURFER_PPMI/stats/$image.nii.gz \
--fsgd $PRJDIR/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd dods \ --C $PRJDIR/FREESURFER_PPMI/stats/contrast_moca_groups.mtx \ --glmdir $PRJDIR/FREESURFER_PPMI/stats/$image.corr_g_glmdir --eres-save --illcond
gdfRead(): reading /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd mri_glmfit-sim --glmdir $PRJDIR/FREESURFER_PPMI/stats/$image.corr_g_glmdir \ --cwp .9 \ INFO: DeMeanFlag keyword not found, DeMeaning will NOT be done. Continuous Variable Means (all subjects) 0 age 0.00273866 1.00533 1 relative_motion 0.0142075 0.995938 2 dis_duration 0.00767562 1.0031 3 moca 0.0168011 0.991508 Class Size and Means of each Continuous Variable 1 MaleHC 35 0.0125 0.2192 -0.6716 0.1872 2 MalePDCN 63 0.0523 -0.0415 0.2829 0.2509 3 MalePDMCI 13 0.4036 0.4899 0.1774 -2.0122 4 FemaleHC 20 -0.2146 -0.0406 -0.6716 0.1562 5 FemalePDCN 37 -0.1703 -0.1990 0.4623 0.4639 6 FemalePDMCI 7 0.2990 -0.1089 0.1496 -1.9348 INFO: gd2mtx_method is dods
7.3.2 cwd /export/home/vferrer cmdline mri_glmfit --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.nii.gz --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd dods --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/contrast_moca_groups.mtx --glmdir /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir --eres-save --illcond sysname Linux hostname ipsnode-7-3.local machine x86_64 user vferrer FixVertexAreaFlag = 1 UseMaskWithSmoothing 1 OneSampleGroupMean 0 y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.nii.gz logyflag 0 usedti 0 FSGD /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd glmdir /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir IllCondOK 1 ReScaleX 1 DoFFx 0 Creating output directory /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir --perm 1000 1.5 abs --overwrite --perm-force --seed 1669778971 Loading y from /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.nii.gz ... done reading. INFO: gd2mtx_method is dods Saving design matrix to /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/Xg.dat Computing normalized matrix Normalized matrix condition is 1e+08 Matrix condition is 3.69944e+08 Pruning voxels by thr: 1.175494e-38 Found 47 voxels in mask Saving mask to /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh search space = 158.625000 DOF = 145 Starting fit and test Fit completed in 0.000783333 minutes Computing spatial AR1 in volume. fMRIspatialAR1(): hit 0 voxels WARNING: no voxels in AR1 computation Residual: ar1mn = (-nan,-nan,-nan) fwhm = (-nan,-nan,-nan) -nan Writing results contrast_moca_groups maxvox sig=0.000253964 F=5.38727e-07 at index 42 0 0 seed=1670287315 Computing efficiency mri_glmfit done [vferrer@ipsnode-7-3 ~]$ [vferrer@ipsnode-7-3 ~]$ mri_glmfit-sim --glmdir $PRJDIR/FREESURFER_PPMI/stats/$image.corr_g_glmdir \
--cwp .9 \ --perm 1000 1.5 abs --overwrite --perm-force --seed 1669778971
cmdline mri_glmfit --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.nii.gz --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd dods --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/contrast_moca_groups.mtx --glmdir /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir --eres-save --illcond mri_info --dim /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh --o /scratch/vferrer/tmp.suxi2S Turning on 1D dim = 47 1 1 1 log file is /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/perm.th15.abs.mri_glmfit-sim.log
cd /export/home/vferrer /opt/freesurfer-7.3.2/bin/mri_glmfit-sim --glmdir /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir --cwp .9 --perm 1000 1.5 abs --overwrite --perm-force --seed 1669778971
mri_glmfit-sim 7.3.2 Tue Nov 29 16:06:54 CET 2022 Linux ipsnode-7-3.local 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux vferrer setenv SUBJECTS_DIR /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI FREESURFER_HOME /opt/freesurfer-7.3.2
Original mri_glmfit command line: cmdline mri_glmfit --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.nii.gz --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/glm_covariates_moca_groups.fsgd dods --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/contrast_moca_groups.mtx --glmdir /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir --eres-save --illcond
DoSim = 1 UseCache = 0 DoPoll = 0 DoPBSubmit = 0 DoBackground = 0 DiagCluster = 0 gd2mtx = dods Seed = 1669778971 fwhm = 0 nSimPerJob = 1000 1/1 Tue Nov 29 16:06:54 CET 2022 mri_glmfit --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/tmp.mri_glmfit-sim-21231/contrast_moca_groups.mtx --sim perm 1000 1.5 /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/csd/perm.th15.abs.j001 --sim-sign abs --fwhm 0 --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd dods --mask /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh --perm-force --seed 1669778971 INFO: ignoring tag Creator INFO: ignoring tag SUBJECTS_DIR INFO: ignoring tag SynthSeed simbase /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/csd/perm.th15.abs.j001 FWHM = 0.000000 gdfRead(): reading /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd INFO: NOT demeaning continuous variables Continuous Variable Means (all subjects) 0 age 0.00273894 1.00533 1 relative_motion 0.0142074 0.995939 2 dis_duration 0.0076754 1.0031 3 moca 0.0168018 0.991509 Class Size and Means of each Continuous Variable 1 MaleHC 35 0.0125 0.2192 -0.6716 0.1872 2 MalePDCN 63 0.0523 -0.0415 0.2829 0.2509 3 MalePDMCI 13 0.4037 0.4899 0.1774 -2.0122 4 FemaleHC 20 -0.2146 -0.0406 -0.6716 0.1562 5 FemalePDCN 37 -0.1703 -0.1990 0.4623 0.4639 6 FemalePDMCI 7 0.2990 -0.1089 0.1496 -1.9348 INFO: gd2mtx_method is dods
7.3.2 cwd /export/home/vferrer cmdline mri_glmfit --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/tmp.mri_glmfit-sim-21231/contrast_moca_groups.mtx --sim perm 1000 1.5 /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/csd/perm.th15.abs.j001 --sim-sign abs --fwhm 0 --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd dods --mask /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh --perm-force --seed 1669778971 sysname Linux hostname ipsnode-7-3.local machine x86_64 user vferrer FixVertexAreaFlag = 1 UseMaskWithSmoothing 1 OneSampleGroupMean 0 y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh logyflag 0 usedti 0 FSGD /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd mask /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh maskinv 0 glmdir (null) IllCondOK 0 ReScaleX 1 DoFFx 0 Loading y from /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh ... done reading. INFO: gd2mtx_method is dods Computing normalized matrix Normalized matrix condition is 1e+08 Design matrix ------------------ -------------------------------- ERROR: matrix is ill-conditioned or badly scaled, condno = 1e+08
-------------------------------- -------- ERROR: READ THIS ----------------- --vvvvvvvvvvvvvvvvvvvvvvvvvvvvvv----- Possible problem with experimental design: Check for duplicate entries and/or lack of range of continuous variables within a class. If you seek help with this problem, make sure to send: 1. Your command line: mri_glmfit --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/tmp.mri_glmfit-sim-21231/contrast_moca_groups.mtx --sim perm 1000 1.5 /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/csd/perm.th15.abs.j001 --sim-sign abs --fwhm 0 --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd dods --mask /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh --perm-force --seed 1669778971 2. The terminal output of this program (ie, everything it prints to the screen) 3. The FSGD file (/bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd) 4. The design matrix (null)/Xg.dat Attempting to diagnose further SumSq: Min=1.455099 (col 24), Max=8.657582 (col 23) ... could not determine the cause of the problem --^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^----
Vicente Ferrer Predoctoral Researcher BCBL https://secure-web.cisco.com/12Qp9WHMJD61cOgJkvr_aW6rn7hQpkT1jyv9RXJdg2TkiSG...
Legal disclaimer/Aviso legal/Lege-oharra: https://secure-web.cisco.com/10QQqC3yL1zlnxukn9FKyjUpa-obVbh3uY9lJH7xCD98Hb2...
----- Original Message ----- From: "Douglas N. Greve" dgreve@mgh.harvard.edu To: "freesurfer" freesurfer@nmr.mgh.harvard.edu Sent: Monday, November 28, 2022 3:04:50 AM Subject: Re: [Freesurfer] ERROR: matrix is ill-conditioned or badly scaled, condno = 1e+08
that is quite strange. Does it happen every time? What is the mri_glmfit-sim command line and terminal output?
On 11/23/2022 2:47 PM, Vicente Ferrer wrote:
External Email - Use CautionDear Freesurfer users,
I was trying yo use mri_glmfit-sim after using mri_glmfit that run successfully and I encounter this error:
ERROR: matrix is ill-conditioned or badly scaled, condno = 1e+08
-------- ERROR: READ THIS ----------------- --vvvvvvvvvvvvvvvvvvvvvvvvvvvvvv----- Possible problem with experimental design: Check for duplicate entries and/or lack of range of continuous variables within a class. If you seek help with this problem, make sure to send:
- Your command line:
mri_glmfit --C /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/tmp.mri_glmfit-sim-29440/contrast_moca_groups.mtx --sim perm 1000 1.5 /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_b br.FA_Avg.corr_g_glmdir/csd/perm.th15.abs.j001 --sim-sign abs --fwhm 0 --fsgd /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd dods --mask /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg 16_syn_bbr.FA_Avg.corr_g_glmdir/mask.mgh --y /bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/eres.mgh --perm-force --seed 1669778971
- The terminal output of this program (ie, everything it prints to the screen)
- The FSGD file (/bcbl/home/public/PD_ANALYSIS/FREESURFER_PPMI/stats/lh.atr.avg16_syn_bbr.FA_Avg.corr_g_glmdir/y.fsgd)
- The design matrix (null)/Xg.dat
Attempting to diagnose further SumSq: Min=1.455099 (col 24), Max=8.657582 (col 23) ... could not determine the cause of the problem --^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^----
Seems like there is a problem with the condition matrix, but I do not understand why. Could you help me with this problem? I attached the contrast and fsgd files
Best, Vicente Ferrer Predoctoral Researcher BCBL https://secure-web.cisco.com/1OSix7XvH4Hpvsz0Ig1bkLPL5-req1MpAIrVbcxwoogFjZ3...
Legal disclaimer/Aviso legal/Lege-oharra: https://secure-web.cisco.com/1eewAlDYBH4yNaqR3T9LtmOW9A3ATHhSVEm7sgF9ecgPqf_...
Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://secure-web.cisco.com/1-d96koosL1MBSnuGB514qj0im3bqGCaLuYf5BFoyjnhD95...
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://secure-web.cisco.com/1-d96koosL1MBSnuGB514qj0im3bqGCaLuYf5BFoyjnhD95... The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1DzE8T41ZHcyBPWgS55kiiAt_gez132NHTJ2JFgpqsFSzHd... <https://secure-web.cisco.com/1xSXBP8VFW8vr0PS-2Uvbw_1Ws8cAKIokNrDsSqC1c4KqAJ... Z9Rd1D5I-xO9vzfUwYwLdrJWi5OgzVN4zY6MtQZzVMNH83go7ydkfbinkG7r4zDLSzRR2lIap9Sa6M86OhONd5ZnQV-cBjZgLf5Ur0-flQU4Y1W2NJzZJ0AXI5QwnulcM1UyRLjq2fP0NpCYYFtjfjyy5L/https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fwww.massgeneralbrigham.org%2Fcomplianceline__;!!D9dNQwwGXtA!TjwEwxlcphP94Eem0VV_Ac5cyRfTuBgB2oEHsWsri2po-UDTIxqY3ZvzNfn5hDA4gGsn1MkRFVGg6ngS2gD2Tw$ > . Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://secure-web.cisco.com/1rzV6-VEUAd7tNcHUmQ0L2sA-p17P8e_56U9w4cQrmJinDq... The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Mass General Brigham Compliance HelpLine at https://secure-web.cisco.com/1KAuA_SDL-JtLlpPFsGHDNh7EO7I_40dvlsW5T0s38cATx_... <https://secure-web.cisco.com/1KAuA_SDL-JtLlpPFsGHDNh7EO7I_40dvlsW5T0s38cATx_... > . Please note that this e-mail is not secure (encrypted). If you do not wish to continue communication over unencrypted e-mail, please notify the sender of this message immediately. Continuing to send or respond to e-mail after receiving this message means you understand and accept this risk and wish to continue to communicate over unencrypted e-mail.