[Mne_analysis] Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
Cushing, Cody
CCUSHING1 at mgh.harvard.edu
Tue Apr 12 14:05:40 EDT 2016
Hi Alex,
I've got one final probe for you on the topic, if you don't mind. I've been reading up on the estimation of the noise covariance matrix, since that was the main player in activation differences. It seems the standard approach for event-related studies is to compute the noise covariance based on a pre-stimulus baseline if "enough samples are available", but without any explicit suggestions of what is enough. Do you have an opinion on enough samples? And also, you are correct about me computing the noise covariance over my whole trial in mne-c, but that is what seems to make my activations higher (in dspm values) which seems counter-intuitive to me? Or perhaps it is my understanding that is inversed. And, also, I know the simple answer to this question is "just use mne-python", but the parameters in the .cov text file for mne-c now leave me confused. If tmin/tmax defines the time period to compute the noise covariance over, why is there a bmin/bmax option, i.e. why do I want to baseline-correct a baseline? Should those values (tmin/tmax, bmin/bmax) be equivalent to get the "best" result? Thanks for any input.
Cheers,
Cody
________________________________________
From: alexandre.gramfort at gmail.com [alexandre.gramfort at gmail.com] on behalf of Alexandre Gramfort [alexandre.gramfort at m4x.org]
Sent: Saturday, April 09, 2016 7:31 AM
To: Cushing, Cody
Subject: Re: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
hi Cody,
the problem was actually the use of
tmin -0.03
tmax 0.06
and not
tmin -0.03
tmax 1.
see notebook attached and the modified cov file that make everything match.
Your C pipeline was actually computing the noise cov on the entire
epochs not just the baseline.
If you confirm please conclude the thread on the mailing list.
Alex
On Fri, Apr 8, 2016 at 8:05 PM, Cushing, Cody <CCUSHING1 at mgh.harvard.edu> wrote:
> Hey Alex,
>
> Sure, no problem. I tried it on a handful of subjects, so I would imagine it would be replicable with the sample data, but I haven't confirmed that. I'm sending a .zip via our transfer service here that should contain everything you need to replicate it (just tried it again myself to be sure), but let me know if anything is missing that you need. I appreciate your help and time.
>
> Cheers,
> Cody
> ________________________________________
> From: alexandre.gramfort at gmail.com [alexandre.gramfort at gmail.com] on behalf of Alexandre Gramfort [alexandre.gramfort at m4x.org]
> Sent: Friday, April 08, 2016 12:34 PM
> To: Cushing, Cody
> Subject: Re: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
>
> ok
>
> please share the fif, .ave, .cov files and python/bash scripts to replicate
>
> I'll look into it
>
> A
>
> On Fri, Apr 8, 2016 at 6:25 PM, Cushing, Cody <CCUSHING1 at mgh.harvard.edu> wrote:
>> Hi Alex,
>>
>> So, in the .cov text file, the user defines 4 parameters, tmin/tmax of the epoch, which in this case should be -.03, 1.0s, and then you define the bmin/bmax that you would like the noise covariance to be calculated over, which in this case is -0.03, 0.06. This is set up to deal with the raw file as the input, thus it is epoching the data according to events (defined by tmin/tmax) and then calculating the noise covariance based on the bmin/bmax.
>>
>> Alternatively, for the mne-python commmand, the raw data is already epoched, so the user only defines the tmin/tmax to compute the cov on (there's no bmin/bmax option). Thus, if I'm understanding correctly, the bmin/bmax from the .cov txt file in mne-c is the equivalent of the tmin/tmax parameters for computing the cov with mne-py, which in both cases I have set here at -.03, .06.
>>
>> Cheers,
>> Cody
>> ________________________________________
>> From: alexandre.gramfort at gmail.com [alexandre.gramfort at gmail.com] on behalf of Alexandre Gramfort [alexandre.gramfort at m4x.org]
>> Sent: Friday, April 08, 2016 10:23 AM
>> To: Cushing, Cody
>> Subject: Re: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
>>
>>> sorry if i'm misunderstanding you here, but the original plot of the mne-c cov I sent you should be only the grads pulled from the full noise cov? I plotted with this:
>>>
>>> cov.plot(evoked.info,exclude=evoked.info['bads'])
>>>
>>> And if you look at the axes, they both end at what would appear to be 187? It certainly doesn't go all the way to 306.
>>
>> my bad. indeed.
>>
>> looking at your .cov txt file I see :
>>
>> tmin -0.03
>> tmax 1.0
>>
>> so you're actually computing the noise cov on the full epochs not just
>> the baseline.
>>
>> I think it should be :
>>
>> tmin -0.03
>> tmax 0.06
>>
>>
>> can you check?
>>
>> A
>>
>>> Or are you telling me to re-calculate a noise covariance in c using only the grads? In which case, I'm not quite sure what to do different if mne_process_raw doesn't take into account the bad channels marked in the raw file its being calculated on? Sorry if that is naive
>>>
>>> Cheers,
>>> Cody
>>> ________________________________________
>>> From: alexandre.gramfort at gmail.com [alexandre.gramfort at gmail.com] on behalf of Alexandre Gramfort [alexandre.gramfort at m4x.org]
>>> Sent: Friday, April 08, 2016 9:45 AM
>>> To: Cushing, Cody
>>> Subject: Re: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
>>>
>>> indeed mne-python removes the bad from the noise cov.
>>>
>>> restrict the noise cov from MNC-C to the grad and compare.
>>>
>>> let me know what you get
>>>
>>> A
>>>
>>> On Fri, Apr 8, 2016 at 3:26 PM, Cushing, Cody <CCUSHING1 at mgh.harvard.edu> wrote:
>>>> Hi Alex,
>>>>
>>>>
>>>> Same channels marked as bad (both marked from the same txt file).
>>>>
>>>> size of the python noise cov:
>>>>
>>>> size : 187 x 187, n_samples : 15510
>>>>
>>>> size of the mne-c noise cov:
>>>>
>>>> size: 306 x 306, n_samples : 166511
>>>>
>>>> I just loaded up the raw file in mne_browse_raw and the bad channels are all marked, but it doesn't look like the mne-c noise cov is excluding any of the bad channels?
>>>>
>>>> Cheers,
>>>> Cody
>>>> ________________________________________
>>>> From: alexandre.gramfort at gmail.com [alexandre.gramfort at gmail.com] on behalf of Alexandre Gramfort [alexandre.gramfort at m4x.org]
>>>> Sent: Friday, April 08, 2016 9:12 AM
>>>> To: Cushing, Cody
>>>> Subject: Fwd: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
>>>>
>>>> yes your noise cov look very different.
>>>>
>>>> that's where you need to investigate Do they have the same size? same
>>>> channels marked as bad?
>>>>
>>>> FYI we have tests against MNE C at the level so it has to match.
>>>>
>>>> A
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: <mne_analysis-owner at nmr.mgh.harvard.edu>
>>>> Date: Fri, Apr 8, 2016 at 3:05 PM
>>>> Subject: Mne_analysis post from ccushing1 at mgh.harvard.edu requires approval
>>>> To: mne_analysis-owner at nmr.mgh.harvard.edu
>>>>
>>>>
>>>> As list administrator, your authorization is requested for the
>>>> following mailing list posting:
>>>>
>>>> List: Mne_analysis at nmr.mgh.harvard.edu
>>>> From: ccushing1 at mgh.harvard.edu
>>>> Subject: RE: [Mne_analysis] Differences between command line and python?
>>>> Reason: Message body is too big: 673761 bytes with a limit of 500 KB
>>>>
>>>> At your convenience, visit:
>>>>
>>>> https://mail.nmr.mgh.harvard.edu/mailman/admindb/mne_analysis
>>>>
>>>> to approve or deny the request.
>>>>
>>>>
>>>> The information in this e-mail is intended only for the person to whom it is
>>>> addressed. If you believe this e-mail was sent to you in error and the e-mail
>>>> contains patient information, please contact the Partners Compliance HelpLine at
>>>> http://www.partners.org/complianceline . If the e-mail was sent to you in error
>>>> but does not contain patient information, please contact the sender and properly
>>>> dispose of the e-mail.
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: "Cushing, Cody" <CCUSHING1 at mgh.harvard.edu>
>>>> To: Discussion and support forum for the users of MNE Software
>>>> <mne_analysis at nmr.mgh.harvard.edu>
>>>> Cc:
>>>> Date: Fri, 8 Apr 2016 13:05:51 +0000
>>>> Subject: RE: [Mne_analysis] Differences between command line and python?
>>>> Hi Alex,
>>>>
>>>> Attached are plots of the evoked and covariances. The evoked look to
>>>> match but the noise covariances do not. So, you can see the commands
>>>> I'm implementing in the original message, and then just so its all on
>>>> the table, this is copy and pasted from my .cov txt file:
>>>>
>>>> gradReject 2000e-13
>>>>
>>>> eogReject 800e-6
>>>> #
>>>> # What to include in the covariance matrix?
>>>> #
>>>> def {
>>>> name "Condition 1"
>>>> event 1
>>>> tmin -0.03
>>>> tmax 1.0
>>>> bmin -0.03
>>>> bmax 0.06
>>>> color 1 0 0
>>>> }
>>>>
>>>>
>>>> Could the differences lie in how the noise covariance matrix is
>>>> calculated? It's being performed on epochs that have been baseline
>>>> corrected in the flow of those python commands, no? Whereas, I didn't
>>>> find anything explicitly clear in the mne-c manual, but it would seem
>>>> that the covariance is calculated on just the raw data? Thanks for
>>>> your time.
>>>>
>>>> Cheers,
>>>> Cody
>>>> ________________________________________
>>>> From: mne_analysis-bounces at nmr.mgh.harvard.edu
>>>> [mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Alexandre
>>>> Gramfort [alexandre.gramfort at telecom-paristech.fr]
>>>> Sent: Friday, April 08, 2016 4:55 AM
>>>> To: Discussion and support forum for the users of MNE Software
>>>> Subject: Re: [Mne_analysis] Differences between command line and python?
>>>>
>>>> ok
>>>>
>>>> let's check at which step the difference appears.
>>>>
>>>> Can you check that the evoked match by plotting them in Python?
>>>> Same question for the noise covariances
>>>>
>>>> If they match let's look at the inverse computation but first please
>>>> check the above.
>>>>
>>>> Alex
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Apr 7, 2016 at 6:42 PM, Cushing, Cody <CCUSHING1 at mgh.harvard.edu> wrote:
>>>>> Hi Alex,
>>>>>
>>>>> Thanks for the reply. I'm happy to provide whatever additional details, I just didn't want to bloat the original email too much.
>>>>>
>>>>> So, I switched to 'meg=True' in my picks, but it didn't change the end result. Also, just so its clear, I have all the mags listed in the bad channel file to essentially pick grads for the batch version, so all mags are excluded in both streams.
>>>>>
>>>>> And I do really want my tmax at 0.06 since there is a 60ms delay on our projector here, so time 0.06s is my true time 0s in the epoch. bmax is set at 0.06 in the .ave/.cov txt file as well (batch bmax equals python tmax for the covariance computation, yes? the .cov txt file has the user define the tmin/tmax of the epoch [-.03 1.0] as well as the bmin/bmax for the computation.
>>>>>
>>>>> Cheers,
>>>>> Cody
>>>>> ________________________________________
>>>>> From: mne_analysis-bounces at nmr.mgh.harvard.edu [mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Alexandre Gramfort [alexandre.gramfort at telecom-paristech.fr]
>>>>> Sent: Thursday, April 07, 2016 12:18 PM
>>>>> To: Discussion and support forum for the users of MNE Software
>>>>> Subject: Re: [Mne_analysis] Differences between command line and python?
>>>>>
>>>>> I miss some details but at first sight meg='grad' could be the error.
>>>>> Can you try with meg='True'
>>>>>
>>>>> next test is to make sure you really want to do:
>>>>>
>>>>> cov=mne.compute_covariance(epochs, tmin=-0.03, tmax=0.06);
>>>>>
>>>>> and not
>>>>>
>>>>> cov=mne.compute_covariance(epochs, tmin=-0.03, tmax=0);
>>>>>
>>>>> to be checked in the .ave txt file you used with the command line.
>>>>>
>>>>> Alex
>>>>>
>>>>> On Thu, Apr 7, 2016 at 4:32 PM, Cushing, Cody <CCUSHING1 at mgh.harvard.edu> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I've been noticing some pretty serious differences in dSPM values as I
>>>>>> transfer my dataset from the batch stream to the python stream. I believe
>>>>>> these differences are related to the computation of the noise covariance
>>>>>> matrix. Can anyone tell me the difference between implementing either of
>>>>>> these on the same raw file with the same bad channels marked and the same
>>>>>> events file:
>>>>>>
>>>>>> mne_process_raw --raw $rawfile1 --digtrig STI101 --projon --cov $covfile
>>>>>> --ave $avefile --events $eventfile --allevents --filteroff --saveavetag
>>>>>> "_cmdtest-ave" --savecovtag "_cmdtest-cov"
>>>>>>
>>>>>> with rejection parameters identical to below. In the .ave & .cov files:
>>>>>> tmin, tmax = -0.03, 1.0 bmin, bmax = -.03, .06
>>>>>>
>>>>>> versus implementing this in python:
>>>>>>
>>>>>> picks = mne.pick_types(raw.info, meg='grad', eeg=False, stim=False,
>>>>>> eog=True,
>>>>>> include=include, exclude='bads')
>>>>>>
>>>>>> epochs = mne.Epochs(raw, events, event_id, tmin, tmax, picks=picks,
>>>>>> baseline=(-0.03, 0.06), reject=dict(grad=2000e-13,
>>>>>> eog=800e-6),
>>>>>> preload=False, proj=True)
>>>>>> epochs.drop_bad_epochs()
>>>>>>
>>>>>> cov=mne.compute_covariance(epochs, tmin=-0.03, tmax=0.06);
>>>>>> mne.write_cov(cov2save,cov);
>>>>>>
>>>>>> evoked =
>>>>>> [epochs['1'].average(),epochs['2'].average(),epochs['3'].average(),epochs['4'].average(),epochs['5'].average(),epochs['6'].average(),epochs['7'].average(),epochs['8'].average(),epochs['9'].average
>>>>>> (),epochs['10'].average(),epochs['11'].average(),epochs['12'].average()]
>>>>>>
>>>>>> mne.write_evokeds(fname2save, evoked) # save evoked data to disk
>>>>>>
>>>>>> Because as far as I can tell they seem to be doing the same thing? But
>>>>>> there's some pretty stark differences in the SNR estimate and dSPM values
>>>>>> after putting both -ave.fif & -cov.fif files through the same fwd/inverse
>>>>>> solution commands based on the same source space for things as simple as the
>>>>>> initial response in the visual cortex (that I'd be happy to share with
>>>>>> whoever, I just wanted to make sure some silly option wasn't off that was
>>>>>> throwing everything off).
>>>>>>
>>>>>> Thanks for any help.
>>>>>>
>>>>>> Cheers,
>>>>>> Cody
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mne_analysis mailing list
>>>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>>>
>>>>>>
>>>>>> The information in this e-mail is intended only for the person to whom it is
>>>>>> addressed. If you believe this e-mail was sent to you in error and the
>>>>>> e-mail
>>>>>> contains patient information, please contact the Partners Compliance
>>>>>> HelpLine at
>>>>>> http://www.partners.org/complianceline . If the e-mail was sent to you in
>>>>>> error
>>>>>> but does not contain patient information, please contact the sender and
>>>>>> properly
>>>>>> dispose of the e-mail.
>>>>>>
>>>>> _______________________________________________
>>>>> Mne_analysis mailing list
>>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>>
>>>>> _______________________________________________
>>>>> Mne_analysis mailing list
>>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>
>>>> _______________________________________________
>>>> Mne_analysis mailing list
>>>> Mne_analysis at nmr.mgh.harvard.edu
>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: mne_analysis-request at nmr.mgh.harvard.edu
>>>> To:
>>>> Cc:
>>>> Date:
>>>> Subject: confirm 9942edb75546612f7c446bd57fb99488768663f5
>>>> If you reply to this message, keeping the Subject: header intact,
>>>> Mailman will discard the held message. Do this if the message is
>>>> spam. If you reply to this message and include an Approved: header
>>>> with the list password in it, the message will be approved for posting
>>>> to the list. The Approved: header can also appear in the first line
>>>> of the body of the reply.
More information about the Mne_analysis
mailing list