[Mne_analysis] Resampling with TMS artifacts

Rockhill, Alexander P. AROCKHILL at mgh.harvard.edu
Wed Feb 14 15:57:21 EST 2018
Search archives:

Any advice on how to avoid the runtime error?

I think it has to do with resampling the epochs to a different rate than the original raw but I was using the MNE examples section about creating an Epochs object from scratch as a template.

This was my attempt at resampling without a low-pass filter so that I could remove the artifact afterwards.

epochs = read_epochs(os.getcwd() + '/' + subject + '/TMS-epo.fif',preload=False)
new_sfreq = 1000.0
new_ntimes = int((epochs.tmax-epochs.tmin)*new_sfreq)+1
if int(epochs.info['sfreq'])%int(new_sfreq) != 0:
    print('Sampling frequencies are not integer multiples, will not work.')
else:
    factor = int(epochs.info['sfreq'])/int(new_sfreq)
times = epochs.times[::factor]
tind = epochs.time_as_index(times)
epochs_resampled_data = np.zeros((len(epochs.events),epochs.info['nchan'],len(times)))
for i,epoch in enumerate(epochs):
    epochs_resampled_data[i] = epoch[:,tind]

ch_names = [ch['ch_name'] for ch in epochs.info['chs']]
ch_types = []
for ch in ch_names:
    if 'EOG' in ch:
        ch_types.append('eog')
    elif 'ECG' in ch:
        ch_types.append('ecg')
    elif 'STI' in ch:
        ch_types.append('stim')
    elif 'EMG' in ch:
        ch_types.append('emg')
    else:
        ch_types.append('eeg')
info = create_info(ch_names=ch_names,
                   ch_types=ch_types,
                   sfreq=new_sfreq)
epochs = EpochsArray(epochs_resampled_data,epochs.info,events=events2,tmin=tmin)
epochs.save(os.getcwd() + '/' + subject + '/' + 'TMS_resampled-epo.fif')


but again when I go to read the epochs in with preload=True, I get the RunTime error.

Thanks for all the help,

Alex
________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu [mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Alexandre Gramfort [alexandre.gramfort at inria.fr]
Sent: Wednesday, February 14, 2018 3:22 PM
To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] Resampling with TMS artifacts

we load the data as float64 for stable numerics so you have data of 30GB in float32
then yes it will blow up...

I will crop the raw, extract epochs in each chunk and then eventually concatenate
the epochs.

HTH
Alex

On Wed, Feb 14, 2018 at 9:11 PM, Rockhill, Alexander P. <AROCKHILL at mgh.harvard.edu<mailto:AROCKHILL at mgh.harvard.edu>> wrote:
Hi Alex,

    The raw .fif file is 30 GB and I have 60 GB of RAM. When I load the epochs, it takes up more than the 60 GB I have of RAM. I have tried iterating over the epochs, loading each section and then putting them back into an EpochsArray but then when I go to read them in I get the following error:

   Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 448, in load_data
    self._data = self._get_data()
  File "<string>", line 2, in _get_data
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1170, in _get_data
    epoch_noproj = self._get_epoch_from_raw(idx)
  File "<string>", line 2, in _get_epoch_from_raw
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 2606, in _get_epoch_from_raw
    raise RuntimeError('Correct epoch could not be found, please '
RuntimeError: Correct epoch could not be found, please contact mne-python developers

________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu> [mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu>] on behalf of Alexandre Gramfort [alexandre.gramfort at inria.fr<mailto:alexandre.gramfort at inria.fr>]
Sent: Wednesday, February 14, 2018 3:04 PM

To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] Resampling with TMS artifacts

to interpolate the artifact you need to load the data from disk.

it seems that your data are big and you cannot load them in memory.

how big are your data? How much RAM do you have on your machine?

Alex

On Wed, Feb 14, 2018 at 8:11 PM, Rockhill, Alexander P. <AROCKHILL at mgh.harvard.edu<mailto:AROCKHILL at mgh.harvard.edu>> wrote:
Hi Mainak,

    Thank you for the suggestion. When I load in the raw data using the string preload option, there isn't a problem, but when try giving a string for preload to the epochs, I get:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "<string>", line 2, in __init__
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1960, in __init__
    verbose=verbose)
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 386, in __init__
    self.load_data()  # this will do the projection
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 448, in load_data
    self._data = self._get_data()
  File "<string>", line 2, in _get_data
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1212, in _get_data
    dtype=epoch_out.dtype, order='C')
MemoryError



In order to interpolate the artifact and downsample the data, I am following recommendations that the data be in epochs. Is there any suggestions for how to avoid this?

Thanks,

Alex
________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu> [mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu>] on behalf of Mainak Jas [mainakjas at gmail.com<mailto:mainakjas at gmail.com>]
Sent: Wednesday, February 14, 2018 11:29 AM
To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] Resampling with TMS artifacts

Hi,

How long is the artifact? Probably you could use fix_stim_artifact for this: https://mne-tools.github.io/dev/generated/mne.preprocessing.fix_stim_artifact.html

or use decim parameter when epoching

Mainak

On Wed, Feb 14, 2018 at 3:49 PM, Rockhill, Alexander P. <AROCKHILL at mgh.harvard.edu<mailto:AROCKHILL at mgh.harvard.edu>> wrote:
Hi all,

    I was wondering about the best way to resample data with a high-frequency TMS artifact. I first epoched the raw data so that the TMS events would be kept but when I resample the epochs, a low-pass filter is applied which makes sense but causes a large filter artifact from the TMS pulse. Is there any way to not apply this filter?

Thanks,

Alex

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.



_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.



_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20180214/a2579879/attachment-0001.html 


More information about the Mne_analysis mailing list