[Mne_analysis] Resampling with TMS artifacts
Rockhill, Alexander P.
AROCKHILL at mgh.harvard.edu
Wed Feb 14 15:11:17 EST 2018
Hi Alex,
The raw .fif file is 30 GB and I have 60 GB of RAM. When I load the epochs, it takes up more than the 60 GB I have of RAM. I have tried iterating over the epochs, loading each section and then putting them back into an EpochsArray but then when I go to read them in I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 448, in load_data
self._data = self._get_data()
File "<string>", line 2, in _get_data
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
return function(*args, **kwargs)
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1170, in _get_data
epoch_noproj = self._get_epoch_from_raw(idx)
File "<string>", line 2, in _get_epoch_from_raw
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
return function(*args, **kwargs)
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 2606, in _get_epoch_from_raw
raise RuntimeError('Correct epoch could not be found, please '
RuntimeError: Correct epoch could not be found, please contact mne-python developers
________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu [mne_analysis-bounces at nmr.mgh.harvard.edu] on behalf of Alexandre Gramfort [alexandre.gramfort at inria.fr]
Sent: Wednesday, February 14, 2018 3:04 PM
To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] Resampling with TMS artifacts
to interpolate the artifact you need to load the data from disk.
it seems that your data are big and you cannot load them in memory.
how big are your data? How much RAM do you have on your machine?
Alex
On Wed, Feb 14, 2018 at 8:11 PM, Rockhill, Alexander P. <AROCKHILL at mgh.harvard.edu<mailto:AROCKHILL at mgh.harvard.edu>> wrote:
Hi Mainak,
Thank you for the suggestion. When I load in the raw data using the string preload option, there isn't a problem, but when try giving a string for preload to the epochs, I get:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "<string>", line 2, in __init__
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
return function(*args, **kwargs)
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1960, in __init__
verbose=verbose)
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 386, in __init__
self.load_data() # this will do the projection
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 448, in load_data
self._data = self._get_data()
File "<string>", line 2, in _get_data
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/utils.py", line 728, in verbose
return function(*args, **kwargs)
File "/autofs/space/karima_001/users/alex/software/anaconda2.7/lib/python2.7/site-packages/mne-0.16.dev0-py2.7.egg/mne/epochs.py", line 1212, in _get_data
dtype=epoch_out.dtype, order='C')
MemoryError
In order to interpolate the artifact and downsample the data, I am following recommendations that the data be in epochs. Is there any suggestions for how to avoid this?
Thanks,
Alex
________________________________
From: mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu> [mne_analysis-bounces at nmr.mgh.harvard.edu<mailto:mne_analysis-bounces at nmr.mgh.harvard.edu>] on behalf of Mainak Jas [mainakjas at gmail.com<mailto:mainakjas at gmail.com>]
Sent: Wednesday, February 14, 2018 11:29 AM
To: Discussion and support forum for the users of MNE Software
Subject: Re: [Mne_analysis] Resampling with TMS artifacts
Hi,
How long is the artifact? Probably you could use fix_stim_artifact for this: https://mne-tools.github.io/dev/generated/mne.preprocessing.fix_stim_artifact.html
or use decim parameter when epoching
Mainak
On Wed, Feb 14, 2018 at 3:49 PM, Rockhill, Alexander P. <AROCKHILL at mgh.harvard.edu<mailto:AROCKHILL at mgh.harvard.edu>> wrote:
Hi all,
I was wondering about the best way to resample data with a high-frequency TMS artifact. I first epoched the raw data so that the TMS events would be kept but when I resample the epochs, a low-pass filter is applied which makes sense but causes a large filter artifact from the TMS pulse. Is there any way to not apply this filter?
Thanks,
Alex
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20180214/9cfd42cb/attachment.html
More information about the Mne_analysis
mailing list