<div dir="ltr">Are you using return_generator=True in the functions as you go? For example, returning generators for the STCs instead of actually calculating them all in `apply_inverse_epochs` is a pretty good memory saving feature because it delays computation as long as possible, keeping few items in memory:<div><br></div><div><a href="http://mne-tools.github.io/dev/generated/mne.minimum_norm.apply_inverse_epochs.html#mne.minimum_norm.apply_inverse_epochs">http://mne-tools.github.io/dev/generated/mne.minimum_norm.apply_inverse_epochs.html#mne.minimum_norm.apply_inverse_epochs</a><br></div><div><br></div><div>If you are already doing that, or if that doesn't help, please open an mne-python issue so we can investigate further.</div><div><br></div><div>Cheers,</div><div>Eric</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, May 23, 2015 at 12:38 AM, Christos Michas <span dir="ltr"><<a href="mailto:cmichas@bu.edu" target="_blank">cmichas@bu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thank you very much for your suggestions.<div><br></div><div>Unfortunately, I have to examine the whole signal, so cropping is not a viable solution. </div><div>I haven't low-passed my raw data though. I will try it. Nevertheless, I am also interested in frequencies as high as 80Hz, so low-passing will be limited and it won't reduce memory allocation as much as it's needed. So if you have any other suggestions, please do<span></span> share them.<br></div><div><br></div><div><span></span>Thanks again,</div><div>Christos<div><div class="h5"><br><br>On Friday, May 22, 2015, Eric Larson <<a>larson.eric.d@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If your raw data are low-passed sufficiently, then decimating during epoch creation should help, and shouldn't adversely affect frequency-domain measures. A sample rate of 1000 for a signal where you only care about < 20 Hz is using an order of magnitude more memory than it needs to -- low-passing raw and decimating during epoching should be fine.<div><br></div><div>Then if you're only interested in some small window during the epoch, you can use epochs.crop() to select that small time window, which should also help.</div><div><br></div><div>Cheers,</div><div>Eric</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 22, 2015 at 12:38 PM, Christos Michas <span dir="ltr"><<a>cmichas@bu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Dear everyone,<br><br></div>I am trying to use the spectral_connectivity function to calculate the coherence between several source space areas, but I run into memory errors and the calculations don't finish. I am following the example in [1], my source space has 68 labels and my stc has a time length of about 4000 samples. The function calculates the cwt at all time points, so it returns an array of size 68 x 68 x n_freqs x 4000. But when I use n_freqs > 12, the function crashes because either I ran out of memory, or the functions in spectral_connectivity fail because of memory errors.<br><br></div><div>For example, I use the parameters:<br><br>fmin = 6.<br>fmax = 16.<br>sfreq = 1000<br>faverage=True<br>method = 'coh'<br>mode = 'cwt_morlet'<br>cwt_frequencies = numpy.arange(6.,16.1,0.5)<br>cwt_n_cycles = 7<br>block_size = 1000<br>n_jobs = 1<br><br></div><div>the function crashes and I get the following traceback:<br><br>....<br>....<br><b>/home/christos/src/mne/mne/connectivity/spectral.pyc in _epoch_spectral_connectivity</b><br><br> 441 csd = x_cwt[idx_map[0][con_idx]] * \<br>--> 442 np.conjugate(x_cwt[idx_map[1][con_idx]])<br> 443 for method in con_methods:<br> 444 method.accumulate(con_idx, csd)<br><br>MemoryError: <br><br><br>My question is whether I can use time decimation for the cwt calculation. Spectral_connectivity computes the cwt for all time points, but in the end I don't need them. I'd like to use the wavelet transform only at specific time points of my signal. A decimation would speed up the computation and I think it would bypass the memory errors. I could decimate my own signal, but that will affect the quality of the coherence returned (won't it?), so I'd like to keep my signal as it is, with 4000 samples.<br><br></div><div>Is there a solution to my problem other than changing the mne code? Have I missed something so it's my fault this is happening? Your help is much appreciated.<br><br></div><div>Best regards,<br>Christos Michas<br><br>[1] <a href="https://martinos.org/mne/stable/auto_examples/connectivity/plot_mne_inverse_label_connectivity.html#example-connectivity-plot-mne-inverse-label-connectivity-py" target="_blank">https://martinos.org/mne/stable/auto_examples/connectivity/plot_mne_inverse_label_connectivity.html#example-connectivity-plot-mne-inverse-label-connectivity-py</a><br></div></div></div>
<br>_______________________________________________<br>
Mne_analysis mailing list<br>
<a>Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a><br>
<br>
<br>
The information in this e-mail is intended only for the person to whom it is<br>
addressed. If you believe this e-mail was sent to you in error and the e-mail<br>
contains patient information, please contact the Partners Compliance HelpLine at<br>
<a href="http://www.partners.org/complianceline" target="_blank">http://www.partners.org/complianceline</a> . If the e-mail was sent to you in error<br>
but does not contain patient information, please contact the sender and properly<br>
dispose of the e-mail.<br>
<br></blockquote></div><br></div>
</blockquote></div></div></div>
<br>_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a><br>
<br>
<br>
The information in this e-mail is intended only for the person to whom it is<br>
addressed. If you believe this e-mail was sent to you in error and the e-mail<br>
contains patient information, please contact the Partners Compliance HelpLine at<br>
<a href="http://www.partners.org/complianceline" target="_blank">http://www.partners.org/complianceline</a> . If the e-mail was sent to you in error<br>
but does not contain patient information, please contact the sender and properly<br>
dispose of the e-mail.<br>
<br></blockquote></div><br></div>