[Mne_analysis] Filtering and ICA memory issues
Mads Jensen
mje.mads at gmail.com
Wed Nov 25 09:47:23 EST 2015
Hi,
I use sklearn 0.17 (from anaconda). I have tried to the
"decim" param. I remember it as being "3" for data with 1000Hz sfreq.
But it didn't help much.
I have attach a script to show how I used it.
cheers,
mads
On 25/11/15 15:29, Denis-Alexander Engemann wrote:
> Hi Mads,
>
> Which version of sklearn are you using?
> Do you use the decim parameter for ICA?
> How do axactly do you use ICA?
> 50GB of memory is unexpected, it would mean that you make up to 10
> copies of your data.
>
>
> On Wed, Nov 25, 2015 at 3:24 PM, Mads Jensen <mje.mads at gmail.com
> <mailto:mje.mads at gmail.com>> wrote:
>
> Hi all,
>
> I would like to hear what people do to filter and run ICA and if there
> is any advise.
>
> We usually have around an hour of recording which gives ~4.5 to 5GB of
> raw fiff files. First filtering and then running ICA in MNE-python
> requires a lot of memory, sometimes as much as 50GB. So, I fairly often
> get a memory error.
>
> I would prefer not to downsample at this stage in the process. So, I
> kindly ask if anybody has any thoughts and/or practises to avoid very
> heavy memory use.
>
> best wishes,
> mads
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> <mailto:Mne_analysis at nmr.mgh.harvard.edu>
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to
> whom it is
> addressed. If you believe this e-mail was sent to you in error and
> the e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to
> you in error
> but does not contain patient information, please contact the sender
> and properly
> dispose of the e-mail.
>
>
>
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in error
> but does not contain patient information, please contact the sender and properly
> dispose of the e-mail.
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ica_test.py
Type: text/x-python
Size: 4840 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20151125/ccfbe99d/attachment.py
More information about the Mne_analysis
mailing list