[Mne_analysis] shared memory using joblib and clusterering-permutation tests

Denis-Alexander Engemann denis.engemann at gmail.com
Wed Aug 6 06:38:50 EDT 2014
Search archives:

Dear joblib experts on this list,

I'm wondering which steps are required to use joblib's shared memory
feature.
I used `mne.set_cache_dir` and made sure my `stat_fun` treats variables
independently.
I started with a moderate buffer_size of 500.
As a result my memory consumption increases radically with 4 jobs and my
system starts swapping excessively.
This is not expected, I think. Given the small buffer size I would not have
expected any significant memory increase.
After some time I witness a `SystemError: error return without exception
set` which is not good.

How can I be really sure it actually works?
I feel we should add more verbosity and logging here.

Best,
Denis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20140806/e8ac804d/attachment.html 


More information about the Mne_analysis mailing list