[Mne_analysis] PSDEstimator using multiple cores even when n_jobs=1
lee.kyuh at gmail.com
Mon Jul 2 07:51:15 EDT 2018
External Email - Use Caution
I found a problem in mne.decoding.PSDEstimator.transform() method. In
short, it does not respect the attribute "n_jobs". This problem does not
exist in MNE 0.14 and only exists in MNE 0.15 and above. I tested only on
Normally, I restrict the PSD estimation to use a single core (n_jobs=1)
because there are many other time-critical processes running together in
the system. In MNE 0.15 and above, the transform() function uses up to 4
cores equivalent (over 50% total usage on an 8-core system). This is in
contrast with the same function in MNE 0.14, where only 1 core is used as
The following minimal code replicates the problem.
import numpy as npimport mneimport time
psde = mne.decoding.PSDEstimator(sfreq=512, fmin=1, fmax=40, n_jobs=1)
iterations = 1000
w = np.random.rand(1, 64, 256)print('Start %d iteratinos' % iterations)
t_start = time.time()for i in range(iterations):
psd = psde.transform(w)print('Took %.3f seconds' % (time.time() - t_start))
In Windows 10 Pro with i7-4790K CPU (8 logical cores), I ran the above code
5 times each with 1000 computations. In the idle period, the cpu usage was
- mne 0.14: CPU usage 17% (equivalent to approx. 1 core = 12.5%)
10.461, 10.755, 10.861, 10.843, 10.636 seconds.
Average: 10.711s per 1000 computations.
- mne 0.15, 0.16: CPU usage 56% (equivalent to approx. 4 cores = 50%)
11.608, 10.737, 10.734, 11.283, 10.909 seconds.
Average: 11.054s per 1000 computations.
So, I don't see much difference in computation time but the CPU usage was
4x higher in MNE 0.15 and above. It seems to me that there's a bug in
PSDEstimator class that always prefers to use multiple cores even when
n_jobs=1, causing a lot of overhead by forced multiprocessing.
Please have a look and fix this issue. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Mne_analysis