<p><span style="padding: 3px 10px; border-radius: 5px; color: #ffffff; font-weight: bold; display: inline-block; background-color: #ff0000;"> External Email - Use Caution </span></p><p></p><div dir="ltr">Dear
Jean-<span id="gmail-m_-944367879157324119gmail-m_-6257750222229065525:88n.11"><span class="" id=":7jc.1" tabindex="-1" style="">Rémi</span>,</span><div><span><br></span></div><div>Thank you for the suggestion and, above all, thank you so much for your help and assistance!</div><div>My scripts have been working just fine while I would have never been able to implement my current analysis without your prompts.</div><div><br></div><div>All the best,</div><div><br></div><div>Giulia </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 7, 2020 at 3:10 PM Jean-Rémi KING <<a href="mailto:jeanremi.king@gmail.com">jeanremi.king@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Hi Giula,<div><br></div><div>In the long run, for batch optimization of parallel tasks (here each slided time sample), I would encourage you to have a look at pytorch; sklearn is not really optimal for this because it can't make use of gpu.</div><div><br></div><div><br></div><div><div>In the meantime, here is a solution to your problem: simply put your new class in a separate script e.g.</div><div><br></div><div># in mymodel.py<br>import numpy as np<br>from sklearn.linear_model import SGDClassifier<br><br>class MyModel(SGDClassifier):<br> def fit(self, X, y):<br> if not hasattr(self, 'classes_'):<br> self.classes_ = np.unique(y)<br> super().partial_fit(X, y, self.classes_)<br> return self <br><br># main script <br>import numpy as np<br>from mne.decoding import SlidingEstimator<br>from mymodel import MyModel<br>model = MyModel()<br>slider = SlidingEstimator(model, scoring='roc_auc', n_jobs=2)<br><br>X = np.random.randn(100, 10, 3)<br>y = np.random.randint(0, 2, 100)<br>slider.fit(X, y)<br>slider.score(X, y)<br></div><div><br></div><div>hope that helps</div><div><br></div><div>JR</div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 7 Aug 2020 at 14:34, Giulia Gennari <<a href="mailto:giulia.gennari1991@gmail.com" target="_blank">giulia.gennari1991@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Dear Jean-<span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-6257750222229065525:88n.11">Rémi and dear Alex,</span><div><br></div><div><b>Thank you!</b><br></div><div><b><br></b></div><div>A solution based on this:</div><div>class MyModel(SGDClassifier):<br> def fit(self, X, y):<br> super().partial_fit(X, y)<br> return self </div><div><br></div><div>..works fine! </div><div>Except for the crucial fact that parallel processing (n_jobs>1) seems not feasible.</div><div>This is what I get when I try to score the slider (apologies for the ugliness, I copy-paste everything since it might be meaningful to catch what is wrong):</div><div><font color="#351c75">---------------------------------------------------------------------------<br>_RemoteTraceback Traceback (most recent call last)<br>_RemoteTraceback:<br>"""<br>Traceback (most recent call last):<br> File "/usr/local/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/queues.py", line 150, in _feed<br> obj_ = dumps(obj, reducers=reducers)<br> File "/usr/local/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/reduction.py", line 243, in dumps<br> dump(obj, buf, reducers=reducers, protocol=protocol)<br> File "/usr/local/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/reduction.py", line 236, in dump<br> _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)<br> File "/usr/local/anaconda3/lib/python3.7/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 267, in dump<br> return Pickler.dump(self, obj)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 437, in dump<br> self.save(obj)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 549, in save<br> self.save_reduce(obj=obj, *rv)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce<br> save(state)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 859, in save_dict<br> self._batch_setitems(obj.items())<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 885, in _batch_setitems<br> save(v)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 549, in save<br> self.save_reduce(obj=obj, *rv)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce<br> save(state)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 859, in save_dict<br> self._batch_setitems(obj.items())<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 890, in _batch_setitems<br> save(v)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 549, in save<br> self.save_reduce(obj=obj, *rv)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce<br> save(state)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 859, in save_dict<br> self._batch_setitems(obj.items())<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 885, in _batch_setitems<br> save(v)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 819, in save_list<br> self._batch_appends(obj)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 846, in _batch_appends<br> save(tmp[0])<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 774, in save_tuple<br> save(element)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 504, in save<br> f(self, obj) # Call unbound method with explicit self<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 789, in save_tuple<br> save(element)<br> File "/usr/local/anaconda3/lib/python3.7/pickle.py", line 510, in save<br> rv = reduce(obj)<br> File "/usr/local/anaconda3/lib/python3.7/site-packages/joblib/_memmapping_reducer.py", line 361, in __call__<br> return (loads, (dumps(a, protocol=HIGHEST_PROTOCOL),))<br>_pickle.PicklingError: Can't pickle <class '__main__.MyModel'>: it's not the same object as __main__.MyModel<br>"""<br><br>The above exception was the direct cause of the following exception:<br><br>PicklingError Traceback (most recent call last)<br>/neurospin/grip/protocols/EEG/Giulia_NUM_MUSIK/NUM_MUSIK_DECODING_incremental_learning_test_on_VISUAL_DRAFT.py in <module><br> 278 y_test = test_epochs.events[:,2]<br> 279<br>--> 280 scores = time_gen.score(X_test, y_test)<br> 281 all_scores_D.append(scores)<br> 282<br><br><decorator-gen-375> in score(self, X, y)<br><br>~/.local/lib/python3.7/site-packages/mne/decoding/search_light.py in score(self, X, y)<br> 583 for pb_idx, x in array_split_idx(<br> 584 X, n_jobs, axis=-1,<br>--> 585 n_per_split=len(self.estimators_)))<br> 586<br> 587 score = np.concatenate(score, axis=1)<br><br>~/.local/lib/python3.7/site-packages/mne/parallel.py in run(*args, **kwargs)<br> 126 def run(*args, **kwargs):<br> 127 try:<br>--> 128 return fun(*args, **kwargs)<br> 129 except RuntimeError as err:<br> 130 msg = str(err.args[0]) if err.args else ''<br><br>/usr/local/anaconda3/lib/python3.7/site-packages/joblib/parallel.py in __call__(self, iterable)<br> 932<br> 933 with self._backend.retrieval_context():<br>--> 934 self.retrieve()<br> 935 # Make sure that we get a last message telling us we are done<br> 936 elapsed_time = time.time() - self._start_time<br><br>/usr/local/anaconda3/lib/python3.7/site-packages/joblib/parallel.py in retrieve(self)<br> 831 try:<br> 832 if getattr(self._backend, 'supports_timeout', False):<br>--> 833 self._output.extend(job.get(timeout=self.timeout))<br> 834 else:<br> 835 self._output.extend(job.get())<br><br>/usr/local/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)<br> 519 AsyncResults.get from multiprocessing."""<br> 520 try:<br>--> 521 return future.result(timeout=timeout)<br> 522 except LokyTimeoutError:<br> 523 raise TimeoutError()<br><br>/usr/local/anaconda3/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)<br> 433 raise CancelledError()<br> 434 elif self._state == FINISHED:<br>--> 435 return self.__get_result()<br> 436 else:<br> 437 raise TimeoutError()<br><br>/usr/local/anaconda3/lib/python3.7/concurrent/futures/_base.py in __get_result(self)<br> 382 def __get_result(self):<br> 383 if self._exception:<br>--> 384 raise self._exception<br> 385 else:<br> 386 return self._result<br><br>PicklingError: Could not pickle the task to send it to the workers.<br></font></div><div><br></div><div>Would you know to solve it?</div><div>Without parallel processing I don't think I can get to the end of the analysis before Christmas 😌</div><div><br></div><div>Thank you very much again!!!!</div><div><br></div><div>Giulia</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 6, 2020 at 4:12 PM Jean-Rémi KING <<a href="mailto:jeanremi.king@gmail.com" target="_blank">jeanremi.king@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Hi Giula,<div><br></div><div>good catch, I had forgotten that we're cloning the estimator for each time sample; you'll thus need to do this:</div><div><br></div><div>class MyModel(SGDClassifier):<br> def fit(self, X, y):<br> super().partial_fit(X, y)<br> return self<br><br>model = MyModel(loss='log', class_weight='balanced') <br>slider = SlidingEstimator(model, scoring='roc_auc')<br></div><div><br></div><div>Hope that helps</div><div><br></div><div>JR</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 6 Aug 2020 at 15:56, Giulia Gennari <<a href="mailto:giulia.gennari1991@gmail.com" target="_blank">giulia.gennari1991@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Dear Jean-<span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.11">Rémi</span>,<div><br></div><div>Thank you for the nice suggestion!</div><div><br></div><div>Just to make sure that this is working (I apologize for my ignorance):</div><div><br></div><div>When I run:</div><div>model = <span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.12">SGDClassifier</span>(loss='log', class_weight='balanced')<br>model.fit = model.partial_fit<br>slider1 = <span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.13">SlidingEstimator</span>(model, scoring='roc_auc')<br></div><div>slider1.fit(X_train, y_train)<br></div><div><br></div><div>or </div><div><br></div><div><span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.14">clf</span> = make_pipeline(<span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.15">Vectorizer</span>(), <span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.16">StandardScaler</span>(), model)<br>slider2 = <span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.17">SlidingEstimator</span>(<span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.18">clf</span>, scoring='roc_auc')<br></div><div>slider2.fit(X_train, y_train)<br></div><div><br></div><div>I do not get any error, while I would expect:</div><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><span style="box-sizing:border-box;color:rgb(178,43,49);font-weight:bold"><span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.19">ValueError</span></span>: class_weight 'balanced' is not supported for partial_fit. In order to use 'balanced' weights, use compute_class_weight('balanced', classes, y). Pass the resulting weights as the class_weight parameter.</pre><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline"><br></pre></div><div>Since this is what I get with:</div><div>model.fit(X_train[:,:,single_time_point], y_train)<br></div><div><br></div><div>Is there a good reason for that? E.g. class weights are computed internally beforehand by <span id="gmail-m_-1015324247553749545gmail-m_2700484388706076436gmail-m_-1869393484759478652gmail-m_-1700891060312205861:88n.20">SlidingEstimator</span>?</div><div><br></div><div>Thank you again!</div><div><br></div><div>Giulia</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Aug 5, 2020 at 7:18 PM Jean-Rémi KING <<a href="mailto:jeanremi.king@gmail.com" target="_blank">jeanremi.king@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Hi Giulia,<div><br></div><div>I think you should be able to change the method:</div><div><br></div><div>model = sklearn.linear_model.SGDClassifier()</div><div>model.fit = model.partial_fit</div><div>slider = mne.decoding.SlidingEstimator(model)</div><div>for X, y in train_batches:</div><div> slider.fit(X, y)</div><div><br></div><div>Best</div><div><br></div><div>JR</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 5 Aug 2020 at 18:40, Giulia Gennari <<a href="mailto:giulia.gennari1991@gmail.com" target="_blank">giulia.gennari1991@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p><span style="padding:3px 10px;border-radius:5px;color:rgb(255,255,255);font-weight:bold;display:inline-block;background-color:rgb(255,0,0)"> External Email - Use Caution </span></p><p></p><div dir="ltr">Hi!<div><br></div><div>I would need to try decoding with incremental learning (EEG data).</div><div>I was planning to use logistic regression by means of the <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html" target="_blank">SGDClassifier</a> .</div><div>I would then need to call .partial_fit to make my estimator learn on each of my training sets.</div><div>However:</div><div><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:1px 0px;margin-top:0px;margin-bottom:0px;line-height:inherit;color:rgb(0,0,0);word-break:break-all;border:0px;border-radius:0px;white-space:pre-wrap;vertical-align:baseline">'GeneralizingEstimator' object has no attribute 'partial_fit'</pre></div><div>Same issue for SlidingEstimator. <br></div><div>Is there a way to work around this limitation?</div><div><br></div><div>Thank you so so much in advance!</div><div><br></div><div>Giulia Gennari </div></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>
_______________________________________________<br>
Mne_analysis mailing list<br>
<a href="mailto:Mne_analysis@nmr.mgh.harvard.edu" target="_blank">Mne_analysis@nmr.mgh.harvard.edu</a><br>
<a href="https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis" rel="noreferrer" target="_blank">https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis</a></blockquote></div>