finds.dev

ogrisel/parallel_ml_tutorial

Tutorial on scikit-learn and IPython for parallel machine learning

View on GitHub

★ 1,588Jupyter Notebookupdated Oct 2016

// the take

A 2013 PyCon tutorial (notebooks) teaching core scikit-learn concepts plus parallelizing cross-validation and grid search with IPython's parallel computing tools. Aimed at developers who already know basic scikit-learn and want to learn distributed model selection and memory-mapped feature extraction for text classification.

The progression from single-machine model selection to distributed grid search is well structured and still conceptually sound. The memory-mapping material (mmap_utils.py) for sharing large feature matrices across worker processes without copying is a genuinely useful technique that most intro ML tutorials skip. Text classification exercises use real-ish scale data rather than toy examples.

Last commit was 2016 — the parallelism layer is IPython.parallel (ipyparallel's predecessor), which has since been superseded, and the setup instructions target scikit-learn 0.15 and IPython 2.2, both many major versions behind current APIs. StarCluster, the recommended tool for spinning up the EC2 IPython cluster, is unmaintained and effectively dead, so the distributed exercises likely won't run as written. Nothing here reflects how people actually do distributed hyperparameter search today (joblib/dask/ray), so it's a history lesson more than a working guide for a 2026 reader.

We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
1,588
Forks
585
Language
Jupyter Notebook
Last commit
4 Oct 2016

// nearby

If parallel_ml_tutorial isn't quite it.

All repos →