// the find
iusztinpaul/energy-forecasting
🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
A 7-lesson MLOps course that walks you through building a batch ML system for energy consumption forecasting in Denmark. Covers the full pipeline from feature engineering through training, batch prediction, monitoring, and GCP deployment. Aimed at ML engineers who want to see how the pieces actually connect, not just read about them.
The three-pipeline split (feature / training / batch-prediction) is correctly modeled as separate deployable units with their own Poetry packages and private PyPi server — this mirrors real production setups where these run on different schedules. Uses Hopsworks as a feature store rather than rolling a bespoke one, which is the right call for a course at this level. The Airflow DAG orchestrates the full pipeline end-to-end with backfill support, so you can see how scheduling and dependency management actually work together. GitHub Actions CI/CD is wired to deploy to GCP VMs, not just run tests — students see the full deploy path.
Pinned to Python 3.9 and Poetry 1.4.2 from 2023 — the data API went offline mid-2023, and the workaround is a static Google Drive file, so any infrastructure rot means you're chasing dead links before you write a line of code. The private PyPi server setup is the most fragile part: four manual credential steps before anything else works, and a single wrong `htpasswd` invocation breaks downstream Docker builds in ways that aren't obvious. Hopsworks and W&B both require account creation with project name uniqueness constraints across their entire user base, which is an annoying dependency for a self-contained course. Last code push was April 2024 — sktime and Great Expectations both had major breaking changes since then, so `poetry install` may not succeed without manual intervention.