// the find
time-series-foundation-models/lag-llama
Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Lag-Llama is a transformer-based foundation model for probabilistic time series forecasting, built on a LLaMA architecture with lag features as input tokens. It targets zero-shot generalization across datasets and frequencies, outputting full probability distributions rather than point estimates. The audience is ML researchers and practitioners who need a pretrained starting point for forecasting tasks without building from scratch.
Probabilistic output is the right call — you get prediction intervals, not just a point estimate, which matters for anything where uncertainty quantifies risk. The lag-feature tokenization is a clean idea: instead of raw values, it feeds lagged observations at multiple scales, giving the transformer temporal structure without needing explicit periodicity baked in. Pretrained weights are on HuggingFace and the Colab demos are actually runnable, which is not a given for academic ML repos. The June 2024 kv_cache bug fix (resetting y_cache globally, correct causal attention during init, RoPE adjustment post-concat) suggests someone is still maintaining accuracy-critical code paths.
Activity has effectively stalled — last commit was June 2025 but the substantive development stopped around mid-2024, and the model hasn't kept pace with newer time series foundation models like Chronos or TimesFM that have more training data and better benchmarks. The dependency on GluonTS is a non-trivial operational burden; GluonTS has its own data format conventions that don't map cleanly onto pandas DataFrames without adapter code, and the repo gives you that friction without smoothing it. Context length sensitivity is a known issue the authors acknowledge but don't fully solve — you have to grid-search context length per dataset, which undermines the zero-shot pitch. There are no production-readiness affordances: no ONNX export, no serving example, no batching guidance for throughput — this is a research artifact, not something you drop into a pipeline.