// the find
sebastianruder/NLP-progress
Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.
A community-maintained reference tracking state-of-the-art results across NLP benchmarks — datasets, metrics, and leaderboard numbers organized by task. It's aimed at researchers who need a quick orientation on where a given problem stands before digging into papers. Think of it as a living survey paper in Markdown.
Covers an unusually wide range of tasks from POS tagging to dialogue and multimodal, making it useful as a map of the NLP landscape rather than just a leaderboard. The structured export script lets you pull all the tables into JSON programmatically, which is handy for tooling. Community contribution model via GitHub PRs has produced 3,500+ forks and genuine coverage that a solo maintainer couldn't manage. Multilingual coverage (Vietnamese, Chinese, Arabic, Hindi, etc.) is broader than most similar references.
Last push was July 2024 and the pre-transformer era tables are visibly stale — BERT-era numbers sitting at the top of 'SOTA' tables that haven't been touched in years. LLMs have completely reshuffled most of these benchmarks and the repo doesn't reflect that reality; it increasingly looks like a historical artifact rather than a current reference. No automation: every update is a manual PR editing a Markdown table, which doesn't scale and explains the staleness. The non-English coverage is thin and uneven — Vietnamese has seven tasks while Arabic has one.