// the find
benchflow-ai/awesome-evals
A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
An annotated 'awesome list' for AI agent evaluation — papers, blog posts, talks, and eval tooling, organized into a taxonomy from academic RL-environment theory down to practical eval frameworks like Inspect AI and promptfoo. Maintained by BenchFlow (an eval-tooling company) and aimed at engineers building or researching agent evals who want a reading list with actual context instead of a raw bookmark dump.
Each entry has a one-line 'what it is and why it matters' annotation plus a source type tag (blog/paper/talk/tool), which is the actual differentiator from a typical awesome-list. There's a genuine secondary layer of value in notes/ — 143 deep-dive summary files on individual papers and talks, which is more work than most maintainers put into this kind of repo. The GitHub Actions setup (eval-scan.yml, respond.yml, scan-recipe.md) shows they built actual automation to re-scan for new sources and keep links current rather than letting it go stale after the initial burst of stars.
It's a markdown list with no machine-readable export — no JSON/YAML index despite having automated scanning infrastructure behind it, so you can't consume it programmatically or diff it easily. The maintainer is a company whose own product (BenchFlow/SkillsBench/ClawsBench) appears inside its own curated list, which isn't disclosed as a conflict anywhere in the README even though the entries are otherwise unbiased. Several entries are tagged with an explicit unverified-URL warning, which undercuts the 'every URL is checked' claim in the pitch. At 443+ links across 12 sections, there's no real filtering mechanism beyond the 12-item 'must-read starter set' — someone who isn't already deep in eval research will have a hard time knowing where to stop reading.