// the take
A set of smart contract security challenges, each modeled on real findings from the author's audits at Cyfrin, run through Foundry, Echidna, Medusa, and occasionally Halmos/Certora to compare how well each tool finds the bug out of the box. It's aimed at auditors and protocol developers deciding which fuzzer/formal-verification tool to adopt, or wanting worked examples of invariant tests that catch real bug classes.
Challenges are derived from actual audit findings rather than contrived toy bugs, so the invariants being tested map to real exploit classes (flash loan drains, NFT voting manipulation, token sale rounding, liquidation DoS). Each challenge ships both a 'basic' unguided config and an 'advanced' guided config per tool, which is a genuinely useful signal for how much hand-tuning each fuzzer needs to find the same bug. The repo includes complete, runnable artifacts for every tool (echidna.yaml, medusa.json, certora.spec/conf, Foundry invariant tests) rather than just describing results, so you can copy a harness and adapt it directly.
The comparisons are entirely qualitative and anecdotal ('Medusa typically breaks 2 invariants within 5 minutes, often sooner on subsequent runs') with no seeds, run counts, or timing data, so results aren't independently reproducible and there's no way to confirm a 'winner' claim yourself. The README documents that Medusa regressed on Challenge #6 after an upstream change, which shows these rankings go stale as each tool's fuzzing engine evolves, and there's no CI rerunning the challenges to catch when a verdict flips. The 'Challenge #9 -> #14' section has no written-up results at all, just a one-line mention that solvers exist, so roughly a third of the challenge set has no documented comparison. Last pushed February 2025, and since it tracks fast-moving tools (Echidna, Medusa, Foundry invariant testing) whose performance characteristics change release to release, the conclusions have a short shelf life.
We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →