finds.dev

DataHenHQ/till

DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.

View on GitHubHomepage ↗

★ 815GoApache-2.0updated Dec 2021

// the take

Till is a Go-based MITM proxy you put in front of an existing scraper (any language) to get proxy rotation, user-agent randomization, cookie/session stickiness, and response caching without touching your scraping logic. It's aimed at teams running scrapers at real scale who are tired of rebuilding the same plumbing (retry, cache, request logging) for every project.

The MITM-proxy approach is the right call for this problem — any HTTP client in any language gets the benefits just by setting a proxy env var, no SDK or library lock-in. The GID (request fingerprint) plus full response caching means a failed multi-day scrape can be replayed from cache instead of re-hitting the target site, which is a real cost saver on bandwidth and proxy fees. Sticky sessions (same UA + proxy + cookie jar across a request group) solve a problem most scrapers hand-roll badly. Ships as a standalone binary via goreleaser, so there's no Go toolchain needed to run it.

No commits since late 2021 — four years dormant, so compatibility with current anti-bot measures (TLS fingerprinting, modern Cloudflare/Akamai challenges) is unknown and likely degraded. Running Till requires an auth token from DataHen's hosted service (till.datahen.com) to even start the server, so this isn't a fully self-contained open-source tool — you're dependent on a vendor's signup flow for a core feature. MITM on HTTPS means installing a custom root CA on every machine running a scraper, which is an operational headache in containerized or ephemeral environments (CI, serverless). No mention of how the cache scales or gets pruned, so long-running scrapers could accumulate unbounded local disk usage.

We read the README, the directory tree and a key source file in November 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
815
Forks
23
Language
Go
License
Apache-2.0
Last commit
5 Dec 2021

// nearby

If till isn't quite it.

All repos →