// the find
jaypyles/Scraperr
Self-hosted webscraper.
Scraperr is a self-hosted web scraper with a Next.js frontend and FastAPI backend, built for people who want point-and-click scraping without writing code. You configure XPath selectors through a UI, submit jobs to a queue, and get results as CSV or markdown. Playwright handles JS-heavy pages.
Playwright integration means it handles SPAs and JS-rendered content, not just static HTML. The job queue with cron scheduling is genuinely useful — you can set recurring scrapes without babysitting a script. Helm chart and Docker Compose are both present, so deploying to k8s or a single VPS are both first-class options. The Cypress + pytest test matrix is more than most self-hosted tools bother with.
XPath-only targeting is a significant constraint — CSS selectors are missing entirely, which will frustrate anyone used to browser devtools copy-selector workflow. MongoDB for job storage is an odd choice that adds a hard dependency with no obvious benefit over SQLite or Postgres for this use case. The AI/agent feature is bolted on with its own router and prompts but there's no documentation on what model it calls or what it actually does beyond a chat page. Last push was October 2025 and the single Alembic migration suggests the schema hasn't been touched since the initial release.