// the find
mikeal/spider
Programmable spidering of web sites with node.js and jQuery
A Node.js web scraping library from 2013 that loads pages into a DOM and lets you write jQuery-style handlers matched against URL patterns. It predates modern scraping tools by a decade and was built for a very different Node.js ecosystem.
The URL routing approach — defining handlers per host/path pattern rather than imperative crawl loops — is a genuinely clean API design. Bundling jsdom + jQuery so you can write familiar DOM traversal code was pragmatic for its time. The pluggable cache interface is a decent abstraction.
Abandoned since 2019, and effectively unmaintained well before that. Ships with a vendored jquery.js in the root, which is a red flag for a library. No async/await, no Promises — pure callbacks throughout, which makes error handling a mess. Superseded entirely by Playwright, Puppeteer, and Cheerio+got, which are actively maintained and handle modern JS-rendered pages, anti-bot measures, and stealth requirements that this library has no concept of.