// the find
tomnomnom/meg
Fetch many paths for many hosts - without killing the hosts
meg is a small Go CLI that fetches a list of paths against a list of hosts, iterating path-by-path across all hosts rather than hammering one host at a time. It's built for security researchers doing recon across many targets (bug bounty, pentesting) who need bulk HTTP probing without triggering rate limits or abuse flags.
Single static binary via `go install`, zero runtime deps. The path-major request order (one path across all hosts, then the next path) is a genuinely good design choice — it spreads load thinly instead of bursting any single host. Concurrency and per-host delay are both separately tunable, which matters when scanning thousands of hosts. Output is raw request/response text on disk plus an index file, so it composes naturally with grep/awk instead of forcing you into a JSON pipeline.
No test files anywhere in the tree — for a tool that hand-rolls raw HTTP requests (via the companion rawhttp lib), that's a real gap. The rawhttp mode is flagged experimental and explicitly doesn't handle chunked transfer encoding, so verbose output can get corrupted mid-response. No structured output option (JSON/CSV) — you're stuck grepping flat files, which gets painful once you're scanning results programmatically rather than by eye. Last push was Feb 2024, and the low fork count relative to its stars/usage in security tooling suggests it's mostly dormant rather than actively maintained.