// the find
kovidgoyal/html5-parser
Fast C based HTML 5 parsing for python
A Python binding for a forked and extended version of Google's Gumbo HTML5 parser, written in C. It targets Python developers who need to parse large volumes of HTML quickly and find html5lib's speed unacceptable. The 30x speed claim over html5lib is credible given the C implementation.
The C-level implementation of the HTML5 parsing algorithm means you get spec-compliant parsing (handles malformed HTML the same way browsers do) without the overhead of pure Python. It outputs directly into multiple tree formats — lxml etree, BeautifulSoup, and Python's stdlib ElementTree — so you can slot it in without changing your downstream code. The fork of Gumbo includes modifications (gumbo_edit.c) that the upstream library dropped, meaning it's maintained with specific goals rather than just being an old snapshot. CI covers multiple platforms including Windows.
703 stars and 38 forks over what appears to be years of existence is a weak adoption signal — this is a niche tool that never got traction outside of Kovid's own projects (calibre). The Gumbo fork is a liability: Google abandoned Gumbo, so this repo is now solely responsible for keeping up with HTML5 spec changes and any security fixes in the parser core. No significant contributors outside the author means a bus-factor of one. The setup.py / pyproject.toml hybrid build is a sign of accumulated technical debt that will eventually cause packaging headaches on newer pip versions.