Blog
Web extraction, LLMs, and building in public.
Technical deep dives on web extraction, content parsing for LLMs, anti-bot bypass, and building open-source infrastructure in Rust. Written by the team behind webclaw.
webclaw turns any website into clean, structured content for AI applications. These posts cover the engineering decisions, trade-offs, and lessons learned building a web extraction toolkit from scratch.
92 postsPage 11 / 11
Mar 17, 2026Massi
Web scraping for AI agents: 3 hidden problems
Most scraping tools were built for data pipelines, not AI agents. Three things quietly break your pipeline and how to fix them.
Mar 12, 2026Massi
Why I built webclaw (Rust scraper for LLMs)
I was tired of scrapers that return 403 or need headless Chrome for basic HTML. So I built one in Rust that actually works.
Get started
Stop reading. Start scraping.
Cancel anytime. Turn any page into clean, structured content your agent can actually use.