LLM-ready formats
Choose markdown, JSON, llm, text, or raw HTML per request. No HTML parsing on your side.
POST /v1/scrape
Web scraping built for LLMs.
Turn any URL into clean, LLM-ready content with one call.
Send a URL, get back markdown, JSON, structured text, or raw HTML, stripped of nav, ads, and boilerplate. Built in Rust, returns static pages in around 118ms, and handles JavaScript rendering and bot protection automatically.
What you get
Everything in one call.
~90% fewer tokens
Boilerplate, nav, and ads are stripped so your model spends tokens on signal, not chrome.
Protected sites, handled
JavaScript rendering and bot-protection bypass kick in automatically when a page needs them.
Rich metadata
Title, description, language, status, and timing come back alongside the content on every call.
How it works
From URL to output in four steps.
01
Send a URL
POST the URL plus an optional format. There's no infrastructure to run and no proxies to manage on your side.
02
We fetch it
The engine fetches the page and automatically handles JavaScript rendering and bot protection only when the page needs it.
03
Content is cleaned
HTML is parsed and converted to your chosen format, with boilerplate removed.
04
Structured output returned
Clean content plus metadata comes back, ready to pass to any LLM, vector store, or pipeline.
API
One request, structured back.
POST /v1/scrape
POST the URL to /v1/scrape with the markdown format. webclaw fetches the page, strips nav, ads, and boilerplate, and returns GitHub-flavored markdown ready for an LLM, with no HTML parsing on your side.
Ship an agent that actually sees the web.
One credit pool, every endpoint. Cancel anytime, or self-host the open-source core for free.