webclaw-mcp (stdio)

Real-time web data for AI agents.

Give your AI agent live eyes on the web. One MCP install.

AI agents that can only read their training data are stuck in the past. webclaw exposes 14 web tools over the Model Context Protocol so Claude, Cursor, and any MCP-compatible agent can scrape, crawl, search, and extract from the live web in real time.

Try a tool call

View API docs

How it works

Build it step by step.

The real flow, one step at a time. Switch between TypeScript, Python, and cURL on any snippet.

Install the MCP server

Add webclaw over MCP (an API key unlocks the cloud tools) so the agent gets 14 web tools over stdio.

claude_desktop_config.json

{  "mcpServers": {    "webclaw": {      "command": "npx",      "args": ["-y", "@webclaw/mcp"],      "env": { "WEBCLAW_API_KEY": "wc_..." }    }  }}

Agent calls a tool

When the agent needs live data it invokes scrape, search, or extract instead of relying on stale training data.

// The agent invokes a tool when it needs live dataconst result = await webclaw.scrape({  url: "https://news.ycombinator.com",  formats: ["markdown"],});

Fetch and clean

webclaw fetches the URL, handles bot protection, and returns structured markdown or JSON.

// webclaw handled bot protection and returned clean contentconst markdown = result.markdown;const title = result.metadata?.title;console.log(markdown);

4
Reason over the result
The agent reads the clean output and continues its task with fresh, real-world context.

Try a tool call

Why webclaw

Built for ai agents.

Native MCP server, works with Claude Desktop, Cursor, Windsurf out of the box

Automatic bot-protection handling for sites that block plain requests

14 tools exposed: scrape, search, crawl, map, batch, extract, summarize, diff, brand, research, lead, lead_batch, list_extractors, vertical_scrape

118ms on static pages keeps agents responsive, no multi-second waits

Structured output (JSON, markdown) that agents can reason over

What you get

Everything this use case needs.

MCP server with 14 tools over stdio
Works with Claude Desktop, Claude Code, Cursor, Windsurf, OpenCode
Automatic bot-protection handling
Real-time web access (no stale training data)
Structured JSON or markdown output

Where it fits

Built for the messy parts.

Your AI agent can write code, analyze documents, and reason about complex problems, but it cannot read a webpage reliably. Most scraping tools fail on modern bot-protected sites, and integrating them requires custom middleware for every agent framework.

webclaw-mcp is a Model Context Protocol server that exposes 8 web extraction tools over stdio. Install it once and your AI agent gets scrape, crawl, search, batch, extract, summarize, diff, and screenshot tools that handle bot-protected sites automatically. No middleware, no custom code.

Common questions

Frequently asked questions

Which AI clients support webclaw's MCP server?

webclaw-mcp works with any MCP-compatible client: Claude Desktop, Claude Code, Cursor, Windsurf, OpenCode, Codex, Antigravity, and any custom agent built on the official MCP SDK.

Can my agent read bot-protected sites?

Yes. webclaw handles bot-protected sites automatically and returns real content from pages that block plain requests. Your agent does not need to configure anything, it just calls the tool and gets clean content back.

How do I give my Cursor agent live web access?

Add webclaw over MCP to your Cursor config (an API key unlocks the cloud tools), and your agent will have 14 web tools available immediately. No other setup required.

For AI agents

Or hand it to your agent.

Add the webclaw MCP server to Claude, Cursor, or any MCP client, then paste this prompt. The agent calls the webclaw tools and hands the result back to your model — no code to write.

PROMPT FOR YOUR AGENT

Using the webclaw tools, get me current real-world information on [your topic or question] instead of relying on what you already know. Start by calling search with the query [your search query] to find the most relevant live pages, then call scrape on the top results to pull their full, clean content. If you need specific structured fields from any page, call extract with a schema covering [the fields you care about, e.g. name, price, date, author]; if a source is long, call summarize to condense it first. Reason over the fresh results and reply with a short written answer that directly addresses my question, followed by a table of the key facts you found and a list of the source URLs you used so I can verify each one.

Set up the MCP server