MCP SERVER

Give AI agents the web.

Name: webclaw
Author: webclaw

The Model Context Protocol connects Claude, Cursor, and any MCP-compatible AI agent directly to webclaw's extraction engine. One JSON config. Full web access. 8 tools your agent can call.

Quick setup

Pick your client. Paste the config. Done.

{
  "mcpServers": {
    "webclaw": {
      "command": "webclaw-mcp",
      "env": {
        "WEBCLAW_API_KEY": "<YOUR_API_KEY>"
      }
    }
  }
}

Installation

Two ways to get the binary.

Pre-built binary

bash

curl -fsSL https://github.com/0xMassi/webclaw/releases/latest/download/webclaw-mcp-aarch64-apple-darwin \
  -o webclaw-mcp && chmod +x webclaw-mcp

From source

bash

cargo install --git https://github.com/0xMassi/webclaw webclaw-mcp

Configuration

All configuration is through environment variables. Set them in your MCP client config or your shell.

WEBCLAW_API_KEY

Enables cloud fallback for bot-protected sites (Cloudflare, DataDome, AWS WAF)

OPENAI_API_KEY / ANTHROPIC_API_KEY

Enables extract and summarize tools (LLM features)

OLLAMA_HOST

Use local Ollama for LLM features (default: localhost:11434)

proxies.txt

Place a proxies.txt file in working directory for automatic proxy rotation

8 tools

Every tool your agent needs to read, extract, and monitor the web.

scrape

Extract content from any URL

url, format (markdown | text | llm | json)

crawl

Crawl entire sites with BFS traversal

url, depth, max_pages

map

Discover all URLs via sitemap parsing

url, include, exclude

batch

Extract multiple URLs concurrently

urls[], format, concurrency

extract

LLM-powered structured data extraction

url, prompt, schema

summarize

AI-generated page summaries

url, max_length

diff

Track content changes between snapshots

url, previous_snapshot

brand

Extract brand identity from any site

url (returns colors, fonts, logos)

Smart Fetch

Local-first architecture: 80% of scrapes cost zero API credits. The cloud API only activates for bot-protected or JS-rendered pages. Your agent gets full web access while keeping costs near zero.

Ready to connect?

Get an API key for cloud features, or use the MCP server fully local with zero configuration.

GET YOUR API KEY READ FULL DOCS