RAW HTTP — NO HEADLESS BROWSER OVERHEAD•MARKDOWN · JSON · HTML · LLM-READY FORMATS•MCP SERVER FOR AI AGENTS•TLS FINGERPRINT IMPERSONATION•EXTRACT · SUMMARIZE · DIFF · BRAND•SITEMAP DISCOVERY & DEEP CRAWLING•SELF-HOST OR USE OUR CLOUD API•BUILT IN RUST — FAST BY DEFAULT•DEEP RESEARCH — AI SYNTHESIZES REPORTS FROM 50+ SOURCES•WEB SEARCH — QUERY AND SCRAPE SEARCH RESULTS IN ONE CALL•AGENT SCRAPE — GIVE A GOAL, AI EXTRACTS WHAT YOU NEED•URL MONITORING — WATCH PAGES FOR CHANGES WITH WEBHOOKS•BONUS CREDITS — EARN FREE CREDITS BY STARRING AND REFERRING•RAW HTTP — NO HEADLESS BROWSER OVERHEAD•MARKDOWN · JSON · HTML · LLM-READY FORMATS•MCP SERVER FOR AI AGENTS•TLS FINGERPRINT IMPERSONATION•EXTRACT · SUMMARIZE · DIFF · BRAND•SITEMAP DISCOVERY & DEEP CRAWLING•SELF-HOST OR USE OUR CLOUD API•BUILT IN RUST — FAST BY DEFAULT•DEEP RESEARCH — AI SYNTHESIZES REPORTS FROM 50+ SOURCES•WEB SEARCH — QUERY AND SCRAPE SEARCH RESULTS IN ONE CALL•AGENT SCRAPE — GIVE A GOAL, AI EXTRACTS WHAT YOU NEED•URL MONITORING — WATCH PAGES FOR CHANGES WITH WEBHOOKS•BONUS CREDITS — EARN FREE CREDITS BY STARRING AND REFERRING•

MCP SERVER

Web scraping inside Claude Code

Let Claude Code read live docs, issues, specs, and sites while it works.

Claude Code supports MCP servers through its command-line configuration. Add webclaw-mcp and Claude Code can scrape, crawl, search, summarize, and extract live web content directly inside coding sessions without copy-pasting browser tabs into the prompt.

Setup

Claude Code MCP server

claude mcp add --transport stdio --scope user \
  --env WEBCLAW_API_KEY=wc_... \
  webclaw -- webclaw-mcp

# Verify inside Claude Code
/mcp

Why webclaw for Claude Code

  • Standard MCP server for Claude Code
  • 8 web tools available from one local server
  • Bot bypass for documentation, forums, and protected pages
  • Clean markdown output fits coding-agent context windows

Common use cases

  • Read current framework docs while editing code
  • Scrape API references and examples during implementation
  • Research GitHub issues, changelogs, and release notes
  • Extract structured data from public pages into code or tests

Frequently asked questions

How do I add webclaw to Claude Code?

Use claude mcp add with the stdio transport and pass WEBCLAW_API_KEY as an environment variable. The example above installs it at user scope so it is available across projects.

How do I check whether Claude Code sees the tools?

Run /mcp inside Claude Code. Claude Code shows configured MCP servers and their connection status there.

Get started

500 pages/month free. No credit card. Open source.

GET STARTEDCLAUDE CODE WEBSITE

OTHER INTEGRATIONS

LangChainLlamaIndexCrewAIMastraClaude DesktopCodexCursorn8nZapier