CLI Tool
Extraction from your terminal.
A single binary that scrapes, crawls, extracts structured data, runs web searches, deep research, and AI-guided agents. Built in Rust. Runs anywhere.
Install
Three ways to get started.
curl -fsSL https://github.com/0xMassi/webclaw/releases/latest/download/webclaw-aarch64-apple-darwin \ -o webclaw && chmod +x webclawPlatform support
Pre-built binaries are available for macOS (Apple Silicon & Intel), Linux (x86_64 & aarch64), and Windows (x86_64). Check the releases page for all targets.
Usage
Real commands you can run right now.
# Basic extractionwebclaw https://example.com# LLM-optimized formatwebclaw https://example.com --format llm# Filter contentwebclaw https://example.com --include "article" --exclude "nav,footer"LLM features
Extract, summarize, research, and agent commands require either a local Ollama instance (auto-detected at localhost:11434) or an OPENAI_API_KEY / ANTHROPIC_API_KEY environment variable.
Reference
Complete reference for every flag.
| Flag | Values | Description |
|---|---|---|
--format, -f | markdown, text, llm, json, html | Output format |
--include | CSS selectors | Only extract matching elements |
--exclude | CSS selectors | Remove matching elements |
--crawl | - | Enable BFS crawl mode |
--depth | 1-10 | Crawl depth (default: 2) |
--max-pages | number | Maximum pages to crawl |
--map | - | Discover URLs via sitemap |
--extract-prompt | string | LLM extraction prompt |
--summarize | - | Generate AI summary |
--brand | - | Extract brand identity |
--diff-with | file path | Diff against previous snapshot |
--search | "query" | Web search via Serper |
--research | "query" | Deep research with AI synthesis |
--agent | "goal" | AI-guided scraping with a goal |
--browser | chrome, firefox, safari | Browser profile to use |
--proxy-file | file path | Proxy list for rotation |
--timeout | seconds | Request timeout (default: 30) |
--output, -o | file path | Write output to file |
--verbose, -v | - | Verbose logging |
Try it now. No account required.
Install the binary, point it at a URL. Local extraction needs no sign-up.