Structured results
Every match comes back as title, URL, snippet, and ranking position, ready to feed straight into your model.
POST /v1/search
Web search your agent can act on.
Run a query, get structured results, and scrape every page in the same call.
Give your agent live web results it can reason over and act on, not a wall of HTML. Send a query, get back ranked titles, URLs, and snippets, and flip on scrape to pull clean, LLM-ready content from each result in one request.
What you get
Everything in one call.
Scrape in one call
Pass scrape and each result returns clean, LLM-ready content alongside its metadata, no second round trip.
Topic targeting
Pick general, news, or finance to shift ranking and source selection toward what your agent actually needs.
Tunable result count
Ask for anything from one result to a hundred so you only pull the depth your workflow can use.
How it works
From URL to output in four steps.
01
Send a query
POST a query string with an optional num and topic to point the search at the right scope.
02
We fetch results
Webclaw runs the search and ranks the matches into a clean, structured list.
03
Scrape on demand
Set scrape and each result is fetched and reduced to LLM-ready content, JS rendering and bot protection handled for you.
04
Results returned
You get a JSON array of titles, URLs, snippets, and positions your agent can act on immediately.
API
One request, structured back.
POST /v1/search
Search: retrieval augmented generation
-
What is RAG? - Retrieval-Augmented Generation AI Explained - AWS RAG is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data s
-
Retrieval-augmented generation - Wikipedia Retrieval-augmented generation (RAG) is a technique that enables large language models (LLMs) to retrieve and incorporate new information.
-
What Is Retrieval-Augmented Generation aka RAG - NVIDIA Blog Retrieval-augmented generation (RAG) is a technique for enhancing the accuracy and reliability of generative AI models with facts fetched ...
-
What is Retrieval-Augmented Generation (RAG)? - Google Cloud Retrieval-augmented generation (RAG) combines LLMs with external knowledge bases to improve their outputs. Learn more with Google Cloud.
Common questions
Frequently asked questions
what is a web search API for AI agents?
It is an endpoint that takes a plain query and returns ranked web results as structured JSON, with title, URL, snippet, and position for each match. Unlike a browser search page, the output is built for an LLM to read directly, and you can ask Webclaw to scrape each result into clean content in the same call.
can I scrape the search results in the same request?
Yes. Pass scrape in the request body and every result comes back with clean, LLM-ready content alongside its metadata, so your agent gets the full page text without a second round trip. JS rendering and bot protection are handled automatically.
how do I limit or target webclaw search results?
Use num to set how many results you want, from 1 to 100, defaulting to 10. Use topic to choose general, news, or finance, which shifts the ranking and source selection toward that domain.
how much does the webclaw search API cost?
Search runs from one credit pool. A query costs 2 credits per 10 results requested. If you turn on scrape, each fetched result adds 1 credit, or a few more when JS rendering or protected-site access is needed.
Am I billed for failed requests?
No. Credits are only consumed on successful responses. A standard page is 1 credit; heavier work like JS rendering or protected-site access costs a few extra credits.
Ship an agent that actually sees the web.
One credit pool, every endpoint. Cancel anytime, or self-host the open-source core for free.