Map
Discover URLs through sitemaps, search, and available indexed results. A response is a page of discovered URLs, not proof that the entire site was found.
POST
/v1/mapDiscover all URLs on a site via sitemap parsing.
Request body
json
{
"url": "https://docs.example.com"
}| Field | Type | Required | Description |
|---|---|---|---|
url | string | Yes | Base URL of the site to map. |
search | string | No | Optional term for the site search used during discovery. |
limit | number | No | Maximum URLs per response. Use the returned cursor to continue. |
cursor | string | No | Opaque next_cursor from an earlier response; keep the same site URL. |
Response
json
{
"urls": [
"https://docs.example.com",
"https://docs.example.com/getting-started",
"https://docs.example.com/api/reference",
"https://docs.example.com/guides/authentication",
"https://docs.example.com/guides/deployment"
],
"count": 5,
"next_cursor": null,
"total_indexed": 5,
"cached": false
}| Field | Type | Description |
|---|---|---|
urls | string[] | URLs returned on this page of discovery results. |
count | number | Number of URLs in this response. |
Note
Use
search to narrow discovery and limit for the page size (default 5,000, maximum 50,000). When next_cursor is non-null, send it as cursor with the same URL to continue. total_indexed describes the index, not a verified site total.Example
curl
curl -X POST https://api.webclaw.io/v1/map \
-H "Authorization: Bearer wc_your_api_key" \
-H "Content-Type: application/json" \
-d '{"url": "https://docs.stripe.com"}'Tip
Use
/v1/map to build a URL list, then feed it to /v1/batch for bulk extraction. This is faster than crawling when the site has a comprehensive sitemap.