SDK integration
Web scraping for Mastra
TypeScript-first AI framework with webclaw as a tool provider.
Mastra is a TypeScript framework for building AI applications with agents, workflows, and RAG. webclaw integrates as a tool that Mastra agents can call, providing web extraction with bot bypass and LLM-optimized output.
Setup
Mastra TypeScript — tool definition
typescript
import { createTool } from "@mastra/core";
import { Webclaw } from "@webclaw/sdk";
import { z } from "zod";
const wc = new Webclaw({ apiKey: process.env.WEBCLAW_API_KEY! });
export const scrapeWebTool = createTool({
id: "scrape-web",
description: "Scrape any URL and return clean markdown content",
inputSchema: z.object({
url: z.string().url(),
}),
execute: async ({ context }) => {
const result = await wc.scrape({
url: context.url,
formats: ["markdown"],
});
return { markdown: result.markdown };
},
});Why webclaw for Mastra
- TypeScript SDK with full type safety
- Fits Mastra's createTool pattern
- Fast enough for interactive workflow steps
- Bot bypass for production scraping
Common use cases
- Mastra agent workflows with web access
- TypeScript RAG pipelines
- Multi-step research workflows
- Real-time data fetching for LLM apps
Use createTool from @mastra/core and wrap a webclaw SDK call inside the execute function. Register the tool with your Mastra agent or workflow step.