scrapers
Run hundreds of scrapers in parallel with intelligent load balancing. Monitor multiple websites, aggregate data, and detect changes with distributed scraping at 300+ edge locations.
scrapers
Scrapers.do
Overview
The scrapers primitive is part of the .do platform's semantic API system. It follows the $.Subject.verb.Object pattern for consistent, intuitive interactions across SDK, CLI, API, and MCP interfaces.
Semantic Pattern
All scrapers operations follow the semantic $.Scrapers.verb.Object pattern:
import { $, scrapers } from 'sdk.do'
// Create resources
await $.Scrapers.create.Resource({ /* config */ })
// Query data
await $.Scrapers.get.Data({ /* params */ })
// Update state
await $.Scrapers.update.Status({ /* changes */ })Quick Start
import { scrapers } from 'sdk.do'
// Initialize and use the primitive
const result = await scrapers.operation({
// configuration parameters
})Access Methods
The scrapers primitive is accessible through multiple interfaces:
SDK
TypeScript/JavaScript library for programmatic access:
import { scrapers } from 'sdk.do'
await scrapers.method({ /* ... */ })CLI
Command-line interface for terminal operations:
do scrapers <command> [options]API
REST/RPC endpoints for HTTP integration:
curl -X POST https://api.do/v1/scrapers \
-H "Content-Type: application/json" \
-d '{"key":"value"}'MCP
Model Context Protocol for AI assistant integration:
Use scrapers primitives with AI assistants via MCPRelated Primitives
See other primitives in the integration category.