Spider AI Studio
AI-powered web data extraction APIs. Use natural language to crawl, scrape, search, and automate the web at scale.
AI Crawl
Crawl websites intelligently using natural language prompts. Let AI guide the crawler to find and extract exactly what you need.
Features
- Natural language crawl guidance
- Smart page discovery
- AI-powered content filtering
- Structured data extraction
Example prompt
"Find all product pages and extract pricing information"
POST /ai/crawlAI Scrape
Extract structured data from any webpage using plain English. Describe what you need, and AI handles the parsing.
Features
- Plain language data extraction
- Automatic schema detection
- Multi-format output
- Context-aware parsing
Example prompt
"Extract all article titles, authors, and publish dates"
POST /ai/scrapeAI Search
Search the web with AI-enhanced queries. Get smarter, more relevant results with natural language understanding.
Features
- Semantic search queries
- Intent understanding
- Relevance ranking
- Structured search results
Example prompt
"Find recent news about AI startups in healthcare"
POST /ai/searchAI Browser
Automate browser interactions using natural language. Configure complex web automation without writing scripts.
Features
- Natural language automation
- Smart element detection
- Action sequencing
- Dynamic page handling
Example prompt
"Click the login button, fill the email field, and submit"
POST /ai/browserAI Links
Extract and filter links intelligently using natural language. Find specific types of links without complex CSS selectors.
Features
- AI-guided link filtering
- Category detection
- Relevance ranking
- Deep link discovery
Example prompt
"Find all links to product pages and documentation"
POST /ai/linksSimple API Integration
All AI endpoints follow the same pattern as our standard APIs. Just add a prompt field to your request and let AI handle the complexity.
curl -X POST https://api.spider.cloud/ai/scrape \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"prompt": "Extract all product names and prices"
}'