Developer Documentation & Guides

1. Quickstart Guide

ScraperScoop offers two distinct methods for consuming data:

• Dataset Marketplace API: Query static or daily-updated dataset snapshots via REST API with simple pagination and filtering (`/api/v1/datasets`).
• Live Web Scraping API: Send real-time extraction requests for custom websites where our cloud workers render JavaScript and bypass anti-bots asynchronously.

2. Authentication & API Keys

All REST API requests require your enterprise API key passed in the `X-API-KEY` HTTP header, or as a standard `Authorization: Bearer ` token.

# Authenticate using X-API-KEY Header
curl -H "X-API-KEY: your_enterprise_api_key_here" https://scraperscoop.com/api/v1/datasets

3. Bulk CSV & Parquet Dumps

If you are training large-scale AI/LLM models or running nightly Snowflake jobs, you can download complete compressed CSV, JSON, or Apache Parquet files directly from your dataset dashboard or via pre-signed S3 URLs.

4. Webhook Push Ingestion

Instead of polling our servers, configure a secure webhook URL in your dashboard. Whenever a dataset finishes syncing or a custom scraping job completes, our workers immediately push an HTTP POST payload with verified HMAC signatures.

# Example Webhook Payload sent by ScraperScoop Workers to your endpoint
{
  "event": "dataset.updated",
  "dataset_slug": "amazon-global-product-review-dataset",
  "records_count": 15420000,
  "download_url": "https://scraperscoop.com/storage/exports/amazon_2026_07_15.csv.gz",
  "timestamp": "2026-07-15T12:00:00Z"
}