ScrapyMCP Web Scraper

Example Use Cases

• 1

  Scrape static web pages quickly and efficiently using HTTP requests.

• 2

  Extract dynamic content from JavaScript-rendered single-page applications.

• 3

  Perform batch scraping with concurrency control and retry mechanisms.

Description

ScrapyMCP is a versatile web scraping tool supporting Model Context Protocol (MCP). It offers simple HTTP scraping for static content, Puppeteer-based scraping for dynamic JavaScript-heavy sites, and batch scraping for concurrent multi-URL processing. Features include CSS selector extraction, proxy support, interaction automation, and real-time progress display, all with type-safe TypeScript implementation.