Crawl4AI MCP Server

Example Use Cases

• 1

  Scrape single web pages and extract markdown content.

• 2

  Perform multi-page web crawling with depth and page limits.

• 3

  Adaptive crawling to stop when sufficient content is gathered.

Description

This MCP server provides four powerful tools—scrape, crawl, crawl_site, and crawl_sitemap—that enable single-page scraping and multi-page crawling with adaptive stopping and safety features. It integrates seamlessly with AI agent frameworks like OpenAI Agents SDK, Cursor, and Claude Code, supporting both Docker and manual installation for flexible deployment. The server ensures safe crawling by blocking internal networks and offers fast, Playwright-powered extraction for efficient data gathering.