MCP Webscan Server

Example Use Cases

• 1

  Fetch and convert web page content to Markdown for downstream analysis or summarization.

• 2

  Discover and crawl website content recursively to build indexes or inventories of pages.

• 3

  Validate and audit links, find URLs matching patterns, or generate XML sitemaps for SEO and monitoring.

Description

MCP Webscan Server provides tools to fetch web pages (converted to Markdown), extract and analyze links, recursively crawl sites, check for broken links, find URLs by pattern, and generate simple XML sitemaps. It is designed to be used as an MCP-compatible stdio server for agents like Claude Desktop and supports configurable depth and limits for crawling tasks. The project emphasizes robust error handling (invalid params, network and parsing errors) and is distributed under an MIT license for easy integration and development.