Loading connector details…
Loading connector details…
Choose a unique username to continue using AgentHotspot
by seungwonme • Uncategorized
A web document crawling tool that recursively crawls documentation sites and saves them as markdown files.
Convert single web documentation pages into markdown format.
Recursively crawl entire documentation sites with control over depth and page limits.
Filter crawling by URL prefix and organize output by domain.
This MCP server is a Crawl4AI library-based web document crawler designed to convert documentation websites into markdown files. It supports single page crawling and deep recursive crawling with URL prefix filtering, automatic navigation/footer removal, and domain-based output directory structuring. It benefits users by enabling easy offline access and structured archiving of web documentation with configurable crawling depth and page limits.