Docs Crawler MCP Server

Example Use Cases

• 1

  Crawl and search documentation websites efficiently.

• 2

  Store and access text chunks and their embeddings from documentation sites.

• 3

  Perform vector similarity searches on locally crawled data.

Description

The Docs Crawler MCP Server allows users to crawl documentation websites, store content locally, and search through it using vector embeddings. It features crawling with Breadth-First Search, text chunking, TF-IDF embeddings, and storage in a Qdrant vector database. The server supports searching with query embeddings and a fallback to text search if needed.