Scrapling Fetch MCP

What is Scrapling Fetch MCP?

Scrapling Fetch MCP provides a lightweight MCP server that retrieves text‑only content (HTML or markdown) from websites that employ anti‑automation measures. It bridges the gap between what a human sees in a browser and what an LLM can access, focusing on low‑volume documentation and reference material.

How to Use Scrapling Fetch MCP?

Install (requires Python 3.10+ and uv):

uv tool install scrapling
scrapling install
uv tool install scrapling-fetch-mcp

Configure the MCP server in the Claude (or other LLM) client:

{
  "mcpServers": {
    "Cyber-Chitta": {
      "command": "uvx",
      "args": ["scrapling-fetch-mcp"]
    }
  }
}

Invoke the provided tools via MCP function calls:
- s-fetch-page – retrieve a full page (with optional pagination).
- s-fetch-pattern – extract text matching a regular‑expression pattern, with surrounding context.

Key Features

Protection Levels – basic, stealth, and max‑stealth modes trade speed for success on heavily guarded sites.
Two Dedicated Tools – complete page fetching with pagination support and pattern‑based extraction.
Context‑Aware Results – pattern matches include position data for follow‑up page fetches.
Low‑Volume Optimized – designed for documentation, articles, and reference material rather than bulk scraping.
Easy Integration – callable directly from LLM‑driven workflows via MCP.

Use Cases

Summarizing technical documentation behind Cloudflare or similar bot‑detectors.
Pulling API reference sections for on‑the‑fly code assistance.
Extracting specific legal or policy statements from protected sites.
Providing up‑to‑date product manuals to chat‑based support agents.

FAQ

Q: Do I need a browser driver? A: No. Scrapling handles the anti‑automation challenges internally; no Selenium or Playwright setup is required.

Q: Can it log in to sites requiring authentication? A: Authentication is not supported; the tool works with publicly accessible pages.

Q: What is the typical response time? A: basic mode returns in 1–2 seconds, stealth in 3–8 seconds, and max‑stealth may take 10 seconds or more.

Q: Is high‑volume scraping allowed? A: The project is intentionally limited to low‑volume, documentation‑type retrieval and should not be used for mass data harvesting.

Q: How does pagination work? A: Use the start_index and max_length parameters of s-fetch-page to fetch large documents in chunks.

An MCP server that helps AI assistants access text content from websites that implement bot detection, bridging the gap between what you can see in your browser and what the AI can access.

Intended Use

This tool is optimized for low-volume retrieval of documentation and reference materials (text/HTML only) from websites that implement bot detection. It has not been designed or tested for general-purpose site scraping or data harvesting.

Note: This project was developed in collaboration with Claude Sonnet 3.7, using LLM Context.

Installation

Requirements:
- Python 3.10+
- uv package manager
Install dependencies and the tool:

uv tool install scrapling
scrapling install
uv tool install scrapling-fetch-mcp

Setup with Claude

Add this configuration to your Claude client's MCP server configuration:

{
  "mcpServers": {
    "Cyber-Chitta": {
      "command": "uvx",
      "args": ["scrapling-fetch-mcp"]
    }
  }
}

Available Tools

This package provides two distinct tools:

s-fetch-page: Retrieves complete web pages with pagination support
s-fetch-pattern: Extracts content matching regex patterns with surrounding context

Example Usage

Fetching a Complete Page

Human: Please fetch and summarize the documentation at https://example.com/docs

Claude: I'll help you with that. Let me fetch the documentation.

<mcp:function_calls>
<mcp:invoke name="s-fetch-page">
<mcp:parameter name="url">https://example.com/docs</mcp:parameter>
<mcp:parameter name="mode">basic</mcp:parameter>
</mcp:invoke>
</mcp:function_calls>

Based on the documentation I retrieved, here's a summary...

Extracting Specific Content with Pattern Matching

Human: Please find all mentions of "API keys" on the documentation page.

Claude: I'll search for that specific information.

<mcp:function_calls>
<mcp:invoke name="s-fetch-pattern">
<mcp:parameter name="url">https://example.com/docs</mcp:parameter>
<mcp:parameter name="mode">basic</mcp:parameter>
<mcp:parameter name="search_pattern">API\s+keys?</mcp:parameter>
<mcp:parameter name="context_chars">150</mcp:parameter>
</mcp:invoke>
</mcp:function_calls>

I found several mentions of API keys in the documentation:
...

Functionality Options

Protection Levels:
- basic: Fast retrieval (1-2 seconds) but lower success with heavily protected sites
- stealth: Balanced protection (3-8 seconds) that works with most sites
- max-stealth: Maximum protection (10+ seconds) for heavily protected sites
Content Targeting Options:
- s-fetch-page: Retrieve entire pages with pagination support (using start_index and max_length)
- s-fetch-pattern: Extract specific content using regular expressions (with search_pattern and context_chars)
  - Results include position information for follow-up queries with s-fetch-page

Tips for Best Results

Start with basic mode and only escalate to higher protection levels if needed
For large documents, use the pagination parameters with s-fetch-page
Use s-fetch-pattern when looking for specific information on large pages
The AI will automatically adjust its approach based on the site's protection level

Limitations

Designed only for text content: Specifically for documentation, articles, and reference materials
Not designed for high-volume scraping or data harvesting
May not work with sites requiring authentication
Performance varies by site complexity

License

Apache 2

Scrapling Fetch MCP