返回 MCP 目录
public公开dns本地运行

氧元素网络抓取工具

一个利用Oxylabs Web Scraper API的爬虫工具,能够灵活选择解析和渲染页面的选项,从而高效地从复杂网站中抓取和处理网页内容。

article

README

Oxylabs + MCP

Oxylabs MCP Server

The missing link between AI models and the real‑world web: one API that delivers clean, structured data from any site.

smithery badge pypi package Licence Verified on MseeP Coverage badge


Oxylabs Server MCP server

📖 Overview

The Oxylabs MCP server provides a bridge between AI models and the web. It enables them to scrape any URL, render JavaScript-heavy pages, extract and format content for AI use, bypass anti-scraping measures, and access geo-restricted web data from 195+ countries.

🛠️ MCP Tools

Oxylabs MCP provides two sets of tools that can be used together or independently:

Oxylabs Web Scraper API Tools

  1. universal_scraper: Uses Oxylabs Web Scraper API for general website scraping;
  2. google_search_scraper: Uses Oxylabs Web Scraper API to extract results from Google Search;
  3. amazon_search_scraper: Uses Oxylabs Web Scraper API to scrape Amazon search result pages;
  4. amazon_product_scraper: Uses Oxylabs Web Scraper API to extract data from individual Amazon product pages.

Oxylabs AI Studio Tools

  1. ai_scraper: Scrape content from any URL in JSON or Markdown format with AI-powered data extraction;
  2. ai_crawler: Based on a prompt, crawls a website and collects data in Markdown or JSON format across multiple pages;
  3. ai_browser_agent: Based on prompt, controls a browser and returns data in Markdown, JSON, HTML, or screenshot formats;
  4. ai_search: Search the web for URLs and their contents with AI-powered content extraction.

✅ Prerequisites

Before you begin, make sure you have at least one of the following:

  • Oxylabs Web Scraper API Account: Obtain your username and password from Oxylabs (1-week free trial available);
  • Oxylabs AI Studio API Key: Obtain your API key from Oxylabs AI Studio. (1000 credits free).

📦 Configuration

Environment variables

Oxylabs MCP server supports the following environment variables: | Name | Description | Default | |----------------------------|-----------------------------------------------|---------| | OXYLABS_USERNAME | Your Oxylabs Web Scraper API username | | | OXYLABS_PASSWORD | Your Oxylabs Web Scraper API password | | | OXYLABS_AI_STUDIO_API_KEY| Your Oxylabs AI Studio API key | | | LOG_LEVEL | Log level for the logs returned to the client | INFO |

Based on provided credentials, the server will automatically expose the corresponding tools:

  • If only OXYLABS_USERNAME and OXYLABS_PASSWORD are provided, the server will expose the Web Scraper API tools;
  • If only OXYLABS_AI_STUDIO_API_KEY is provided, the server will expose the AI Studio tools;
  • If both OXYLABS_USERNAME and OXYLABS_PASSWORD and OXYLABS_AI_STUDIO_API_KEY are provided, the server will expose all tools.

❗❗❗ Important note: if you don't have Web Scraper API or Oxylabs AI studio credentials, delete the corresponding environment variables placeholders. Leaving placeholder values will result in exposed tools that do not work.

Configure with uvx

  • Install the uvx package manager:
    # macOS and Linux
    curl -LsSf https://astral.sh/uv/install.sh | sh
    
    OR:
    # Windows
    powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
    
  • Use the following config:
    {
      "mcpServers": {
        "oxylabs": {
          "command": "uvx",
          "args": ["oxylabs-mcp"],
          "env": {
            "OXYLABS_USERNAME": "OXYLABS_USERNAME",
            "OXYLABS_PASSWORD": "OXYLABS_PASSWORD",
            "OXYLABS_AI_STUDIO_API_KEY": "OXYLABS_AI_STUDIO_API_KEY"
          }
        }
      }
    }
    

Configure with uv

  • Install the uv package manager:

    # macOS and Linux
    curl -LsSf https://astral.sh/uv/install.sh | sh
    

    OR:

    # Windows
    powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
    
  • Use the following config:

    {
      "mcpServers": {
        "oxylabs": {
          "command": "uv",
          "args": [
            "--directory",
            "/<Absolute-path-to-folder>/oxylabs-mcp",
            "run",
            "oxylabs-mcp"
          ],
          "env": {
            "OXYLABS_USERNAME": "OXYLABS_USERNAME",
            "OXYLABS_PASSWORD": "OXYLABS_PASSWORD",
            "OXYLABS_AI_STUDIO_API_KEY": "OXYLABS_AI_STUDIO_API_KEY"
          }
        }
      }
    }
    

Configure with Smithery Oauth2

  • Go to https://smithery.ai/server/@oxylabs/oxylabs-mcp;
  • Click Auto to install the Oxylabs MCP configuration for the respective client;
  • OR use the following config:
  {
    "mcpServers": {
      "oxylabs": {
        "url": "https://server.smithery.ai/@oxylabs/oxylabs-mcp/mcp"
      }
    }
  }
  • Follow the instructions to authenticate Oxylabs MCP with Oauth2 flow

Configure with Smithery query parameters

In case your client does not support the Oauth2 authentication, you can pass the Oxylabs authentication parameters directly in url

  {
    "mcpServers": {
      "oxylabs": {
        "url": "https://server.smithery.ai/@oxylabs/oxylabs-mcp/mcp?oxylabsUsername=OXYLABS_USERNAME&oxylabsPassword=OXYLABS_PASSWORD&oxylabsAiStudioApiKey=OXYLABS_AI_STUDIO_API_KEY"
      }
    }
  }

Manual Setup with Claude Desktop

Navigate to Claude → Settings → Developer → Edit Config and add one of the configurations above to the claude_desktop_config.json file.

Manual Setup with Cursor AI

Navigate to Cursor → Settings → Cursor Settings → MCP. Click Add new global MCP server and add one of the configurations above.

📝 Logging

Server provides additional information about the tool calls in notification/message events

{
  "method": "notifications/message",
  "params": {
    "level": "info",
    "data": "Create job with params: {\"url\": \"https://ip.oxylabs.io\"}"
  }
}
{
  "method": "notifications/message",
  "params": {
    "level": "info",
    "data": "Job info: job_id=7333113830223918081 job_status=done"
  }
}
{
  "method": "notifications/message",
  "params": {
    "level": "error",
    "data": "Error: request to Oxylabs API failed"
  }
}

🛡️ License

Distributed under the MIT License – see LICENSE for details.


About Oxylabs

Established in 2015, Oxylabs is a market-leading web intelligence collection platform, driven by the highest business, ethics, and compliance standards, enabling companies worldwide to unlock data-driven insights.

image

Made with ☕ by Oxylabs. Feel free to give us a ⭐ if MCP saved you a weekend.

✨ Key Features

Scrape content from any site
  • Extract data from any URL, including complex single-page applications
  • Fully render dynamic websites using headless browser support
  • Choose full JavaScript rendering, HTML-only, or none
  • Emulate Mobile and Desktop viewports for realistic rendering
Automatically get AI-ready data
  • Automatically clean and convert HTML to Markdown for improved readability
  • Use automated parsers for popular targets like Google, Amazon, and more
Bypass blocks & geo-restrictions
  • Bypass sophisticated bot protection systems with high success rate
  • Reliably scrape even the most complex websites
  • Get automatically rotating IPs from a proxy pool covering 195+ countries
Flexible setup & cross-platform support
  • Set rendering and parsing options if needed
  • Feed data directly into AI models or analytics tools
  • Works on macOS, Windows, and Linux
Built-in error handling and request management
  • Comprehensive error handling and reporting
  • Smart rate limiting and request management

Why Oxylabs MCP?  🕸️ ➜ 📦 ➜ 🤖

Imagine telling your LLM "Summarise the latest Hacker News discussion about GPT‑5" – and it simply answers.
MCP (Multi‑Client Proxy) makes that happen by doing the boring parts for you:

| What Oxylabs MCP does | Why it matters to you | |-------------------------------------------------------------------|------------------------------------------| | Bypasses anti‑bot walls with the Oxylabs global proxy network | Keeps you unblocked and anonymous | | Renders JavaScript in headless Chrome | Single‑page apps, sorted | | Cleans HTML → JSON | Drop straight into vector DBs or prompts | | Optional structured parsers (Google, Amazon, etc.) | One‑line access to popular targets |

mcp-name: io.github.oxylabs/oxylabs-mcp

help

运行方式说明

cloud

托管运行

托管运行通常表示这个 MCP Server 由服务方环境承载,用户一般按页面提供的连接方式或授权流程接入,不需要在本地长期启动一个 MCP 进程

  1. 打开服务方连接页
  2. 完成授权或复制端点
  3. 在 MCP 客户端中连接
terminal

本地运行 / 其它方式

本地运行通常需要用户在自己的电脑或服务器上安装依赖,把 server_config 复制到 MCP 客户端,并按 env_schema 补齐环境变量、密钥或其它配置

  1. 复制 server_config
  2. 安装所需依赖
  3. 补齐环境变量后重启客户端