shield_lock
JinDaGe
← Back to skills
extension
Category: Data & AnalyticsNo API key required

Scrapling 自适应网络爬虫框架

Scrapling 是一个自适应的 Python 网页抓取框架，能够自动处理网站结构变化。其解析器从页面更新中学习并重新定位元素，其获取器可以绕过 Cloudflare Turnstile 等反机器人系统，其蜘蛛框架支持通过代理轮换进行并发多会话爬取。

personAuthor: user_3c6cb52ehubcommunity
downloadDownload package

Scrapling 自适应网络爬虫框架

Scrapling is an adaptive Python web scraping framework that automatically handles website structure changes. Its parser learns from page updates and relocates elements, its fetchers bypass anti-bot systems like Cloudflare Turnstile, and its spider framework scales to concurrent multi-session crawls with proxy rotation.

Installation

Requirements and caveats from upstream:

  • <a href="https://pypi.org/project/scrapling/" alt="Supported Python versions">
  • <img alt="Supported Python versions" src="https://img.shields.io/pypi/pyversions/scrapling.svg"></a>
  • python

Basic usage or getting-started notes:

  • Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scal...

  • p = StealthyFetcher.fetch('https://example.com', headless=True, network_idle=True) # Fetch website under the radar!

  • start_urls = ["https://example.com/"]

  • Source: https://github.com/1991513ccie-png/skills

  • Extracted from upstream docs: https://raw.githubusercontent.com/D4Vinci/Scrapling/HEAD/README.md