shield_lock
JinDaGe
← Back to skills
extension
Category: Data & AnalyticsNo API key required

从网页中提取schema.org、Open Graph和JSON-LD元数据以用于索引

使用 extruct 从原始 HTML 中提取机器可读的元数据，以便代理可以在不依赖脆弱的全页解析的情况下对页面进行分类、去重或丰富。它最适合元数据采集工作流，而不适用于爬取整个网站或渲染 JavaScript 密集型页面。

personAuthor: user_3c6cb52ehubcommunity
downloadDownload package

从网页中提取schema.org、Open Graph和JSON-LD元数据以用于索引

Uses extruct to pull machine-readable metadata from raw HTML so an agent can classify, deduplicate, or enrich pages without brittle full-page parsing. It is best for metadata harvesting workflows, not for crawling an entire site or rendering JavaScript-heavy pages.

Prerequisites

Python 3 environment

Installation

Use the upstream install or setup path that matches your environment:

  • pip install extruct
  • pip install 'extruct[cli]'
  • pip install -r requirements-dev.txt

Requirements and caveats from upstream:

  • :target: https://pypi.python.org/pypi/extruct
  • .. _rdflib: https://pypi.python.org/pypi/rdflib/
  • First fetch the HTML using python-requests and then feed the response body to extruct::

Basic usage or getting-started notes:

  • ::
  • Source: https://github.com/1991513ccie-png/skills
  • Extracted from upstream docs: https://raw.githubusercontent.com/scrapinghub/extruct/HEAD/README.rst

Documentation

  • https://github.com/1991513ccie-png/skills