shield_lock
JinDaGe
← Back to skills
extension
Category: Data & AnalyticsNo API key required

Fetch

Public web retrieval and clean extraction engine. Use whenever the user wants to fetch, download, inspect, clean, or save content from a public URL. Supports...

personAuthor: agistackhubclawhub
open_in_newRepositorydownloadDownload package

Fetch

Turn public URLs into usable local content.

Core Philosophy

  1. Fetch only public web content.
  2. Prefer clean extracted text over noisy raw HTML.
  3. Save both the raw response and structured extraction locally.
  4. Keep a simple local job history so previous fetches are easy to inspect.

Runtime Requirements

  • Python 3 must be available as python3
  • No external packages required

Safety Boundaries

  • Public URLs only
  • No login flows
  • No cookies or browser automation
  • No API keys or credentials
  • No external uploads or cloud sync
  • All fetched data is stored locally only

Local Storage

All data is stored under:

  • ~/.openclaw/workspace/memory/fetch/jobs.json
  • ~/.openclaw/workspace/memory/fetch/pages/

Key Workflows

  • Fetch URL: fetch_url.py --url "https://example.com"
  • Save cleaned output: save_output.py --url "https://example.com" --title "Example"
  • List history: list_jobs.py
  • Show job details: show_job.py --id JOB-XXXX

Scripts

| Script | Purpose | |---|---| | init_storage.py | Initialize local storage | | fetch_url.py | Fetch a public URL and extract content | | save_output.py | Save cleaned output with a custom title | | list_jobs.py | List previous fetch jobs | | show_job.py | Show one saved fetch job |