DataHub Resource Management

Manage the full lifecycle of Alibaba Cloud DataHub resources (Project → Topic → Subscription) using the Aliyun CLI plugin mode.

Architecture: Region → Project → Topic → Subscription

Prerequisites

Pre-check: Aliyun CLI >= 3.3.3 required

Run aliyun version to verify >= 3.3.3. If not installed or version too low, see references/cli-installation-guide.md for installation instructions.

Pre-check: Aliyun CLI plugin update required

[MUST] run aliyun configure set --auto-plugin-install true to enable automatic plugin installation. [MUST] run aliyun plugin update to ensure that any existing plugins are always up-to-date.

Authentication

Pre-check: Alibaba Cloud Credentials Required

Security Rules:

NEVER read, echo, or print AK/SK values (e.g., echo $ALIBABA_CLOUD_ACCESS_KEY_ID is FORBIDDEN)

NEVER ask the user to input AK/SK directly in the conversation or command line

NEVER use aliyun configure set with literal credential values

ONLY use aliyun configure list to check credential status
aliyun configure list
Check the output for a valid profile (AK, STS, or OAuth identity).

If no valid profile exists, STOP here.

Obtain credentials from Alibaba Cloud Console

Configure credentials outside of this session (via aliyun configure in terminal or environment variables in shell profile)

Return and re-run after aliyun configure list shows a valid profile

RAM Policy

This skill requires DataHub permissions. See references/ram-policies.md for the full policy document.

Required actions: dhs:ListProjects, dhs:CreateProject, dhs:GetProject, dhs:UpdateProject, dhs:DeleteProject, dhs:ListTopics, dhs:CreateTopic, dhs:GetTopic, dhs:UpdateTopic, dhs:DeleteTopic, dhs:ListSubscriptions, dhs:CreateSubscription, dhs:GetSubscription, dhs:DeleteSubscription.

[MUST] Permission Failure Handling: When any command or API call fails due to permission errors at any point during execution, follow this process:

Read references/ram-policies.md to get the full list of permissions required by this SKILL

Use ram-permission-diagnose skill to guide the user through requesting the necessary permissions

Pause and wait until the user confirms that the required permissions have been granted

Parameter Confirmation

IMPORTANT: Parameter Confirmation — Before executing any command or API call, ALL user-customizable parameters (e.g., RegionId, instance names, CIDR blocks, passwords, domain names, resource specifications, etc.) MUST be confirmed with the user. Do NOT assume or use default values without explicit user approval.

| Parameter | Required | Description | Default | |-----------|----------|-------------|---------| | region | Yes | Target region for DataHub resources | — (must confirm) | | project-name | Yes | DataHub project name (3-32 chars, letter start, alphanumeric + underscore) | — | | topic-name | Yes (for Topic ops) | Topic name (3-128 chars) | — | | record-type | Yes (for create-topic) | BLOB or TUPLE | — | | shard-count | Yes (for create-topic) | Number of shards | — | | lifecycle | Yes (for create-topic) | Data retention in days | — | | record-schema | Conditional | Required when record-type is TUPLE | — | | application | Yes (for create-subscription) | Application description (max 256 chars) | — | | subscription-id | Optional | Manual subscription ID (4-40 chars, lowercase) | Auto-generated |

Observability (MUST follow for every aliyun command)

Upon loading this skill, generate a random session ID (32-char lowercase hex string) once for the entire session. Use it as {session-id} below.

Rule: Every aliyun CLI command that calls a cloud API MUST include the --user-agent flag. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

Example (assuming session-id is a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6):

aliyun datahub list-projects --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Do not skip, alter the format, or omit --user-agent on any aliyun API command invocation.

Core Workflow

Resource Dependency

Region
  └── Project
        └── Topic (BLOB / TUPLE)
              └── Subscription

Creation order: Project → Topic → Subscription Deletion order (reverse): Subscription → Topic → Project

Step 1: Create Project

# 1.1 List existing projects to check name conflicts
aliyun datahub list-projects \
  --region <region> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

# 1.2 Create project
aliyun datahub create-project \
  --region <region> \
  --project-name <project-name> \
  --comment "<project-description>" \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

# 1.3 Verify creation
aliyun datahub get-project \
  --region <region> \
  --project-name <project-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

Step 2: Create Topic

2.1 Create BLOB Topic (binary data)

aliyun datahub create-topic \
  --region <region> \
  --project-name <project-name> \
  --topic-name <blob-topic-name> \
  --comment "<topic-description>" \
  --shard-count <shard-count> \
  --record-type BLOB \
  --lifecycle <days> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

2.2 Create TUPLE Topic (structured data with schema)

aliyun datahub create-topic \
  --region <region> \
  --project-name <project-name> \
  --topic-name <tuple-topic-name> \
  --comment "<topic-description>" \
  --shard-count <shard-count> \
  --record-type TUPLE \
  --lifecycle <days> \
  --record-schema '{"fields":[{"name":"<field_name>","type":"STRING","notnull":"false"}]}' \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

2.3 Verify topic

aliyun datahub get-topic \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

2.4 Update topic description

aliyun datahub update-topic \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --comment "<new-description>" \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

Step 3: Create Subscription

# Option A: Auto-generated subscription-id
aliyun datahub create-subscription \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --application "<application-description>" \
  --comment "<subscription-description>" \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

# Option B: Manual subscription-id
aliyun datahub create-subscription \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --application "<application-description>" \
  --comment "<subscription-description>" \
  --subscription-id <subscription-id> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

Verify:

aliyun datahub get-subscription \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --subscription-id <subscription-id> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

Step 4: Resource Cleanup

[MUST] Delete in reverse dependency order.

⚠️ DESTRUCTIVE OPERATION — User Confirmation Required

Before executing ANY delete command (delete-subscription, delete-topic, delete-project), you MUST explicitly list the resources to be deleted and ask the user for confirmation. Do NOT proceed until the user explicitly approves the deletion.

Example prompt to user:

"I am about to delete the following resources (this action is irreversible):

Subscription: <subscription-id> on topic <topic-name>

Topic: <topic-name> in project <project-name>

Project: <project-name>

Do you confirm? (yes/no)"

4.1 Delete Subscription

aliyun datahub delete-subscription \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --subscription-id <subscription-id> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

4.2 Delete Topic (pre-check: no active subscriptions)

# Pre-check
aliyun datahub list-subscriptions \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

# Delete topic
aliyun datahub delete-topic \
  --region <region> \
  --project-name <project-name> \
  --topic-name <topic-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

4.3 Delete Project (pre-check: no active topics)

# Pre-check
aliyun datahub list-topics \
  --region <region> \
  --project-name <project-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

# Delete project
aliyun datahub delete-project \
  --region <region> \
  --project-name <project-name> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-datahub-manage/{session-id}

⚠️ WARNING: Deleting a Project removes all its Topics, data, and Subscriptions. This operation is irreversible. Always obtain explicit user confirmation before execution.

Success Verification

See references/verification-method.md for detailed verification steps for each operation.

Best Practices

Always verify after creation — Use get-project, get-topic, get-subscription to confirm resources exist.
Follow dependency order — Create: Project → Topic → Subscription; Delete: reverse.
Pre-check before deletion — List child resources before deleting parent resources.
Use meaningful names — Follow naming conventions (see references/related-commands.md).
Set appropriate lifecycle — Configure data retention (--lifecycle) based on business requirements.
Choose correct record-type — Use BLOB for binary streams, TUPLE for structured data requiring schema.
Confirm parameters — Always confirm region, project name, and other user-specific values before execution.

Reference Links

| Document | Description | |----------|-------------| | references/ram-policies.md | Required RAM permissions | | references/related-commands.md | Full CLI command reference | | references/verification-method.md | Success verification steps | | references/acceptance-criteria.md | Testing acceptance criteria | | references/cli-installation-guide.md | CLI installation guide | | DataHub Product Docs | DataHub product introduction | | DataHub Help Center | Detailed usage guide |