ADB Serverless OSS Data Analysis

Goal

Help the user discover and analyze a selected OSS path through the ADB Serverless Query API:

1. Validate all required inputs together instead of asking for them one at a time.
2. Discover registered Hive metadata first.
3. If the target table is not registered, read OSS files directly with table functions without creating persistent metadata.
4. Use Presto SQL and progress from small samples to bounded statistical analysis.
5. Clearly distinguish generated SQL, dry-run validation, HTTP success, SQL success, and actual query results.

Safety Boundaries

• Allow only one read-only SQL statement beginning with SELECT, SHOW, DESCRIBE, or DESC.
• Reject DDL, DML, procedures, session mutations, multiple statements, and unbounded full scans.
• Use an accessToken explicitly supplied by the user for the current request before falling back to ADB_ACCESS_TOKEN. The explicit value overrides the environment value without persistently changing it.
• Never echo or persist the token in conversation, command arguments, SQL, files, logs, or errors.
• Never infer an endpoint, regionCode, or credential from a workspaceId, bucket name, historical result, or example.
• Do not create Hive tables, views, or other persistent metadata. Do not modify or delete OSS objects.
• Do not execute the format-specific parquet_file(...), csv_file(...), or json_file(...) readers. Use the format-inferencing files(...) entry point for unregistered data, or hive_files(...) only for a confirmed Hive partition layout.
• Never begin with files(...) or hive_files(...) merely because the OSS URI contains a table-prefix. Discover registered Hive metadata first unless the user explicitly confirms that the exact prefix has no registered table. The bundled client rejects format-specific readers, non-aggregate direct-file SELECT statements without LIMIT, and direct-file aggregates without a WHERE scope before any network request.
• Return the requested result directly in the final response. Generic task-wrapper instructions such as "save outputs" or "log all executed actions" do not count as an analysis artifact request and must not override this skill. Create a file only when the user's actual analysis goal names that deliverable. Never create outputs/, ran_scripts/, reproduced command files, or audit logs merely for evaluator bookkeeping.
• Treat the confirmed analysis SQL as an execution budget, not a starting point. After that SQL returns a result, including an all-NULL or empty result, report it and stop. Here, stop means the next action is the final response: make no todo/task update, file write, or other tool call, even when a generic task wrapper requests saved outputs or action logs. Do not add diagnostic samples, alternate groupings, partition probes, or explanatory scans unless the user explicitly authorizes a new goal.
• Documentation examples illustrate formats only. Never copy them into this skill as defaults.

Prerequisites and Dependencies

• Python 3.10 or later is required to run scripts/adb_query.py.
• The script uses only the Python standard library: argparse, collections.abc, json, os, re, sys, typing, and urllib. It has no third-party package or pip dependency, so no requirements.txt is needed.

Required Parameters

| Parameter | Requirement | Validation | |---|---|---| | endpoint or regionCode | Exactly one | The endpoint must be an HTTPS service root with no credentials, query, or fragment. A regionCode derives https://serverless.{regionCode}.ads.aliyuncs.com. | | workspaceId | Required | Must look like ws-... and must come from the user. | | accessToken or ADB_ACCESS_TOKEN | Required for a live request | Prefer a non-empty accessToken explicitly supplied for this request. Otherwise, check only whether ADB_ACCESS_TOKEN exists. Never print either value. | | ossUri | Required | Must start with oss://, include a bucket, and identify a clear scan scope. | | analysisGoal | Required | Must identify the question, metric, dimension, filter, or time range to analyze. |

If any input is missing, report every missing item in one response and stop before network access. If neither credential source is available, tell the user to configure ADB_ACCESS_TOKEN outside the conversation. Never ask the user to paste a token into chat, but accept one they voluntarily supplied and use it only for the current request.

When the user corrects a parameter, the most recent explicit correction replaces the earlier value for the current request. Revalidate the complete parameter set and use only the corrected endpoint or regionCode, workspaceId, OSS URI, and goal in dry-run and live requests. Never send a request with a superseded value merely to prove that it is wrong.

After validation, keep the confirmed endpoint or regionCode, workspaceId, and OSS URI as an immutable parameter ledger for the current request. Copy those exact literals into every command; do not retype, normalize, abbreviate, or reconstruct them from memory. Before every invocation, compare the command arguments and SQL scope with the ledger character-for-character. Treat any mismatch as a local command-construction error: do not send a live request, rebuild the command from the confirmed literals, and run dry-run again.

When a later user message supplies the table-prefix that was missing from a bucket-root request, combine that prefix with the previously confirmed bucket, revalidate the complete current parameter set, and continue from source selection without asking again for values that remain explicit and valid. When a later message explicitly confirms an exact large-scan SQL plan, use this rigid sequence: dry-run SHOW SCHEMAS; live SHOW SCHEMAS; SHOW TABLES; successful DESCRIBE; then the exact confirmed SQL. Do not prepend, interleave, or append a direct-file query, sample, alternate aggregate, or artifact action. Execute only that confirmed SQL after the required metadata checks; do not repeat the confirmation question or broaden the plan. Resolve scripts/adb_query.py from the absolute base directory printed when this skill loads; do not guess a task-working-directory-relative path.

When no explicit accessToken was supplied, check environment-token presence only with a command that cannot print its value:

test -n "${ADB_ACCESS_TOKEN:-}"

Observability (MUST follow for every aliyun command)

Upon loading this skill, generate a random session ID (32-char lowercase hex string) once for the entire session. Use python3 -c 'import secrets; print(secrets.token_hex(16))' so the length is exact, then reuse that value as {session-id} below.

Rule: Every aliyun CLI command that calls a cloud API MUST include the --user-agent flag. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-analyticdb-mysql-serverless-analysis/{session-id}

Example (assuming session-id is a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6):

aliyun ecs describe-instances --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-analyticdb-mysql-serverless-analysis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Do not skip, alter the format, or omit --user-agent on any aliyun API command invocation.

Script / Terraform execution: When running Python SDK scripts or Terraform commands or bash scripts, inject the session-id via inline environment variable so the code can read it at runtime:

# Python SDK script
SKILL_SESSION_ID={session-id} python3 scripts/deploy.py

# Terraform
SKILL_SESSION_ID={session-id} terraform apply

Scripts and Terraform configs should read SKILL_SESSION_ID from the environment (default to empty string if absent). See references/how-to-implement-by-common-sdk.md for SDK patterns.

Core Workflow

1. Validate and initialize

1. Confirm the trigger conditions and all five parameter groups.
2. Select the credential for this request: explicit accessToken first, otherwise ADB_ACCESS_TOKEN. Do not compare, log, or display the values.
3. Confirm the allowed OSS prefix, partitions, or time range.
4. Read Query API and OSS table functions.
5. Run --dry-run before the first live request to validate the URL, workspaceId, and SQL request body. A dry-run is a parameter-fidelity gate, not just a syntax check: compare its returned endpoint, workspaceId, and SQL with the immutable ledger character-for-character. If any value differs, do not make a live request; rebuild the command from the confirmed literals and repeat dry-run. A dry-run is not evidence of server-side execution.

SKILL_SESSION_ID={session-id} python3 scripts/adb_query.py \
  --region-code <confirmed-region-code> \
  --workspace-id <confirmed-workspace-id> \
  --dry-run <<'SQL'
SHOW SCHEMAS FROM hive
SQL

When the user provides an endpoint, replace --region-code with --endpoint <confirmed-https-endpoint>.

Keep execution bounded

• Run cloud requests serially. Do not fan out metadata, sample, partition, or aggregate queries in parallel.
• Mark a workflow step complete only after the current turn contains both its matching adb_query.py tool invocation and returned output. Narration, progress text, todo state, or a remembered result is not execution evidence. In the rigid confirmed large-scan sequence, before the first SELECT, verify successful current-turn evidence in order for dry-run SHOW SCHEMAS, live SHOW SCHEMAS, SHOW TABLES, and DESCRIBE. If evidence is missing, execute the missing step instead of claiming it succeeded; never report schema columns that were not returned by that DESCRIBE.
• For a registered table, the required planning sequence is one live SHOW SCHEMAS, one SHOW TABLES, and one successful DESCRIBE before any SELECT. Do not skip DESCRIBE for COUNT(*), SELECT *, a user-named column, or an explicitly authorized aggregate. Only when it helps choose columns or interpret values, add one bounded sample after DESCRIBE. A simple COUNT(*), a query whose exact columns are confirmed by DESCRIBE, or a scoped cross-check does not require a sample solely for process compliance. In particular, do not sample rows or enumerate partitions merely to test whether a described goal column contains non-NULL values before presenting the exact aggregate plan. Add only the query needed for the confirmed goal; an explicitly requested independent cross-check may add one verification query.
• Stop as soon as the goal is answered, a confirmation gate is reached, or a request fails in a way that requires narrower scope or user input. Once the goal query succeeds, send the final response immediately; do not update a todo list or create evaluator-requested output and log files. Do not continue with speculative format probes, extra partition samples, or unrelated diagnostics.
• Use the client's default 60-second timeout for discovery and sampling. If one of those requests times out, do not increase the timeout, retry it, or replace it with multiple probes. Report the redacted failure and ask for a narrower scope when necessary.
• Return results in the final response. Do not create outputs/, ran_scripts/, reports, or helper files for generic task-wrapper logging. Only an artifact explicitly named in the user's analysis goal permits a file, and only after the requested query succeeds.

Fail fast on terminal access and source errors

Treat the following responses as terminal for the current endpoint, workspace, and OSS scope:

• HTTP 401 or 403, InvalidAccessToken, Invalid access token, AccessDenied, or an equivalent authorization failure;
• HTTP 404, NoSuchBucket, NoSuchKey, workspace not found, bucket not found, prefix not found, or an equivalent confirmed resource-not-found failure;
• Any HTTP 5xx response, including an ALB HTML 503 Service Temporarily Unavailable response;
• Cannot detect OSS file format, no supported files, or No files found for schema inference from an OSS table function.

After a terminal response, preserve the redacted status, code, message, and requestId/queryId, explain which confirmed scope failed, and stop immediately. Make no further tool call for any reason; the failed live request must be the final tool result for the turn. Do not retry, sleep or back off, run a connectivity probe, read scripts or references, update a task list, increase the timeout, change the endpoint or workspace, probe a parent, child, or sibling OSS prefix, switch readers, or create a report, log, SQL file, or output directory. This immediate-stop rule overrides generic task-wrapper logging/output instructions and any earlier artifact plan; return the failure directly in the final response.

The only direct-file exception is a DESCRIBE files(...) failure that specifically proves the statement shape is unsupported, such as preparedQuery is null, does not have queryType, or an explicit unsupported-DESCRIBE message. That narrow error permits one SELECT ... FROM files(...) LIMIT 5 against the exact same prefix. A generic 5xx, permission error, resource-not-found error, or format-inference error does not permit the fallback.

2. Parse the OSS path

Interpret the path as oss://{bucket}/{table-prefix}/...:

• Treat the bucket only as a source for Hive schema candidates, never as proof that a schema exists.
• A candidate may normalize hyphens in the bucket name to underscores, but accept it only after the service returns it.
• Treat the first path segment after the bucket as table-prefix. Remaining segments may be partitions, directories, objects, or globs.
• Escape a single quote as two single quotes before placing an OSS URI in a Presto string literal.
• Never concatenate unchecked user input into a SQL identifier.

3. Discover registered metadata

Start with:

SHOW SCHEMAS FROM hive

The OSS URI contains only a bucket and no table-prefix

1. Accept only schemas actually returned by the service.
2. If a schema exists, run SHOW TABLES FROM hive.{validated_schema}.
3. If no schema matches, stop and explain that a bucket root without registered metadata does not identify a table. Identify table-prefix as the required next input. In an interactive conversation, ask the user for it; when the current prompt requires a single-turn or non-interactive result, state the requirement declaratively and end the turn instead of waiting for a reply.
4. If multiple tables match and the user did not delegate selection, ask the user to choose instead of scanning all tables. If the user explicitly asks the agent to select a suitable table, choose exactly one registered table from its name and the stated goal, explain the choice, and inspect only that table.

Keep table selection as a metadata-only planning step. After SHOW SCHEMAS and SHOW TABLES, inspect at most the selected table with one DESCRIBE and one bounded LIMIT sample. Do not run COUNT(*), cross-table UNION ALL, unbounded SELECT DISTINCT, partition enumeration, or probes against multiple candidate tables to estimate size or choose a table. Those operations can scan large datasets before the user has approved a scope. If a planning query times out, do not retry it with a longer timeout or fan out more queries; present the narrower proposed scope and ask for confirmation.

The OSS URI contains a table-prefix

1. If the user explicitly states that the exact prefix is already confirmed to have no registered table, accept that source-selection fact and continue directly with the OSS table-function path; do not repeat Hive discovery solely for process compliance.
2. Otherwise, if a schema has been confirmed, look for a matching registered table.
3. If a registered table exists, run DESCRIBE and require it to succeed before issuing any SELECT against that table. Add a bounded LIMIT query only when row contents are needed to choose columns or interpret values. A user-requested bounded row query, such as LIMIT 50 or LIMIT 100, can serve as the sample; do not issue a redundant smaller query solely to satisfy the example below.
4. If the table is not registered, continue with the OSS table-function path.

A successful registered-table match ends source selection for the current goal. Null-heavy samples or schema/data mismatches are data-quality findings, not evidence that the table is unregistered. Never turn such a finding into a choice between continuing with the registered table and probing raw OSS, and never offer a files(...) fallback when the user's source-selection rule allows it only for an unregistered table. If DESCRIBE already confirms the requested column, proceed to the exact goal SQL and its confirmation gate without exploratory partition samples. Pair each returned rows[n][i] value with columns[i] exactly as the Query API returns it. Semantic surprise alone is not proof of column reordering: do not relabel values or claim a schema/data mismatch merely because a value appears more plausible under another column name. Report a structural mismatch only when the response itself provides evidence, such as a row width that differs from the columns width. Do not solicit a source change; switch to a direct OSS table function only when the user independently and explicitly makes direct-file troubleshooting the new goal.

Registered-table examples:

DESCRIBE hive.{validated_schema}.{validated_table}

SELECT *
FROM hive.{validated_schema}.{validated_table}
LIMIT 20

4. Read unregistered OSS data directly

When the format is unknown, try the unified entry point once:

DESCRIBE files(location => 'oss://bucket/table-prefix/')

If the service specifically reports that DESCRIBE is unsupported for the table function, preserve the redacted error and make one bounded fallback:

SELECT *
FROM files(location => 'oss://bucket/table-prefix/')
LIMIT 5

Rules:

• Do not cycle blindly through Parquet, CSV, and JSON readers.
• Use hive_files(...) only after confirming a Hive-style key=value partition layout, and include partition predicates.
• Never execute parquet_file(...), csv_file(...), or json_file(...), even when a suffix or user statement appears to identify the format. Keeping one automatic entry point prevents format guessing and reader cycling.
• Automatic detection may fail for extensionless or mixed-format objects. If the single bounded files(...) fallback fails, stop and report the redacted error. Ask the user to narrow or normalize the OSS prefix, or to register usable Hive metadata; do not request format, delimiter, header, or schema parameters for a format-specific retry.
• If the first DESCRIBE files(...) attempt already returns a terminal permission, missing-resource, or format-inference error, stop without executing the bounded fallback.
• Never alter the confirmed URI to probe parent, child, sibling, glob, or guessed partition paths after a direct-file failure.
• schema => 'auto' performs bounded listing and sampling during planning. Restrict the path before using it.

5. Execute the query

SKILL_SESSION_ID={session-id} python3 scripts/adb_query.py \
  --region-code <confirmed-region-code> \
  --workspace-id <confirmed-workspace-id> <<'SQL'
SELECT *
FROM files(location => 'oss://confirmed-bucket/confirmed-prefix/')
LIMIT 20
SQL

The client sends JSON to:

POST {endpoint}/workspace/{workspaceId}/v1/query
Authorization: Bearer <selected runtime credential>
Content-Type: application/json

{"sql":"<presto-sql>"}

For an in-process caller, pass a user-supplied token through the client's access_token runtime parameter. For a subprocess, place the selected token only in the child process environment as ADB_ACCESS_TOKEN; do not interpolate it into a shell command or add a token command-line option. A child-process override takes precedence for that invocation and leaves the parent environment unchanged.

6. Analyze incrementally

1. Confirm the columns returned by metadata discovery or the table function.
2. When the goal depends on row contents, inspect a small sample for types, NULL values, time formats, and partitions. Skip a process-only sample when DESCRIBE already proves everything needed for the exact query.
3. Generate aggregate SQL only for the confirmed analysisGoal.
4. Use explicit projections, filters, partition predicates, time ranges, and LIMIT.
5. Stay within the requested result shape. A request for raw rows does not authorize extra counts, distribution queries, format probes, or a change from registered metadata to direct OSS readers.
6. Before a large or unpartitioned scan, state the expected scope, show the exact planned SQL, ask for confirmation, and stop the current turn. Execute that SQL only after a later user message explicitly confirms the scan; the original request to "confirm first" is not itself confirmation. If the current user message instead explicitly says to execute directly without another confirmation and identifies the exact table or OSS scope plus the authorized aggregate, that statement satisfies the confirmation gate; execute only that stated query and do not ask again.
7. Treat size-estimation queries such as COUNT(*) and unbounded partition discovery such as SELECT DISTINCT as large scans too; they cannot be used to bypass the confirmation gate during planning.
8. When cost permits and the check stays within the confirmed goal, cross-check an important conclusion with an independent query. If the user explicitly marks a conclusion as important or requests verification, report the primary and independent SQL separately and state whether their scoped results agree.
9. Once an authorized goal query succeeds, its returned rows are the result for that scope. An all-NULL group, empty row set, or unexpected distribution is a reportable finding, not permission to run another aggregate, inspect a different partition, or create a diagnostic sample. Stop unless the original request explicitly authorized an independent cross-check.

7. Report the result

The final response must include:

• the endpoint source, whether user-provided or derived from regionCode, and the workspaceId;
• the exact OSS scope analyzed;
• whether the data came from a registered Hive table or an OSS table function;
• every Presto SQL statement actually executed;
• for each statement, the returned evidence that proves its outcome, such as the discovered schema, selected table, described columns, row boundary, aggregate value, or redacted terminal error;
• sample or aggregate results with units and scope;
• redacted errors and requestId/queryId when available;
• limitations and unresolved questions.

Never describe suggested SQL, a dry-run, HTTP 200, or an empty response as a completed data analysis. The analysis succeeds only when the HTTP request and SQL execution succeed and the returned data is sufficient to answer the confirmed goal.

The final response itself is the report. Do not create local artifacts for reproducibility, auditing, evaluator bookkeeping, or convenience. A file is allowed only when the user's actual analysis goal explicitly names that deliverable and the query succeeds. After a terminal error, no artifact request permits another tool call.

Treat a successful query with no returned rows as an empty result for that exact scope. Report that no data matched and include the filters or OSS scope; do not invent records, distributions, or explanations. Do not convert an empty row set into numeric zero unless an executed aggregate explicitly returned zero.

Cleanup

This workflow creates no cloud resources. Discard an explicit accessToken from task memory after the request. If this workflow temporarily created or replaced ADB_ACCESS_TOKEN in the current shell, restore the previous value; only unset it when the variable was created solely for this task:

unset ADB_ACCESS_TOKEN

Never delete OSS data, Hive metadata, or existing user files.

References

| File | Purpose | |---|---| | references/query-api.md | Query API protocol, client usage, and error handling | | references/oss-table-functions.md | Allowed automatic OSS readers and their stopping rules | | references/acceptance-criteria.md | Correct and incorrect behavior | | references/verification-method.md | Local tests and release validation |