Exasol UDFs & Script Language Containers

UDFs extend SQL with custom logic that runs inside the Exasol cluster: per-row transforms, custom aggregation, ML inference against a model in BucketFS, external API calls, and DataFrame batch processing. They execute in a Script Language Container — a Docker-based runtime whose contents you can replace when the default packages are not enough.

Two Decisions Before Any Route

SCALAR or SET?

| | SCALAR | SET | |---|--------|-----| | Input | One row at a time | Group of rows (via GROUP BY) | | Output | RETURNS <type> (single value) | EMITS (col1 TYPE, ...) (zero or more rows) | | Row iteration | Not needed | ctx.next() loop required | | SQL usage | SELECT udf(col) FROM t | SELECT udf(col) FROM t GROUP BY key | | Use case | Per-row transforms | Aggregation, ML batch predict, multi-row emit |

Which language?

| Language | Startup | Best For | Expandable via SLC? | |----------|---------|----------|---------------------| | Python 3 (3.10 or 3.12) | ~200ms | ML, data science, pandas, string processing | Yes | | Java (11 or 17) | ~1s | Enterprise libs, type safety, Virtual Schema adapters | Yes | | Lua 5.4 | <10ms | Low-latency transforms, row-level security | No (natively compiled into Exasol) | | R (4.4) | ~200ms | Statistical modeling, R model deployment | Yes |

Routing Algorithm

Choose the narrowest matching route. Several often apply — a Python UDF that reads a model needs routes 1 and 2 — so load all matching references.

1. Write the CREATE SCRIPT statement — syntax for any language, variadic scripts

   • Trigger phrases: CREATE SCRIPT, SCALAR, SET EMITS, RETURNS, variadic script, dynamic parameters, EMITS(...), default_output_columns, %scriptclass, %jar, R UDF, Lua UDF
   • Load: references/create-script-syntax.md

2. Python UDF internals — context API, type mapping, DataFrames, BucketFS reads, testing

   • Trigger phrases: Python UDF, ctx.emit, ctx.next, get_dataframe, pandas, pickle, load model, udf-mock-python, dynamic import, memory limit
   • Load: references/udf-python.md

3. Java or Lua UDF internals — ExaIterator, ExaMetadata, JARs, JVM options, adapters, Lua libraries

   • Trigger phrases: Java UDF, ExaIterator, ExaMetadata, getString, %jar, JVM options, script imports, ADAPTER script, remote debugging, Lua context
   • Load: references/udf-java-lua.md

4. Build or deploy a Script Language Container — flavors, packages, activation, troubleshooting

   • Trigger phrases: SLC, Script Language Container, exaslct, exaslpm, packages.yml, custom packages, flavor, conda, CUDA, GPU UDF, SCRIPT_LANGUAGES, ALTER SESSION, ALTER SYSTEM, security-scan
   • Load: references/slc-reference.md

5. Orchestrate SQL from inside the database — not a UDF

   • Trigger phrases: execute script, Lua execute, pquery, in-database orchestration, iterative algorithm, multi-step SQL workflow
   • Load: references/lua-execute-scripts.md

Route 5 is a genuine fork, not a variant of the others: use a Lua execute script rather than a UDF when the task is to orchestrate multi-step or iterative SQL workflows from within the database.

Performance Notes

• Load once, use many: load models and other resources at module level, outside the row loop.
• Use SET for batching: collect rows into a list or DataFrame and process in bulk.
• Lua for low latency: avoids JVM and Python startup overhead entirely.
• Parallelism is automatic: UDFs run on all cluster nodes simultaneously.
• GPU work needs a CUDA SLC: the UDF API is unchanged, but the container and host driver are not — see references/slc-reference.md.

Related Skills

• exasol-bucketfs: uploading the JARs, models, and containers that UDFs read.
• exasol-distributed-ml: end-to-end distributed training and inference pipelines built on these UDFs.