Keboola Component Tester

Testing approach

Three levels, in priority order:

1. Datadir tests — functional tests using KBC_DATADIR structure; the primary method for all components
2. Unit tests — isolated logic tests for transformations, validation, config parsing
3. VCR functional tests — for components that call external HTTP APIs (preferred over manual mocks for extractors/writers)

Choosing VCR vs mocks

Use VCR (keboola.datadirtest) when the component makes external HTTP calls:

• Records real API interactions once → replays deterministically in CI without credentials
• More realistic than hand-rolled mocks; catches API contract changes
• Required for extractors; recommended for writers that call external APIs

Use mocks (unittest.mock) when:

• Testing pure logic, transformation, or validation code
• Writing unit tests for individual functions
• The component has no HTTP calls (applications, pure transformations)

VCR setup workflow

1. Add keboola.datadirtest>=2.0.0 to pyproject.toml dev dependencies, run uv sync -U
2. Copy tests/test_functional.py from component-developer:component-defaults assets
3. Read component (src/component.py, src/configuration.py, component_config/configSchema.json) and build tests/setup/configs.json — see vcr-configs-format.md
4. For authenticated APIs: ask user for real credentials → secrets.json (verify it's in .gitignore)
5. Scaffold to record: uv run python -m keboola.datadirtest scaffold [--secrets secrets.json] [--chain-state]
6. Check cassettes for unsanitized dynamic values — see vcr-sanitizers.md
7. Update .gitignore, Dockerfile (COPY tests/ tests), and push.yml (use python -m pytest)
8. Verify: python -m pytest locally and in Docker

See vcr-quickstart.md for all scaffold commands. See vcr-troubleshooting.md for common failures.

References

| File | When to read | |------|-------------| | references/datadir-tests.md | Setting up datadir tests — directory structure, config.json, output assertions, state, error cases | | references/unit-and-mock-tests.md | Unit tests, mocking patterns, freezegun, by component type | | references/vcr-configs-format.md | Building configs.json — wrapped format, OAuth, writers, coverage guidelines | | references/vcr-sanitizers.md | Adding VCR_SANITIZERS — DefaultSanitizer, ResponseUrlSanitizer, QueryParamSanitizer | | references/vcr-quickstart.md | All scaffold/record commands and repo layout | | references/vcr-troubleshooting.md | Common VCR failures and fixes | | references/vcr-debug-from-platform.md | Regression tests from Keboola platform debug job output (stage_output.zip) |