Definition Resource Design Patterns

Guide for designing Definition resources that serve as reusable plugin templates. Definitions are the "Law" that Instances must follow.

When to Use

Apply when:

• Creating new plugin types (PipelineDef, StoreDef, etc.)
• Designing the interface contract a plugin must implement
• Specifying input/output schemas for validation
• Defining compatibility rules for upstream/downstream connections
• Embedding guardrail contracts that enforce data quality

Core Concept: The Law

Definition resources contain the rules that Instance resources must follow:

Definition = The Law
├── interface_spec        # Abstract interface to implement
├── input_schema          # Expected input format
├── output_schema         # Expected output format
├── compatibility_rules   # What can connect upstream/downstream
└── guardrail_contracts   # Validation rules (enforced by Engine)

Definition Types

| Type | Purpose | Example Implementations | |------|---------|------------------------| | PipelineDef | Data ingestion/transform | BloombergPipeline, FREDPipeline, ExpressionPipeline | | StoreDef | Storage backend | ParquetStore, SQLiteStore, VirtualStore | | AccessorDef | Data access pattern | SimpleAccessor, PITAccessor, FieldAccessor | | OpDef | Mathematical operator | REF, DELTA, MEAN, CORR, RANK | | OpMacroDef | Saved expression | User-defined formulas | | MLModuleDef | ML model template | XGBoostModule, LSTMModule | | PortfolioOptimizerDef | Optimization algo | MVO, HRP, BlackLitterman, RiskParity |

Schema Structure

See references/schema-structure.md for complete JSON schema.

Minimal Definition

definition_metadata = {
    "interface_spec": "optaic.interfaces.BasePipeline",
    "input_schema": {"datasets": {"type": "array"}},
    "output_schema": {"type": "DataFrame", "columns": ["date", "entity", "value"]},
}

Full Definition with Contracts

definition_metadata = {
    "interface_spec": "optaic.interfaces.SignalPipeline",
    "version": "1.0",

    "input_schema": {
        "datasets": {"type": "array", "items": {"$ref": "#/DatasetInstance"}},
        "parameters": {"type": "object", "properties": {...}}
    },

    "output_schema": {
        "type": "DataFrame",
        "columns": ["date", "entity", "value"],
        "value_range": {"min": -1, "max": 1}
    },

    "compatibility_rules": {
        "upstream_types": ["DatasetInstance"],
        "downstream_types": ["SignalInstance", "BacktestInstance"]
    },

    "guardrail_contracts": [
        {"kind": "signal.bounds", "config": {"min": -1, "max": 1, "allow_nan": False}},
        {"kind": "pit.policy", "config": {"knowledge_date_required": True}},
    ]
}

Implementation Checklist

1. [ ] Define interface_spec pointing to abstract base class
2. [ ] Specify input_schema with JSON Schema format
3. [ ] Specify output_schema with expected structure
4. [ ] Define compatibility_rules for resource graph connections
5. [ ] Embed guardrail_contracts for validation
6. [ ] Create extension table in libs/db/models/
7. [ ] Add to ResourceType enum

Reference Files

• Schema Structure - Complete JSON schema spec
• Examples - Example Definition implementations