OpenTelemetry Demo Architecture

Quick reference for the OpenTelemetry Demo microservices system. Focus on service dependencies, critical paths, and common failure patterns.

For detailed observability queries: See the Observability Query Guides section below for comprehensive metrics, traces, and logs references.

Service Dependency Matrix

| Service | Language | Depends On | Protocol | Memory Limit | |---------|----------|------------|----------|--------------| | frontend | TypeScript | ad, cart, checkout, currency, product-catalog, recommendation, shipping, image-provider | gRPC | 250M | | checkout | Go | cart, currency, email, payment, product-catalog, shipping, kafka | gRPC, HTTP | 20M | | cart | .NET | valkey-cart, flagd | - | 160M | | product-catalog | Go | flagd | - | 20M | | recommendation | Python | product-catalog, flagd | gRPC | 50M | | shipping | Rust | quote | HTTP | 20M | | payment | JavaScript | flagd | - | 120M | | ad | Java | flagd | gRPC | 300M | | email | Ruby | - | - | 100M | | currency | C++ | - | - | 20M | | quote | PHP | - | - | 40M | | fraud-detection | Kotlin | kafka, flagd | TCP | - | | product-reviews | Python | product-catalog, llm, postgresql, flagd | gRPC | - | | accounting | .NET | kafka, postgresql | TCP | - | | frontend-proxy | Envoy | frontend, flagd, flagd-ui, image-provider | HTTP | 65M | | image-provider | nginx | - | - | 120M | | load-generator | Python | frontend-proxy, flagd | HTTP | 120M |

Critical Service Paths

User Request Flow:

Internet → frontend-proxy:ENVOY_PORT → frontend → [cart, product-catalog, recommendation, checkout]

Checkout/Order Flow:

checkout → cart (gRPC)
        → currency (gRPC)
        → payment (gRPC)
        → product-catalog (gRPC)
        → shipping (HTTP) → quote (HTTP)
        → email (HTTP)
        → kafka → [accounting, fraud-detection]

Telemetry Flow:

All services → otel-collector:4317(gRPC)/4318(HTTP) → [jaeger, prometheus, tempo, opensearch]
                                                     → grafana (visualization)

Observability Stack Details

Metrics Pipeline

Services → otel-collector (OTLP) → prometheus (scrape/remote-write) → grafana

Traces Pipeline

Services → otel-collector (OTLP) → [jaeger, tempo] → grafana

Logs Pipeline

Services → docker json-file → alloy → loki → grafana

Log Configuration

All services use json-file driver:

Max size: 5M
Max files: 2
Auto-rotation

Observability Query Guides

Detailed reference guides for querying observability data:

Metrics Guide

See metrics-guide.md for:

Complete Prometheus metrics catalog - All 260+ available metrics organized by category
Service-specific metrics - Application metrics for each demo service
HTTP/RPC metrics - Request rates, latencies, error rates
Runtime metrics - Go, .NET, JVM, Node.js, Python runtime instrumentation
Container metrics - CPU, memory, network, disk I/O
Feature flag metrics - Flag evaluation and impression tracking
OTEL Collector metrics - Pipeline health and throughput
Common label patterns - Service identification, filtering, aggregation
Query patterns - Request rates, error rates, percentiles, top-N

When to use: Building dashboards, investigating performance issues, analyzing resource utilization, monitoring service health.

Traces Guide

See traces-guide.md for:

TraceQL syntax and patterns - Complete query language reference
Resource attributes - Service, host, process, runtime identification (27 attributes)
Span attributes - HTTP, gRPC, database, application-specific (150+ attributes)
Event attributes - Exceptions, feature flags, business events
Intrinsic attributes - Duration, status, kind, instrumentation
Service-specific attributes - Cart, product, payment, shipping, ad service patterns
Common query patterns - Error investigation, performance analysis, feature flag impact

When to use: Debugging distributed transactions, investigating latency issues, understanding service dependencies, tracing business transactions.

Logs Guide

See logs-guide.md for:

LogQL syntax and patterns - Complete query language reference
Available labels - service_name, container, project (5 labels)
Service identification - All 16 application and infrastructure services
Common query patterns - Errors, HTTP requests, performance, business events
Log parsing - JSON parsing, pattern extraction, label filtering
Aggregations - Count, rate, percentiles, error percentages
Multi-service analysis - Cross-service errors, communication tracing

When to use: Investigating errors, debugging application logic, analyzing request patterns, root cause analysis.