Scan to join WeChat groupObservability Knowledge Base
Quick reference for the three pillars of observability, instrumentation patterns, and SLI/SLO/SLA definitions in PHP applications.
Three Pillars Overview
┌─────────────────────────────────────────────────────────────────────────────┐
│ THREE PILLARS OF OBSERVABILITY │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ LOGS │ │ METRICS │ │ TRACES │ │
│ │ │ │ │ │ │ │
│ │ What happened │ │ How much/many │ │ How requests │ │
│ │ (discrete │ │ (aggregated │ │ flow through │ │
│ │ events) │ │ measurements) │ │ services) │ │
│ │ │ │ │ │ │ │
│ │ • Errors │ │ • Counters │ │ • Spans │ │
│ │ • Audit trail │ │ • Gauges │ │ • Context │ │
│ │ • Debug info │ │ • Histograms │ │ • Latency │ │
│ │ │ │ │ │ │ │
│ │ JSON structured │ │ Prometheus │ │ OpenTelemetry │ │
│ │ Monolog │ │ StatsD │ │ Jaeger/Zipkin │ │
│ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │ │
│ └─────────────────────┼──────────────────────┘ │
│ │ │
│ ┌─────────▼─────────┐ │
│ │ CORRELATION ID │ │
│ │ (links all three │ │
│ │ pillars) │ │
│ └───────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Structured Logging
JSON Log Format
| Field | Type | Description | Required |
|-------|------|-------------|----------|
| timestamp | ISO 8601 | When event occurred | Yes |
| level | string | RFC 5424 log level | Yes |
| message | string | Human-readable description | Yes |
| channel | string | Logger channel name | Yes |
| context | object | Structured event data | No |
| correlation_id | string | Request/trace identifier | Yes |
| service | string | Service/app name | Yes |
| environment | string | prod/staging/dev | Yes |
Log Levels (RFC 5424)
| Level | Code | When to Use | |-------|------|-------------| | EMERGENCY | 0 | System is unusable | | ALERT | 1 | Immediate action required | | CRITICAL | 2 | Critical conditions (component failure) | | ERROR | 3 | Runtime errors (not requiring immediate action) | | WARNING | 4 | Exceptional but handled conditions | | NOTICE | 5 | Normal but significant events | | INFO | 6 | Informational messages (request processed) | | DEBUG | 7 | Detailed debug information |
Monolog Context Processor
<?php
declare(strict_types=1);
namespace Infrastructure\Logging;
use Monolog\LogRecord;
use Monolog\Processor\ProcessorInterface;
final readonly class CorrelationIdProcessor implements ProcessorInterface
{
public function __construct(
private CorrelationIdHolder $holder,
) {}
public function __invoke(LogRecord $record): LogRecord
{
return $record->with(
extra: array_merge($record->extra, [
'correlation_id' => $this->holder->get(),
'service' => $_ENV['APP_SERVICE_NAME'] ?? 'unknown',
'environment' => $_ENV['APP_ENV'] ?? 'unknown',
]),
);
}
}
Correlation ID Holder
<?php
declare(strict_types=1);
namespace Infrastructure\Logging;
final class CorrelationIdHolder
{
private ?string $correlationId = null;
public function set(string $correlationId): void
{
$this->correlationId = $correlationId;
}
public function get(): string
{
if ($this->correlationId === null) {
$this->correlationId = uuid_create(UUID_TYPE_RANDOM);
}
return $this->correlationId;
}
}
Distributed Tracing
OpenTelemetry Concepts
| Concept | Description | |---------|-------------| | Trace | End-to-end journey of a request across services | | Span | Single unit of work within a trace (has start/end time) | | SpanContext | Trace ID + Span ID + flags, propagated across boundaries | | Attributes | Key-value metadata on spans | | Events | Timestamped annotations within a span | | Links | Connections between spans in different traces | | Baggage | Cross-cutting key-value pairs propagated with context |
W3C Trace Context Header
traceparent: {version}-{trace-id}-{parent-id}-{trace-flags}
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
tracestate: vendor1=value1,vendor2=value2
| Part | Length | Description |
|------|--------|-------------|
| version | 2 hex | Always 00 |
| trace-id | 32 hex | Globally unique trace identifier |
| parent-id | 16 hex | ID of parent span |
| trace-flags | 2 hex | 01 = sampled |
OpenTelemetry PHP SDK Setup
<?php
declare(strict_types=1);
namespace Infrastructure\Telemetry;
use OpenTelemetry\API\Globals;
use OpenTelemetry\API\Trace\SpanKind;
use OpenTelemetry\API\Trace\StatusCode;
use OpenTelemetry\API\Trace\TracerInterface;
final readonly class TracingService
{
private TracerInterface $tracer;
public function __construct(string $serviceName = 'my-app')
{
$this->tracer = Globals::tracerProvider()->getTracer($serviceName);
}
public function traceOperation(string $operationName, callable $operation, array $attributes = []): mixed
{
$span = $this->tracer
->spanBuilder($operationName)
->setSpanKind(SpanKind::KIND_INTERNAL)
->startSpan();
$scope = $span->activate();
try {
foreach ($attributes as $key => $value) {
$span->setAttribute($key, $value);
}
$result = $operation();
$span->setStatus(StatusCode::STATUS_OK);
return $result;
} catch (\Throwable $e) {
$span->setStatus(StatusCode::STATUS_ERROR, $e->getMessage());
$span->recordException($e);
throw $e;
} finally {
$scope->detach();
$span->end();
}
}
public function traceHttpClient(string $method, string $url, callable $request): mixed
{
$span = $this->tracer
->spanBuilder(sprintf('%s %s', $method, $url))
->setSpanKind(SpanKind::KIND_CLIENT)
->setAttribute('http.method', $method)
->setAttribute('http.url', $url)
->startSpan();
$scope = $span->activate();
try {
$result = $request();
$span->setStatus(StatusCode::STATUS_OK);
return $result;
} catch (\Throwable $e) {
$span->setStatus(StatusCode::STATUS_ERROR, $e->getMessage());
throw $e;
} finally {
$scope->detach();
$span->end();
}
}
}
Metrics
RED Method (Request-Driven Services)
| Metric | What | Unit | Example | |--------|------|------|---------| | Rate | Requests per second | req/s | HTTP requests per second by endpoint | | Errors | Failed requests per second | err/s | 5xx responses per second | | Duration | Latency distribution | ms | Response time p50, p95, p99 |
USE Method (Resource-Oriented)
| Metric | What | Example | |--------|------|---------| | Utilization | % time resource is busy | CPU usage, disk I/O | | Saturation | Queued work | Request queue length | | Errors | Error count | Disk errors, connection failures |
Golden Signals (Google SRE)
| Signal | Description | RED Equivalent | |--------|-------------|----------------| | Latency | Time to service a request | Duration | | Traffic | Demand on the system | Rate | | Errors | Rate of failed requests | Errors | | Saturation | How full the system is | (USE method) |
Prometheus PHP Client
<?php
declare(strict_types=1);
namespace Infrastructure\Metrics;
use Prometheus\CollectorRegistry;
use Prometheus\RenderTextFormat;
use Prometheus\Storage\Redis;
final class PrometheusMetricsCollector
{
private readonly CollectorRegistry $registry;
public function __construct(\Redis $redis)
{
$adapter = Redis::fromExistingConnection($redis);
$this->registry = new CollectorRegistry($adapter);
}
public function incrementRequestCount(string $method, string $route, int $statusCode): void
{
$counter = $this->registry->getOrRegisterCounter(
'app',
'http_requests_total',
'Total HTTP requests',
['method', 'route', 'status_code'],
);
$counter->inc([$method, $route, (string) $statusCode]);
}
public function observeRequestDuration(string $method, string $route, float $durationSeconds): void
{
$histogram = $this->registry->getOrRegisterHistogram(
'app',
'http_request_duration_seconds',
'HTTP request duration in seconds',
['method', 'route'],
[0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0],
);
$histogram->observe($durationSeconds, [$method, $route]);
}
public function setActiveConnections(int $count): void
{
$gauge = $this->registry->getOrRegisterGauge(
'app',
'active_connections',
'Current active connections',
[],
);
$gauge->set($count, []);
}
public function renderMetrics(): string
{
$renderer = new RenderTextFormat();
return $renderer->render($this->registry->getMetricFamilySamples());
}
}
SLI / SLO / SLA
| Concept | Definition | Example | |---------|------------|---------| | SLI (Service Level Indicator) | Measurable metric of service behavior | Request latency p99 < 200ms | | SLO (Service Level Objective) | Target value for an SLI | 99.9% of requests within 200ms | | SLA (Service Level Agreement) | Contract with consequences | 99.5% uptime or credit issued |
Common SLIs
| SLI Type | Formula | Target (SLO) |
|----------|---------|---------------|
| Availability | successful_requests / total_requests | 99.9% (three nines) |
| Latency | requests < threshold / total_requests | 99% < 200ms, 99.9% < 1s |
| Error Rate | error_requests / total_requests | < 0.1% |
| Throughput | requests / time_window | > 1000 req/s |
| Freshness | time_since_last_update | < 5 minutes |
Error Budget
Error Budget = 1 - SLO
Example: SLO = 99.9%
Error Budget = 0.1% = ~43 minutes/month downtime allowed
Budget remaining = Error Budget - Actual Errors
If budget exhausted → freeze deployments, focus on reliability
Quick Reference Tables
Observability Tool Selection
| Need | Tool/Library | PHP Integration |
|------|-------------|----------------|
| Structured logging | Monolog | monolog/monolog |
| Log aggregation | ELK Stack, Loki | Monolog handlers |
| Metrics collection | Prometheus | promphp/prometheus_client_php |
| Metrics visualization | Grafana | Prometheus data source |
| Distributed tracing | Jaeger, Zipkin | OpenTelemetry PHP SDK |
| APM | Datadog, New Relic | PHP extensions/agents |
| Error tracking | Sentry | sentry/sentry-php |
| Health checks | Custom endpoint | PSR-15 middleware |
Alerting Thresholds
| Alert | Condition | Severity | |-------|-----------|----------| | High error rate | > 1% of requests 5xx | Critical | | High latency | p99 > 2s for 5 min | Warning | | Service down | Health check fails 3x | Critical | | Disk usage | > 85% used | Warning | | Queue backlog | > 10k unprocessed | Warning | | Memory usage | > 90% for 10 min | Critical |
Common Violations Quick Reference
| Violation | Where to Look | Severity | |-----------|---------------|----------| | No structured logging (plain text) | Logger config, log output | Warning | | Missing correlation IDs | Middleware, log processors | Critical | | No metrics endpoint | Routes, health controllers | Warning | | Untraced external calls | HTTP clients, adapters | Warning | | Swallowed exceptions without logging | Catch blocks | Critical | | No health check endpoint | Routes, controllers | Warning | | Missing request/response logging | Middleware | Warning | | No alerting rules defined | Monitoring config | Warning |
Detection Patterns
# Logging setup
Grep: "Monolog|LoggerInterface|PsrLogLoggerInterface" --glob "**/*.php"
Grep: "monolog" --glob "**/composer.json"
Grep: "structured|json_formatter|JsonFormatter" --glob "**/*.php"
# Correlation IDs
Grep: "correlation.id|correlationId|X-Correlation-ID|X-Request-ID" --glob "**/*.php"
# Metrics
Grep: "Prometheus|CollectorRegistry|Counter|Histogram|Gauge" --glob "**/*.php"
Grep: "prometheus|promphp" --glob "**/composer.json"
Grep: "/metrics|metricsEndpoint" --glob "**/*.php"
# Tracing
Grep: "OpenTelemetry|Tracer|Span|SpanBuilder" --glob "**/*.php"
Grep: "open-telemetry|opentelemetry" --glob "**/composer.json"
Grep: "traceparent|tracestate|W3C" --glob "**/*.php"
# Health checks
Grep: "health|healthcheck|readiness|liveness" --glob "**/*.php"
Grep: "/health|/ready|/live" --glob "**/routes*.php"
# Error tracking
Grep: "Sentry|sentry|Bugsnag|Rollbar" --glob "**/*.php"
Grep: "sentry/sentry" --glob "**/composer.json"
# Log levels and context
Grep: "->error\(|->critical\(|->warning\(|->info\(" --glob "**/*.php"
Grep: "LogLevel::" --glob "**/*.php"
References
For detailed information, load these reference files:
references/logging-patterns.md— Structured logging, Monolog setup, context processors, log aggregation patternsreferences/metrics-patterns.md— Counter/Gauge/Histogram types, Prometheus PHP client, RED metrics, alerting rulesreferences/tracing-patterns.md— OpenTelemetry PHP SDK, span creation, context propagation, sampling strategies