Claude Code Plugins

Community-maintained marketplace

Feedback

observability-instrumentation

@VilnaCRM-Org/core-service
0
0

Add comprehensive observability to new code with structured logs (correlation ID), metrics (latency, errors, RPS), and traces around DB/HTTP calls. Use when implementing new features, adding command handlers, creating API endpoints, or instrumenting existing code for production monitoring. Automatically attaches evidence to PRs.

Install Skill

1Download skill
2Enable skills in Claude

Open claude.ai/settings/capabilities and find the "Skills" section

3Upload to Claude

Click "Upload skill" and select the downloaded ZIP file

Note: Please verify skill by going through its instructions before using it.

SKILL.md

name observability-instrumentation
description Add business metrics using AWS EMF (Embedded Metric Format) to API endpoints. Focus on domain-specific metrics only - AWS AppRunner provides default SLO/SLA metrics. Use when implementing new endpoints, adding command handlers, or instrumenting business events.

Business Metrics with AWS EMF

Instrument API endpoints with business metrics using AWS CloudWatch Embedded Metric Format (EMF). This skill focuses exclusively on domain-specific metrics - AWS AppRunner already provides infrastructure SLO/SLA metrics automatically.

What This Skill Covers

  • Business metrics - Domain events (customers created, orders placed, payments processed)
  • AWS EMF format - Logs that automatically become CloudWatch metrics
  • Event subscribers - Metrics emitted via domain event subscribers (not in handlers)
  • Type-safe metrics - Concrete metric classes instead of arrays
  • SOLID principles - Single Responsibility (subscribers) + Open/Closed (new metric classes)

What This Skill Does NOT Cover

  • Infrastructure metrics - Latency, error rates, RPS (AWS AppRunner provides these)
  • SLO/SLA metrics - Availability, response times (AWS AppRunner provides these)
  • Distributed tracing - Use AWS X-Ray integration instead

When to Use This Skill

Use this skill when:

  • Implementing new API endpoints that have business significance
  • Adding domain events that should trigger metric emission
  • Tracking domain events for analytics and business intelligence
  • Building dashboards for business KPIs

Architecture Overview

Business metrics follow these patterns:

  1. Metric classes - Each metric type is a concrete class extending BusinessMetric
  2. Event subscribers - Metrics are emitted via domain event subscribers (not hardcoded in handlers)
  3. Symfony logger - EMF output goes through Monolog with a custom EMF formatter
  4. No arrays - All metric configuration uses typed objects, not arrays
  5. Collections - Multiple metrics use MetricCollection, not arrays

SOLID Principles in Observability

Single Responsibility Principle (SRP)

Each class has ONE responsibility:

Class Responsibility
CustomersCreatedMetric Define metric name, value, dimensions
CustomerCreatedMetricsSubscriber Listen to event, emit metric
AwsEmfBusinessMetricsEmitter Format and write EMF logs
MetricCollection Hold multiple metrics for batch emission

Anti-pattern: Metrics emitted directly in command handlers (violates SRP - handler should only handle commands)

Open/Closed Principle (OCP)

  • Open for extension: Add new metrics via new classes
  • Closed for modification: Don't change existing metric/emitter code
// ✅ GOOD: Add new metric by creating new class
final readonly class OrdersPlacedMetric extends EndpointOperationBusinessMetric { ... }

// ❌ BAD: Modify existing emitter to handle new metric type

Why Event Subscribers (Not Handler Injection)

// ❌ BAD: Metrics in command handler (violates SRP)
final class CreateCustomerHandler
{
    public function __construct(
        private CustomerRepository $repository,
        private BusinessMetricsEmitterInterface $metrics  // Wrong!
    ) {}
}

// ✅ GOOD: Metrics in dedicated event subscriber
final class CustomerCreatedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __invoke(CustomerCreatedEvent $event): void
    {
        $this->metricsEmitter->emit($this->metricFactory->create());
    }
}

Benefits:

  • Handler focuses on domain logic only
  • Metrics emission is decoupled and testable
  • Easy to add/remove metrics without touching business logic
  • Multiple subscribers can react to same event

Type-Safe Metric Class Hierarchy

BusinessMetric (abstract)
├── EndpointOperationBusinessMetric (abstract) - for metrics with Endpoint/Operation dimensions
│   ├── CustomersCreatedMetric
│   ├── CustomersUpdatedMetric
│   ├── CustomersDeletedMetric
│   └── EndpointInvocationsMetric
└── (other base classes for different dimension patterns)

MetricDimensionsInterface
├── EndpointOperationMetricDimensions - Endpoint + Operation
└── (custom dimensions for specific metrics)

MetricDimensions - typed collection of MetricDimension objects
MetricDimension - key/value pair

MetricUnit (enum)
├── COUNT, NONE, SECONDS, MILLISECONDS, BYTES, PERCENT

MetricCollection - typed collection implementing IteratorAggregate, Countable

Why no arrays?

Arrays Typed Classes
No type safety Full type checking
No IDE autocomplete IDE support
Runtime errors Compile-time errors
Hard to refactor Easy to refactor
No encapsulation Validation in constructor

Current Implementation

Metric Base Class (Application Layer)

// src/Shared/Application/Observability/Metric/BusinessMetric.php
abstract readonly class BusinessMetric
{
    public function __construct(
        private float|int $value,
        private MetricUnit $unit
    ) {}

    abstract public function name(): string;
    abstract public function dimensions(): MetricDimensionsInterface;

    public function value(): float|int { return $this->value; }
    public function unit(): MetricUnit { return $this->unit; }
}

Concrete Metric Example

// src/Core/Customer/Application/Metric/CustomersCreatedMetric.php
final readonly class CustomersCreatedMetric extends EndpointOperationBusinessMetric
{
    private const ENDPOINT = 'Customer';
    private const OPERATION = 'create';

    public function __construct(
        MetricDimensionsFactoryInterface $dimensionsFactory,
        float|int $value = 1
    ) {
        parent::__construct($dimensionsFactory, $value, MetricUnit::COUNT);
    }

    public function name(): string
    {
        return 'CustomersCreated';
    }

    protected function endpoint(): string
    {
        return self::ENDPOINT;
    }

    protected function operation(): string
    {
        return self::OPERATION;
    }
}

Emitter Interface (Application Layer)

// src/Shared/Application/Observability/Emitter/BusinessMetricsEmitterInterface.php
interface BusinessMetricsEmitterInterface
{
    public function emit(BusinessMetric $metric): void;
    public function emitCollection(MetricCollection $metrics): void;
}

Metrics Event Subscriber

// src/Core/Customer/Application/EventSubscriber/CustomerCreatedMetricsSubscriber.php
/**
 * Error handling is automatic via DomainEventMessageHandler in async workers.
 * Subscribers stay clean - failures are logged + emit metrics automatically.
 * This ensures observability never breaks the main request (AP from CAP theorem).
 */
final readonly class CustomerCreatedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __construct(
        private BusinessMetricsEmitterInterface $metricsEmitter,
        private CustomersCreatedMetricFactoryInterface $metricFactory
    ) {
    }

    public function __invoke(CustomerCreatedEvent $event): void
    {
        $this->metricsEmitter->emit($this->metricFactory->create());
    }

    public function subscribedTo(): array
    {
        return [CustomerCreatedEvent::class];
    }
}

AWS EMF Format

AWS Embedded Metric Format allows you to embed custom metrics in structured log events. CloudWatch automatically extracts metrics from EMF-formatted logs.

EMF Log Structure

{
  "_aws": {
    "Timestamp": 1702425600000,
    "CloudWatchMetrics": [
      {
        "Namespace": "CCore/BusinessMetrics",
        "Dimensions": [["Endpoint", "Operation"]],
        "Metrics": [{ "Name": "CustomersCreated", "Unit": "Count" }]
      }
    ]
  },
  "Endpoint": "Customer",
  "Operation": "create",
  "CustomersCreated": 1
}

When this log is written to stdout via the EMF Monolog channel, CloudWatch automatically:

  1. Extracts CustomersCreated as a metric
  2. Associates it with the CCore/BusinessMetrics namespace
  3. Applies dimensions Endpoint and Operation

Creating New Business Metrics

Step 1: Create the Metric Class

// src/Core/Order/Application/Metric/OrdersPlacedMetric.php
namespace App\Core\Order\Application\Metric;

use App\Shared\Application\Observability\Metric\BusinessMetric;
use App\Shared\Application\Observability\Metric\MetricDimension;
use App\Shared\Application\Observability\Metric\MetricDimensions;
use App\Shared\Application\Observability\Metric\MetricDimensionsFactoryInterface;
use App\Shared\Application\Observability\Metric\MetricDimensionsInterface;
use App\Shared\Application\Observability\Metric\MetricUnit;

final readonly class OrdersPlacedMetricDimensions implements MetricDimensionsInterface
{
    public function __construct(
        private MetricDimensionsFactoryInterface $dimensionsFactory,
        private string $paymentMethod
    ) {
    }

    public function values(): MetricDimensions
    {
        return $this->dimensionsFactory->endpointOperationWith(
            'Order',
            'create',
            new MetricDimension('PaymentMethod', $this->paymentMethod)
        );
    }
}

final readonly class OrdersPlacedMetric extends BusinessMetric
{
    public function __construct(
        private MetricDimensionsFactoryInterface $dimensionsFactory,
        private string $paymentMethod,
        float|int $value = 1
    ) {
        parent::__construct($value, MetricUnit::COUNT);
    }

    public function name(): string
    {
        return 'OrdersPlaced';
    }

    public function dimensions(): MetricDimensionsInterface
    {
        return new OrdersPlacedMetricDimensions(
            dimensionsFactory: $this->dimensionsFactory,
            paymentMethod: $this->paymentMethod
        );
    }
}

Step 2: Create the Event Subscriber

// src/Core/Order/Application/EventSubscriber/OrderPlacedMetricsSubscriber.php
namespace App\Core\Order\Application\EventSubscriber;

use App\Core\Order\Application\Factory\OrdersPlacedMetricFactoryInterface;
use App\Core\Order\Domain\Event\OrderPlacedEvent;
use App\Shared\Application\Observability\Emitter\BusinessMetricsEmitterInterface;
use App\Shared\Domain\Bus\Event\DomainEventSubscriberInterface;

final readonly class OrderPlacedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __construct(
        private BusinessMetricsEmitterInterface $metricsEmitter,
        private OrdersPlacedMetricFactoryInterface $metricFactory
    ) {}

    public function __invoke(OrderPlacedEvent $event): void
    {
        $this->metricsEmitter->emit($this->metricFactory->create($event->paymentMethod()));
    }

    /**
     * @return array<class-string>
     */
    public function subscribedTo(): array
    {
        return [OrderPlacedEvent::class];
    }
}

Step 3: For Multiple Metrics - Use MetricCollection

// Emit multiple metrics together (dimensionsFactory injected via constructor)
$this->metricsEmitter->emitCollection(new MetricCollection(
    $this->ordersPlacedMetricFactory->create($event->paymentMethod()),
    $this->orderValueMetricFactory->create($event->totalAmount())
));

Dimension Best Practices

Recommended Dimensions

Dimension Description Cardinality
Endpoint API resource name Low
Operation CRUD action Very Low
PaymentMethod Payment type Low
CustomerType Customer segment Low

Avoid High-Cardinality Dimensions

Don't use:

  • Customer IDs
  • Order IDs
  • Session IDs
  • Timestamps

These create too many unique metric streams and increase CloudWatch costs.


Metric Naming Conventions

Format

{Entity}{Action}   # PascalCase

Examples

Good Bad
CustomersCreated customer_created
OrdersPlaced orders.placed.count
PaymentsProcessed payment-processed

Guidelines

  • Use PascalCase for metric names
  • Use plural nouns for counters (CustomersCreated not CustomerCreated)
  • Use past tense for completed actions

Testing Business Metrics

Use the Spy in Tests

use App\Shared\Application\Observability\Metric\MetricDimension;
use App\Shared\Infrastructure\Observability\Factory\MetricDimensionsFactory;
use App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy;

final class CustomerCreatedMetricsSubscriberTest extends TestCase
{
    public function testEmitsMetricOnCustomerCreated(): void
    {
        $metricsSpy = new BusinessMetricsEmitterSpy();
        $dimensionsFactory = new MetricDimensionsFactory();
        $metricFactory = new CustomersCreatedMetricFactory($dimensionsFactory);
        $logger = $this->createMock(LoggerInterface::class);

        $subscriber = new CustomerCreatedMetricsSubscriber(
            $metricsSpy,
            $metricFactory,
            $logger
        );

        $event = new CustomerCreatedEvent($customerId, $email);
        ($subscriber)($event);

        self::assertSame(1, $metricsSpy->count());

        foreach ($metricsSpy->emitted() as $metric) {
            self::assertSame('CustomersCreated', $metric->name());
            self::assertSame(1, $metric->value());
            self::assertSame('Customer', $metric->dimensions()->values()->get('Endpoint'));
            self::assertSame('create', $metric->dimensions()->values()->get('Operation'));
        }

        // Or use the assertion helper
        $metricsSpy->assertEmittedWithDimensions(
            'CustomersCreated',
            new MetricDimension('Endpoint', 'Customer'),
            new MetricDimension('Operation', 'create')
        );
    }
}

Test Service Configuration

In config/services_test.yaml, the spy is configured:

App\Shared\Application\Observability\Emitter\BusinessMetricsEmitterInterface: '@App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy'

App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy:
  public: true

CloudWatch Queries

After deploying, query your business metrics:

-- Total endpoint invocations by resource
SELECT SUM(EndpointInvocations)
FROM "CCore/BusinessMetrics"
GROUP BY Endpoint

-- Customers created over time
SELECT SUM(CustomersCreated)
FROM "CCore/BusinessMetrics"
WHERE Endpoint = 'Customer'

What NOT to Track

Remember: AWS AppRunner already provides infrastructure metrics.

Don't track:

  • Request latency
  • Error rates
  • Response times
  • HTTP status codes
  • Memory usage
  • CPU usage

Do track:

  • Business events (orders placed, customers created)
  • Business values (order amounts, payment totals)
  • Domain-specific actions (logins, uploads, exports)

Success Criteria

After implementing business metrics:

  • Each domain event that needs tracking has a corresponding metric subscriber
  • Metrics use typed classes (not arrays)
  • Metrics are emitted via event subscribers (not hardcoded in handlers)
  • Dimensions provide meaningful segmentation
  • Unit tests verify metric emission
  • No infrastructure metrics (AppRunner handles those)

SOLID Compliance Checklist

  • SRP: Each metric class has single purpose (define one metric type)
  • SRP: Event subscriber only emits metrics (no business logic)
  • OCP: New metrics added via new classes (no modification to emitter)
  • OCP: New event subscribers added without changing existing code
  • LSP: All metrics properly extend BusinessMetric base class
  • ISP: MetricDimensionsInterface is minimal (only values())
  • DIP: Handlers depend on EventBusInterface, not concrete metrics

Type Safety Checklist

  • NO arrays for metric configuration - use typed classes
  • NO arrays for metric collections - use MetricCollection
  • All dimensions via MetricDimensionsInterface implementations
  • Arrays are allowed only at infrastructure boundaries (JSON serialization, PSR-3 log context)
  • Unit enum MetricUnit used for all units

Files Reference

Metric Classes

  • src/Shared/Application/Observability/Metric/BusinessMetric.php - Base class
  • src/Shared/Application/Observability/Metric/MetricUnit.php - Unit enum
  • src/Shared/Application/Observability/Metric/MetricDimension.php - Dimension key/value
  • src/Shared/Application/Observability/Metric/MetricDimensions.php - Dimension collection
  • src/Shared/Application/Observability/Metric/MetricCollection.php - Metrics collection
  • src/Shared/Application/Observability/Metric/EndpointInvocationsMetric.php - Endpoint metric
  • src/Core/Customer/Application/Metric/CustomersCreatedMetric.php - Customer create metric
  • src/Core/Customer/Application/Metric/CustomersUpdatedMetric.php - Customer update metric
  • src/Core/Customer/Application/Metric/CustomersDeletedMetric.php - Customer delete metric

Infrastructure

  • src/Shared/Application/Observability/Emitter/BusinessMetricsEmitterInterface.php - Interface
  • src/Shared/Infrastructure/Observability/AwsEmfBusinessMetricsEmitter.php - EMF implementation
  • src/Shared/Infrastructure/Observability/EmfLogFormatter.php - Monolog formatter

Event Subscribers

  • src/Shared/Infrastructure/Observability/ApiEndpointBusinessMetricsSubscriber.php - HTTP metrics
  • src/Core/Customer/Application/EventSubscriber/CustomerCreatedMetricsSubscriber.php
  • src/Core/Customer/Application/EventSubscriber/CustomerUpdatedMetricsSubscriber.php
  • src/Core/Customer/Application/EventSubscriber/CustomerDeletedMetricsSubscriber.php

Configuration

  • config/packages/monolog.yaml - EMF channel configuration
  • config/services.yaml - Production wiring
  • config/services_test.yaml - Test spy wiring

AWS Documentation