Observability Setup

Edictum instruments the pipeline with OpenTelemetry spans, metrics, and structured audit logs. This guide covers what gets emitted, how to configure backends, and what to monitor.

What Edictum Emits

Spans

Each tool call produces two kinds of spans:

tool.execute {tool_name} -- one per tool call, covering the full lifecycle from precondition evaluation through post-execution checks.

edictum.evaluate -- one per audit event (pre-decision, post-execution, and per-rule observed blocks). Contains the full enforcement context.

Note the different value formats: governance.action on the tool.execute span uses compatibility short forms (allowed, denied, would_deny), while edictum.decision on the edictum.evaluate span uses the full AuditAction enum values with the call_ prefix (call_allowed, call_denied, call_would_deny).

Blocked calls set the edictum.evaluate span status to ERROR with the block reason. Specifically, spans with call_denied, call_approval_denied, or call_approval_timeout actions get ERROR status. All other actions (including call_would_deny in observe mode) get OK status.

Counters

Two counters are registered under the edictum meter:

Setup: Grafana Cloud

Set environment variables to send traces and metrics to Grafana Cloud:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp-gateway-prod-us-east-0.grafana.net/otlp"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64-encoded-instance-id:api-key>"
export OTEL_SERVICE_NAME="my-agent"

Then configure Edictum with OTel:

from edictum.otel import configure_otel

configure_otel(
    service_name="my-agent",
    endpoint="https://otlp-gateway-prod-us-east-0.grafana.net/otlp",
)

Standard OTel environment variables take precedence over function arguments, so you can configure purely via env vars if preferred.

Setup: Local Development

For local development, use a docker-compose stack with the OpenTelemetry Collector, Tempo, and Grafana. The edictum-demo repository includes a ready-to-use docker-compose.yaml and dashboard JSON.

Point Edictum at the local collector:

from edictum.otel import configure_otel

configure_otel(
    service_name="my-agent",
    endpoint="http://localhost:4317",
)

YAML Observability Config

Edictum supports an observability block at the top level of your ruleset for configuring audit output:

observability:
  otel:
    enabled: true
    service_name: my-agent
    endpoint: http://localhost:4317
    protocol: grpc        # or "http"
    insecure: true        # default; set false for TLS
    resource_attributes:
      deployment.environment: production
  file: audit.jsonl
  stdout: true

What to Monitor

Block rate

Track the ratio of blocked to total tool calls. A spike in blocks may indicate:

• A misconfigured rule (false positives)
• An agent behaving unexpectedly (attempting blocked actions repeatedly)
• A legitimate rule change that needs communication to users

PII detection frequency

Monitor postcondition warnings with pii tags. Frequent PII detections may indicate:

• Tools returning sensitive data that should be filtered upstream
• Missing input validation in external services
• Need for stricter preconditions to prevent the calls in the first place

Session limit hits

Track session rule blocks. Frequent session limit hits suggest:

• Agents stuck in loops (retry-after-block patterns)
• Limits set too low for the task complexity
• Need for better agent instructions to prevent excessive tool use

Observed blocks

In observe mode, track CALL_WOULD_DENY events to validate new rulesets before enforcement. A high observed-block rate on a new rule may mean it needs tuning before going to enforce mode.

Dashboard and Demo

The edictum-demo repository includes:

• A docker-compose.yaml with OTel Collector, Tempo, and Grafana pre-configured
• Grafana dashboard JSON for visualizing block rates and tool call volumes
• Example agents that produce enforcement telemetry

For full details on span attributes, metric names, and advanced OTel configuration, see the Telemetry Reference.