Skip to content

Metrics

nats-sinks includes a small metrics layer for operators and developers who need to understand how a sink process is behaving. Metrics are intentionally destination-neutral where possible: Oracle Database, Oracle MySQL, Oracle Coherence Community Edition, file, and future sinks share the same core counters, gauges, observations, and Python hooks, while destination-specific counters use clear prefixes such as oracle_* and mysql_*.

The current release provides a dependency-free local snapshot recorder and a standalone inspection command:

nats-sink-metrics show .local/nats-sinks/metrics.json

For controlled external sharing, use the separate observability CLI:

nats-sink-observe init-prometheus-policy \
  /etc/nats-sinks/config.json \
  /etc/nats-sinks/observability.prometheus.json

This command is separate from nats-sink. The nats-sink command runs and manages sink processing. The nats-sink-metrics command reads a local JSON metrics snapshot and renders it in human-readable or script-friendly formats. It never connects to NATS, Oracle Database, Oracle MySQL, Oracle Coherence Community Edition, the file sink directory, or any future destination backend.

The nats-sink-observe command owns external sharing. It can render a Prometheus textfile for node_exporter, run an optional native Prometheus HTTP endpoint, or export approved metrics to an OpenTelemetry Collector through OTLP/HTTP JSON, OCI Monitoring custom metrics, Amazon CloudWatch custom metrics, Azure Monitor custom metrics, StatsD, Datadog, syslog, or another implemented observability connector. These connectors are disabled by default and require an explicit allow-list policy before real metrics are shared.

Why Metrics Matter

Reliable message movement is easier to operate when the runtime exposes what it is doing. Metrics help answer practical questions:

  • is the runner fetching messages from JetStream,
  • are messages reaching the sink,
  • is the sink returning durable success,
  • are messages being ACKed only after durable success,
  • are temporary or permanent failures increasing,
  • is the DLQ being used,
  • is the destination write stage getting slower,
  • has the sink stopped making progress.

For defence, public-sector, and mission-support deployments, metrics are part of the operational picture. They should show service health and throughput without revealing sensitive payloads, credentials, private subjects, or classified operational content.

Architecture

The built-in snapshot path is deliberately local and simple:

flowchart LR
    Runner[JetStreamSinkRunner] --> Recorder[JsonFileMetrics]
    Recorder --> Snapshot[metrics.json]
    Snapshot --> CLI[nats-sink-metrics]
    CLI --> Human[table output]
    CLI --> Shell[shell variables]
    CLI --> JSON[JSON / JSONL]
    CLI --> PromText[Prometheus text]
    Python[Python application] --> Snapshot
    Snapshot --> Observe[nats-sink-observe]
    Observe --> Textfile[Prometheus textfile]
    Observe --> Http[Optional Prometheus HTTP endpoint]
    Observe --> OTLP[OpenTelemetry Collector]
    Observe --> Syslog[Syslog bridge]

The runner emits metric suffixes such as messages_fetched_total. A recorder stores or exports those values. JsonFileMetrics writes a compact local JSON document that the metrics CLI can read. Larger deployments can implement the same MetricsRecorder protocol and send values to Prometheus, OpenTelemetry, OCI Monitoring, StatsD, syslog, or a platform-native telemetry service.

Metrics are observational only. They must never change ACK ordering, inspect plaintext payloads, mutate envelopes, or decide whether a message is successful.

External sharing should go through the observability policy layer described in Observability. The policy layer is intentionally separate from the runner and sinks so the main delivery process and any monitoring connector can run as different services with different permissions.

The default metrics snapshot is aggregate-only. It does not include per-subject labels or per-subject series unless a separate, reviewed subject-family aggregation step attaches prepared labeled_metrics rows. That extension is disabled unless the subject_metrics policy model explicitly allows a subject family, gives it a stable label, and keeps it within cardinality bounds. Raw subjects are not exported by default. See Subject-Aware Observability Evaluation and the Subject-Aware Observability Runbook.

Enabling The Snapshot Recorder

Add metrics.snapshot_file to the same JSON config used by nats-sink run:

{
  "metrics": {
    "enabled": true,
    "namespace": "nats_sinks",
    "snapshot_file": ".local/nats-sinks/metrics.json",
    "event_freshness_enabled": true,
    "event_stale_after_seconds": 300,
    "event_future_skew_tolerance_seconds": 5
  }
}

When metrics.enabled is true and metrics.snapshot_file is set, the CLI uses JsonFileMetrics and keeps the snapshot updated while the sink process runs. If metrics are disabled, or no snapshot file is configured, the CLI uses a no-op recorder and no metrics file is written.

JsonFileMetrics creates parent directories when needed and writes the final snapshot atomically with owner-readable and owner-writable permissions. This is useful for local service scripts and systemd checks because readers should never see a half-written JSON document.

Snapshot Shape

A metrics snapshot is UTF-8 JSON with a small, versioned schema:

{
  "schema": "nats_sinks.metrics.snapshot.v1",
  "namespace": "nats_sinks",
  "generated_at_epoch_seconds": 1779280000.0,
  "counters": {
    "messages_fetched_total": 256,
    "messages_prepared_total": 256,
    "messages_written_total": 256,
    "messages_acked_total": 256,
    "messages_terminated_total": 0,
    "sink_batches_written_total": 4
  },
  "gauges": {
    "current_batch_messages": 64,
    "last_sink_success_epoch_seconds": 1779280000.0
  },
  "observations": {
    "sink_batch_write_seconds": {
      "count": 4,
      "sum": 3.684471,
      "min": 0.812345,
      "max": 1.034210,
      "last": 0.901234
    }
  },
  "labeled_metrics": [
    {
      "kind": "counter",
      "name": "messages_written_total",
      "value": 128,
      "labels": {
        "subject_family": "orders"
      }
    }
  ]
}

Observations are summarized instead of storing raw arrays. This keeps the snapshot bounded and avoids writing unbounded timing history to disk.

labeled_metrics is optional and absent from ordinary aggregate snapshots. It is reserved for prepared low-cardinality series, such as reviewed subject_family counters. Exporters read those prepared rows rather than deriving labels from raw NATS subjects. The snapshot loader validates the row count, known metric names, row kinds, finite values, and bounded label names and values before any connector can render them.

The snapshot should not contain secrets or payload bodies. It can still reveal operational tempo, failure rates, and batch sizes, so store it in a local path with appropriate filesystem permissions.

The subject-aware certification helper under nats_sinks.testing verifies that prepared rows remain disabled by default, low-cardinality, sanitized, and delivery-neutral. Run the focused certification suite before enabling or changing subject-family metrics:

python -m pytest tests/unit/test_subject_observability_certification.py -q

Event Freshness And Staleness Metrics

Event freshness metrics show how old a message was when the runner received it and how old it was after the sink reported durable success. This is useful when operators need to distinguish a healthy high-volume backlog replay from a fresh operational feed, or when delayed sensor or platform telemetry must be investigated without reading message bodies.

Freshness is resolved in this order:

  1. A valid Nats-Time-Stamp header supplied by the publisher.
  2. The JetStream server timestamp exposed by nats-py metadata.
  3. No creation timestamp, recorded as a missing-timestamp counter.

Malformed Nats-Time-Stamp values are counted and then fall back to the JetStream timestamp when one is available. Future-dated timestamps are accepted as observations, not rejected. The runner records positive source clock skew, clamps negative age observations to zero, and increments a future-timestamp counter when the skew exceeds metrics.event_future_skew_tolerance_seconds.

The freshness metrics are:

Metric suffix Type Meaning
event_age_at_receive_seconds observation Event age when the runner received the message.
event_age_at_store_seconds observation Event age after the sink reported durable success.
events_stale_at_receive_total counter Events older than metrics.event_stale_after_seconds at receive time.
events_stale_at_store_total counter Events older than metrics.event_stale_after_seconds after durable sink success.
event_creation_timestamp_missing_total counter Messages without a usable publisher or JetStream creation timestamp.
event_creation_timestamp_malformed_total counter Messages with a malformed publisher timestamp header.
event_creation_timestamp_future_total counter Messages beyond the configured future-skew tolerance.
event_source_clock_skew_seconds observation Positive clock skew seconds for future-dated events.

These metrics are observational only. They do not change ACK behavior, do not reject stale events, and do not affect DLQ routing. The commit-then-acknowledge rule remains unchanged: the sink must report durable success before the runner ACKs JetStream.

Example shell inspection:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --metric "event_*" \
  --metric "events_*"

Example output:

EVENT_AGE_AT_RECEIVE_SECONDS_COUNT=256
EVENT_AGE_AT_RECEIVE_SECONDS_MAX=12.428
EVENT_AGE_AT_RECEIVE_SECONDS_SUM=972.511
EVENT_AGE_AT_STORE_SECONDS_COUNT=256
EVENTS_STALE_AT_RECEIVE_TOTAL=3
EVENTS_STALE_AT_STORE_TOTAL=4
EVENT_CREATION_TIMESTAMP_MISSING_TOTAL=0
EVENT_CREATION_TIMESTAMP_MALFORMED_TOTAL=1
EVENT_CREATION_TIMESTAMP_FUTURE_TOTAL=0

For Prometheus or OTLP export, keep freshness sharing explicitly allow-listed:

{
  "enabled": true,
  "allowed_metric_patterns": ["event_*", "events_*"],
  "include_observations": true,
  "prometheus": {
    "enabled": true
  }
}

Freshness metrics intentionally do not include labels for subject, stream, source system, sensor, table, sink name, or hostname. Those values can be sensitive and can create unbounded cardinality. Use aggregate counters first, then investigate individual records through approved operational tooling when a metric indicates delayed, missing, malformed, or skewed event timestamps.

InProgress Metrics

InProgress metrics describe optional JetStream progress heartbeats for long-running sink writes. The metric contract, snapshot rendering, CLI output, and Prometheus text rendering let operators and connector authors build dashboards without inventing names. Runtime InProgress heartbeats are disabled by default and emit these metrics only when delivery.in_progress.enabled=true and the consumer AckWait guardrails pass startup validation.

These metrics are observational only. A progress heartbeat means "work is still active." It is not durable sink success, not a message ACK, not a DLQ publication, and not an idempotency decision.

Metric suffix Type Meaning
in_progress_attempts_total counter JetStream InProgress heartbeat attempts while sink work is active.
in_progress_successes_total counter Heartbeats accepted by the client path; not sink success.
in_progress_failures_total counter Heartbeat calls that failed before the final ACK decision.
in_progress_max_heartbeats_reached_total counter Batches where the configured heartbeat limit was reached.
current_in_progress_batches_active gauge Batches currently under heartbeat supervision.
in_progress_heartbeat_seconds observation Elapsed time spent sending heartbeat operations.

Example shell inspection:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --metric "in_progress_*" \
  --metric "current_in_progress_*"

Example output:

CURRENT_IN_PROGRESS_BATCHES_ACTIVE=0.0
IN_PROGRESS_ATTEMPTS_TOTAL=18
IN_PROGRESS_FAILURES_TOTAL=0
IN_PROGRESS_HEARTBEAT_SECONDS_COUNT=18
IN_PROGRESS_MAX_HEARTBEATS_REACHED_TOTAL=0
IN_PROGRESS_SUCCESSES_TOTAL=18

For operational interpretation, alerting guidance, and safe Prometheus examples, read the InProgress Metrics Runbook.

Message Authenticity Metrics

When message_authenticity.enabled is true, the core runtime records aggregate verification metrics before any sink write. These counters help operators see whether signed traffic is flowing, whether producers are missing required headers, and whether a signature policy is rejecting messages. They do not export signatures, key material, payload bytes, or raw header values.

Metric suffix Type Meaning
message_authenticity_messages_passed_total counter Messages accepted by message authenticity verification.
message_authenticity_messages_rejected_total counter Messages rejected before sink delivery.
message_authenticity_batches_passed_total counter Batches with at least one accepted message.
message_authenticity_batches_rejected_total counter Batches with at least one rejected message.
message_authenticity_evaluation_errors_total counter Unexpected evaluator failures that left messages redeliverable.

Example shell inspection:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --metric "message_authenticity_*"

Example output:

MESSAGE_AUTHENTICITY_MESSAGES_PASSED_TOTAL=250
MESSAGE_AUTHENTICITY_MESSAGES_REJECTED_TOTAL=6
MESSAGE_AUTHENTICITY_BATCHES_PASSED_TOTAL=4
MESSAGE_AUTHENTICITY_BATCHES_REJECTED_TOTAL=1
MESSAGE_AUTHENTICITY_EVALUATION_ERRORS_TOTAL=0

These metrics are useful in controlled mission networks because a sudden increase in rejected signed messages can indicate a producer deployment mistake, stale key material, replayed traffic, or an attempted injection into a trusted event stream. Keep any external sharing behind an explicit observability policy allow list.

CLI Commands

The metrics CLI provides three commands:

Command Purpose
nats-sink-metrics show SNAPSHOT Show all or selected metrics in table, JSON, JSONL, shell, Prometheus, or names format.
nats-sink-metrics get SNAPSHOT METRIC Print a single metric value for shell scripts.
nats-sink-metrics describe List the metric names emitted by the framework.

The observability CLI provides policy and connector commands:

Command Purpose
nats-sink-observe init-prometheus-policy CONFIG POLICY Generate a disabled observability policy from runtime config.
nats-sink-observe validate-policy POLICY Validate sharing policy before enabling a connector.
nats-sink-observe show-effective-policy POLICY Show policy JSON or a summary for review.
nats-sink-observe list-metrics List metric names that may be added to an allow list.
nats-sink-observe list-subjects POLICY List subject hints discovered from config without exporting them.
nats-sink-observe prometheus-textfile SNAPSHOT POLICY Render policy-filtered Prometheus textfile output.
nats-sink-observe prometheus-http SNAPSHOT POLICY Run or dry-run the optional native Prometheus HTTP endpoint.
nats-sink-observe otlp-export SNAPSHOT POLICY Dry-run or send policy-approved metrics to an OpenTelemetry Collector through OTLP/HTTP JSON.
nats-sink-observe elastic-export SNAPSHOT POLICY Dry-run or send approved metrics through the Elastic OTLP profile.
nats-sink-observe grafana-alloy-export SNAPSHOT POLICY Dry-run or send approved metrics through the Grafana Alloy profile.
nats-sink-observe splunk-hec-export SNAPSHOT POLICY Dry-run or send approved aggregate metrics to Splunk HEC.
nats-sink-observe oci-monitoring-export SNAPSHOT POLICY Dry-run or send approved custom metrics to OCI Monitoring.
nats-sink-observe statsd-export SNAPSHOT POLICY Dry-run or send approved best-effort metric datagrams to StatsD.
nats-sink-observe datadog-export SNAPSHOT POLICY Dry-run or send approved best-effort DogStatsD datagrams to a Datadog Agent.
nats-sink-observe cloudwatch-export SNAPSHOT POLICY Dry-run or send approved custom metrics to Amazon CloudWatch.
nats-sink-observe azure-monitor-export SNAPSHOT POLICY Dry-run or send approved custom metrics to Azure Monitor.
nats-sink-observe syslog-export SNAPSHOT POLICY Dry-run or send approved RFC 5424-style metric messages to syslog.

The global version flag is available too:

nats-sink-metrics --version

Example output:

0.3.0

Table Output

The default show output is intended for humans:

nats-sink-metrics show .local/nats-sinks/metrics.json

Example output:

KIND         METRIC                            VALUE  DESCRIPTION
counter      messages_acked_total                256  Messages acknowledged to JetStream after durable success or DLQ success.
counter      messages_fetched_total              256  Raw JetStream messages fetched by the pull consumer.
counter      messages_prepared_total             256  Messages converted into envelopes and transformed by core policies.
counter      messages_written_total              256  Messages reported durable by the destination sink.
counter      sink_batches_written_total            4  Batches that returned durable success from sink.write_batch.
gauge        current_batch_messages               64  Number of messages in the active batch currently being processed.
observation  sink_batch_write_seconds.count        4  Elapsed seconds spent inside sink.write_batch for successful batches.
observation  sink_batch_write_seconds.sum    3.684471  Elapsed seconds spent inside sink.write_batch for successful batches.

Legacy compatibility metric aliases are hidden by default. Add --include-legacy if you are migrating an older local dashboard.

Shell Output

Use shell output when a script needs to source or parse values:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --kind counter \
  --metric "messages_*"

Example output:

MESSAGES_ACKED_TOTAL=256
MESSAGES_DLQ_TOTAL=0
MESSAGES_FAILED_TOTAL=0
MESSAGES_FETCHED_TOTAL=256
MESSAGES_NACKED_TOTAL=0
MESSAGES_PREPARED_TOTAL=256
MESSAGES_TERMINATED_TOTAL=0
MESSAGES_WRITTEN_TOTAL=256

A simple health gate can use get:

failed=$(nats-sink-metrics get .local/nats-sinks/metrics.json messages_failed_total --default 0)
dlq=$(nats-sink-metrics get .local/nats-sinks/metrics.json messages_dlq_total --default 0)

if [ "$failed" -gt 0 ] || [ "$dlq" -gt 0 ]; then
  echo "nats-sinks has failed or DLQ messages"
  exit 2
fi

get exits with code 4 when the metric is missing and no --default value is supplied. This makes missing metrics visible in strict scripts.

Terminal Acknowledgement Metrics

Terminal acknowledgement metrics are emitted only when a permanent failure is published to DLQ and dead_letter.ack_term_after_publish is enabled. They are separate from normal ACK metrics because AckTerm means "stop redelivery after terminal failure handling", not "the sink successfully wrote the message".

Confirmed ACK metrics are emitted when delivery.ack_confirmation.enabled=true and the runner attempts a server-confirmed ACK after durable sink success or successful DLQ publication. They are separate from ordinary ACK counters so an operator can distinguish "ACK sent" from "ACK confirmation received". Confirmation failure after durable success can still lead to redelivery, so these counters should be interpreted alongside idempotency and duplicate metrics.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "ack_confirmation_*"

Example shell output:

ACK_CONFIRMATION_ATTEMPTS_TOTAL=3
ACK_CONFIRMATION_SUCCESSES_TOTAL=2
ACK_CONFIRMATION_TIMEOUTS_TOTAL=1
ACK_CONFIRMATION_SECONDS_COUNT=1

The InProgress metric contract is available for the optional runtime heartbeat. The metrics and operator interpretation guidance are documented in the InProgress Metrics Runbook, and the timing evaluation remains in InProgress Evaluation. These metrics distinguish "work is still active" from "work is successful" so dashboards do not accidentally treat progress as durable completion.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "*term*"

Example output:

KIND         METRIC                       VALUE  DESCRIPTION
counter      messages_terminated_total       3  Messages terminally acknowledged to JetStream after successful DLQ publication.
counter      term_errors_total               0  Messages whose terminal acknowledgement failed after successful DLQ publication.
observation  message_term_seconds.count      1  Elapsed seconds spent sending terminal acknowledgements after DLQ publication.

Use these metrics together with messages_dlq_total. A terminal acknowledgement without a corresponding DLQ publication would indicate a bug or an unsafe custom integration.

JetStream Advisory Metrics

JetStream advisory metrics are emitted only when advisories.enabled is true. The observer subscribes to selected $JS.EVENT.ADVISORY... subjects and turns supported advisory types into aggregate counters. It does not expose stream names, consumer names, subjects, sequence numbers, message IDs, or advisory payload fields as metric labels.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "jetstream_advisory*"

Example table output:

KIND     METRIC                                             VALUE  DESCRIPTION
counter  jetstream_advisories_received_total                    4  JetStream advisory messages accepted by the optional advisory monitor.
counter  jetstream_advisory_max_deliver_total                   2  JetStream max-deliver advisories observed without exposing stream or consumer names.
counter  jetstream_advisory_terminated_total                    1  JetStream terminal-ack advisories observed without exposing stream or consumer names.
counter  jetstream_advisory_parse_errors_total                  0  JetStream advisory messages rejected by safe JSON parsing and validation.

These counters are operational signals only. They do not cause DLQ publication, ACK, NAK, terminal acknowledgement, retry, or sink writes. Use them to correlate server-side delivery conditions with sink-side counters such as messages_failed_total, messages_dlq_total, and messages_nacked_total.

For configuration and permission guidance, read Configuration and NATS Least-Privilege Permissions.

Pre-Sink Policy Metrics

Pre-sink policy metrics are emitted only when pre_sink_policy.enabled is true. They are aggregate by design. The runner does not expose subjects, priority values, classification values, labels, mission metadata keys, message IDs, table names, file paths, or payload fields as metric labels.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "policy_*"

Example table output:

KIND     METRIC                            VALUE  DESCRIPTION
counter  policy_batches_passed_total          18  Batches whose messages all passed pre-sink policy evaluation.
counter  policy_batches_rejected_total         2  Batches with at least one message rejected by pre-sink policy evaluation.
counter  policy_messages_passed_total       1150  Messages accepted by pre-sink policy evaluation.
counter  policy_messages_rejected_total        3  Messages rejected by pre-sink policy evaluation before any sink write.
counter  policy_evaluation_errors_total        0  Messages affected by unexpected pre-sink policy evaluation errors.

Use these counters with messages_failed_total and messages_dlq_total. A policy rejection is a permanent validation failure. The rejected message does not reach the sink. When DLQ is enabled, the original JetStream message is ACKed or terminally acknowledged only after DLQ publication succeeds.

For configuration details, read Configuration.

Size Policy Metrics

Size policy metrics are emitted only when size_policy.enabled is true. They show aggregate pass and rejection counts without exporting payloads, header values, labels, mission metadata values, subjects, table names, file paths, or other sensitive operational detail.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "size_policy_*"

Example table output:

KIND     METRIC                               VALUE  DESCRIPTION
counter  size_policy_batches_passed_total        24  Batches with at least one message accepted by the core size policy.
counter  size_policy_batches_rejected_total       1  Batches with at least one message rejected by the core size policy.
counter  size_policy_messages_passed_total     1536  Messages accepted by the core size policy before sink delivery.
counter  size_policy_messages_rejected_total      4  Messages rejected by the core size policy before sink delivery.
counter  size_policy_evaluation_errors_total      0  Messages left redeliverable because size-policy evaluation failed unexpectedly.

Use these counters with messages_failed_total and messages_dlq_total. A size-policy rejection is a permanent validation failure. The rejected message does not reach the sink. When DLQ is enabled, the original JetStream message is ACKed or terminally acknowledged only after DLQ publication succeeds.

For configuration and tuning details, read Configuration and Message Sizing.

Priority Lane Metrics

Priority lane metrics are emitted only when delivery.priority_lanes.enabled is true. They are deliberately aggregate. The core does not expose subject names, priority values, classification values, labels, message IDs, stream sequence numbers, or payload fields as metric labels.

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "priority_lane_*"

Example table output:

KIND     METRIC                         VALUE  DESCRIPTION
counter  priority_lane_batches_total       12  Batches ordered by enabled priority-lane scheduling.
counter  priority_lane_defaulted_total     41  Messages routed to the default priority lane because priority was missing or unknown.
counter  priority_lane_messages_total     768  Messages evaluated by priority-lane scheduling without exposing subjects.
counter  priority_lane_rejected_total       0  Messages rejected because priority metadata violated the priority-lane policy.

The active-lane gauge shows how many configured lanes were represented in the current scheduled batch:

nats-sink-metrics get \
  .local/nats-sinks/metrics.json \
  current_priority_lanes_active \
  --default 0

Priority lane metrics are useful for answering operational questions such as:

  • whether priority scheduling is actually being exercised,
  • whether many messages are missing priority and falling back to the default lane,
  • whether strict priority validation is rejecting malformed or spoofed values,
  • whether a mixed-priority backlog is present.

For the scheduling model and configuration details, read Priority-Aware Processing Lanes.

Fan-Out Metrics

Fan-out metrics are emitted by the production FanoutSink through the reusable route-selection and ACK-gate observability helpers. They are aggregate-only and deliberately payload-free. The helpers count how many route policy entries matched, how many messages selected child sinks, how many child sinks were selected, whether required sinks succeeded or failed, whether optional sinks succeeded, failed, or timed out, and whether the original JetStream message became eligible for ACK.

The fan-out metrics do not expose route names, sink instance names, NATS subjects, classification values, labels, message IDs, file paths, database names, or payload fields. If a deployment later wants subject-aware or sink-aware external export, that must be approved through an explicit observability policy with bounded cardinality.

The fan-out metric suffixes are:

Metric suffix Type Meaning
fanout_route_matches_total counter Matched route policy entries without exporting route names.
fanout_messages_routed_total counter Messages with at least one selected child sink target.
fanout_messages_no_route_total counter Messages rejected or ignored because no child sink target was selected.
fanout_child_sinks_selected_total counter Selected child sink operations across routed messages.
current_fanout_child_sinks_selected gauge Child sink targets selected for the latest evaluated message.
fanout_required_child_success_total counter Required child sink operations that committed before ACK.
fanout_required_child_failure_total counter Required child sink operations that failed and blocked ACK.
fanout_optional_child_success_total counter Optional child sink operations that completed before the ACK gate released.
fanout_optional_child_failure_total counter Optional child sink operations that failed without blocking required ACK.
fanout_optional_child_timeout_total counter Optional child sink operations that exceeded their bounded ACK wait window.
fanout_messages_acked_total counter Fan-out messages whose original JetStream message became eligible for ACK.
fanout_messages_ack_blocked_total counter Fan-out messages whose original ACK was blocked by required failure.
fanout_ack_gate_wait_seconds observation Seconds spent waiting at the fan-out ACK gate.
fanout_batch_seconds observation Seconds spent processing a fan-out batch across selected child sinks.

Show only fan-out rows:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --metric "fanout_*" \
  --metric "current_fanout_*"

Example output:

CURRENT_FANOUT_CHILD_SINKS_SELECTED=2
FANOUT_ACK_GATE_WAIT_SECONDS_COUNT=1
FANOUT_ACK_GATE_WAIT_SECONDS_MAX=0.125
FANOUT_CHILD_SINKS_SELECTED_TOTAL=2
FANOUT_MESSAGES_ACKED_TOTAL=1
FANOUT_MESSAGES_ROUTED_TOTAL=1
FANOUT_OPTIONAL_CHILD_TIMEOUT_TOTAL=1
FANOUT_REQUIRED_CHILD_SUCCESS_TOTAL=1

Prometheus text output from the local CLI keeps the same aggregate posture:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format prometheus \
  --metric "fanout_*" \
  --namespace mission_ops

Example output:

# HELP mission_ops_fanout_messages_routed_total Messages with at least one selected fan-out child sink target.
# TYPE mission_ops_fanout_messages_routed_total counter
mission_ops_fanout_messages_routed_total 1
# HELP mission_ops_fanout_ack_gate_wait_seconds Elapsed seconds spent waiting at the fan-out ACK gate.
# TYPE mission_ops_fanout_ack_gate_wait_seconds summary
mission_ops_fanout_ack_gate_wait_seconds_count 1
mission_ops_fanout_ack_gate_wait_seconds_sum 0.125

Python test harnesses and embedded fan-out delivery code can use the helper module directly:

from nats_sinks.core.fanout_observability import record_fanout_ack_gate_result

summary = record_fanout_ack_gate_result(
    metrics,
    ack_gate_result,
    ack_wait_seconds=0.125,
    batch_seconds=0.250,
)
assert summary.messages_acked == 1

Structured fan-out logs are human-readable and intentionally sanitized. A required failure log says that the original message remains unacknowledged; an optional timeout log says that the ACK gate released with optional issues. The logs include counts and outcome categories, not payloads or child sink names.

Oracle Duplicate And Conflict Metrics

Oracle duplicate/conflict metrics are emitted by OracleSink when a write path can prove that a duplicate or conflict happened. They are intentionally low-cardinality counters: they do not include table names, subjects, message IDs, classification values, labels, payload text, or Oracle constraint names.

These counters are operational evidence only. They do not change commit-then-acknowledge behavior. A safely ignored duplicate in insert_ignore mode is still durable success because the idempotency key already exists in Oracle. The core runner may ACK after OracleSink.write_batch(...) returns success.

The Oracle-specific counters are:

Metric suffix Meaning
oracle_conflicts_total Oracle write conflicts observed by OracleSink, such as ORA-00001 duplicate-key conflicts.
oracle_duplicates_total Rows identified as duplicate prior processing through an idempotent Oracle write path.
oracle_duplicate_ignored_total Duplicate rows safely ignored by insert_ignore mode.
oracle_duplicate_noop_total Duplicate rows safely left unchanged by merge with merge_update_columns: [].
oracle_merge_rows_total Rows committed through Oracle merge mode.
oracle_merge_outcome_unknown_total merge rows where Oracle did not reliably expose whether the row was inserted or matched.

For insert_ignore, nats-sinks generates an Oracle merge that inserts only when the idempotency key is not already present. When Oracle reports that fewer rows were affected than attempted, the difference is counted as ignored duplicates. If Oracle raises ORA-00001 and the sink is in insert_ignore mode, the conflict is counted and then treated as a safe duplicate success.

For merge, Oracle updates matching rows and inserts missing rows. The current execution path does not reliably expose per-row "inserted versus matched" counts across driver versions when updates are enabled, so nats-sinks records oracle_merge_outcome_unknown_total instead of guessing. If merge is configured with merge_update_columns: [], matched rows are left unchanged; in that no-update mode a lower affected-row count can be reported as oracle_duplicate_noop_total and included in oracle_duplicates_total.

Show Oracle duplicate and conflict counters:

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "oracle_*"

Example table output:

KIND     METRIC                           VALUE  DESCRIPTION
counter  oracle_conflicts_total               1  Oracle write conflicts observed by OracleSink, such as duplicate-key conflicts.
counter  oracle_duplicate_ignored_total       7  Oracle duplicate rows safely ignored by insert_ignore mode.
counter  oracle_duplicate_noop_total          3  Oracle duplicate rows safely left unchanged by merge mode with no update columns.
counter  oracle_duplicates_total             10  Oracle rows identified as duplicate prior processing through idempotent handling.
counter  oracle_merge_outcome_unknown_total  64  Oracle merge rows where insert-versus-match outcome is not reliably exposed.
counter  oracle_merge_rows_total             64  Oracle rows committed through merge mode.

Use shell output in service scripts:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format shell \
  --metric "oracle_*"

Example shell output:

ORACLE_CONFLICTS_TOTAL=1
ORACLE_DUPLICATE_IGNORED_TOTAL=7
ORACLE_DUPLICATE_NOOP_TOTAL=3
ORACLE_DUPLICATES_TOTAL=10
ORACLE_MERGE_OUTCOME_UNKNOWN_TOTAL=64
ORACLE_MERGE_ROWS_TOTAL=64

Read one value with a safe default:

duplicates=$(nats-sink-metrics get \
  .local/nats-sinks/metrics.json \
  oracle_duplicates_total \
  --default 0)

echo "Oracle duplicate rows observed: $duplicates"

Python applications can read the same snapshot:

from nats_sinks import load_metrics_snapshot, metric_rows_from_snapshot

snapshot = load_metrics_snapshot(".local/nats-sinks/metrics.json")
rows = metric_rows_from_snapshot(snapshot)
oracle_rows = [row for row in rows if row.name.startswith("oracle_")]

for row in oracle_rows:
    print(row.name, row.value)

Oracle MySQL Duplicate And Upsert Metrics

Oracle MySQL duplicate and upsert metrics are emitted by MySqlSink when the write path can observe duplicate-safe behavior through Oracle MySQL execution metadata. The counters are aggregate-only. They do not include table names, subjects, message IDs, payload text, classification values, labels, or constraint names.

The Oracle MySQL-specific counters are:

Metric suffix Type Meaning
mysql_conflicts_total counter Oracle MySQL duplicate-key conflicts observed while writing a batch.
mysql_duplicates_total counter Rows identified as duplicate prior processing through an idempotent Oracle MySQL write path.
mysql_duplicate_ignored_total counter Duplicate rows safely ignored by insert_ignore mode.
mysql_duplicate_noop_total counter Duplicate rows safely left unchanged by upsert with upsert_update_columns: [].
mysql_upsert_rows_total counter Rows committed through Oracle MySQL upsert mode.
mysql_upsert_outcome_unknown_total counter upsert rows where Oracle MySQL did not reliably expose whether the row inserted or matched.

Show Oracle MySQL counters:

nats-sink-metrics show .local/nats-sinks/metrics.json --metric "mysql_*"

Example output:

kind     name                                value  description
counter  mysql_duplicates_total                  3  Oracle MySQL rows identified as duplicate prior processing through idempotent handling.
counter  mysql_duplicate_noop_total              3  Oracle MySQL duplicate rows safely left unchanged by upsert mode with no update columns.
counter  mysql_upsert_rows_total                64  Oracle MySQL rows committed through upsert mode.
counter  mysql_upsert_outcome_unknown_total     61  Oracle MySQL upsert rows where insert-versus-match outcome is not reliably exposed.

Python applications can read the same metrics from the local snapshot:

from nats_sinks import load_metrics_snapshot, metric_rows_from_snapshot

snapshot = load_metrics_snapshot(".local/nats-sinks/metrics.json")
rows = metric_rows_from_snapshot(snapshot)
mysql_rows = [row for row in rows if row.name.startswith("mysql_")]

for row in mysql_rows:
    print(row.name, row.value)

JSON Output

Use JSON output for tools that want one structured document:

nats-sink-metrics show .local/nats-sinks/metrics.json --format json --kind counter

Example output:

{
  "generated_at_epoch_seconds": 1779280000.0,
  "metrics": [
    {
      "description": "Raw JetStream messages fetched by the pull consumer.",
      "kind": "counter",
      "name": "messages_fetched_total",
      "stat": null,
      "value": 256.0
    }
  ],
  "namespace": "nats_sinks",
  "schema": "nats_sinks.metrics.snapshot.v1"
}

Use JSONL when you want one metric per line:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format jsonl \
  --kind observation \
  --metric "sink_batch_write_seconds.*"

Example output:

{"description": "Elapsed seconds spent inside sink.write_batch for successful batches.", "kind": "observation", "name": "sink_batch_write_seconds.count", "stat": "count", "value": 4.0}
{"description": "Elapsed seconds spent inside sink.write_batch for successful batches.", "kind": "observation", "name": "sink_batch_write_seconds.sum", "stat": "sum", "value": 3.684471}
{"description": "Elapsed seconds spent inside sink.write_batch for successful batches.", "kind": "observation", "name": "sink_batch_write_seconds.max", "stat": "max", "value": 1.03421}

Prometheus Text Output

The snapshot CLI can render Prometheus text format for simple textfile collectors:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format prometheus \
  > /var/lib/node_exporter/textfile_collector/nats_sinks.prom

Example output:

# HELP nats_sinks_messages_fetched_total Raw JetStream messages fetched by the pull consumer.
# TYPE nats_sinks_messages_fetched_total counter
nats_sinks_messages_fetched_total 256
# HELP nats_sinks_sink_batch_write_seconds Elapsed seconds spent inside sink.write_batch for successful batches.
# TYPE nats_sinks_sink_batch_write_seconds summary
nats_sinks_sink_batch_write_seconds_count 4
nats_sinks_sink_batch_write_seconds_sum 3.684471

This is a convenience format. It is not a full Prometheus exporter and does not open an HTTP endpoint.

For service deployments, prefer the policy-controlled connector documented in the Observability section's Prometheus Integration sub-page:

nats-sink-observe prometheus-textfile \
  /var/lib/nats-sink/metrics.json \
  /etc/nats-sinks/observability.prometheus.json \
  --output /var/lib/node_exporter/textfile_collector/nats_sinks.prom

The nats-sink-observe path is safer for production because it defaults to no sharing, applies allow and deny lists, supports staleness checks, and can run as a separate Linux service.

You can override the exported namespace for Prometheus output:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --format prometheus \
  --namespace mission_ops

OpenTelemetry OTLP Export

The observability CLI can also export the same policy-approved metric rows to an OpenTelemetry Collector through OTLP/HTTP JSON:

nats-sink-observe otlp-export \
  /var/lib/nats-sink/metrics.json \
  /etc/nats-sinks/observability.prometheus.json \
  --dry-run

Example dry-run output:

{"resourceMetrics":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"nats-sinks"}},{"key":"nats_sinks.namespace","value":{"stringValue":"nats_sinks"}}]},"scopeMetrics":[{"metrics":[{"description":"Raw JetStream messages fetched by the pull consumer.","name":"nats_sinks_messages_fetched_total","sum":{"aggregationTemporality":2,"dataPoints":[{"asDouble":256.0,"timeUnixNano":"1790000000000000000"}],"isMonotonic":true},"unit":"1"}],"scope":{"name":"nats-sinks.observability.otlp"}}]}]}

Live export remains disabled until the top-level observability policy and otlp.enabled are both true. The connector uses explicit timeouts, bounded retries, a maximum request size, and optional HTTP headers sourced from environment variables. See OpenTelemetry OTLP Integration for the full policy reference and service examples.

Names And Descriptions

List metric names:

nats-sink-metrics describe --format names

Example output:

messages_fetched_total
messages_prepared_total
messages_written_total
messages_acked_total
messages_terminated_total
messages_nacked_total
messages_failed_total
messages_dlq_total
batches_fetched_total
nats_fetch_seconds
message_mapping_seconds
sink_batches_written_total
sink_batch_write_seconds
oracle_execute_seconds
oracle_commit_seconds
mysql_execute_seconds
mysql_commit_seconds
message_ack_seconds
message_term_seconds
retry_backoff_delay_seconds
sink_write_errors_total
message_normalization_errors_total
payload_encryption_errors_total
dlq_publish_errors_total
ack_errors_total
term_errors_total
nats_connection_disconnected_total
nats_connection_reconnected_total
nats_connection_closed_total
nats_discovered_servers_total
nats_async_errors_total
fanout_route_matches_total
fanout_messages_routed_total
fanout_required_child_success_total
fanout_optional_child_timeout_total
fanout_ack_gate_wait_seconds
last_sink_success_epoch_seconds
current_batch_messages

Use the default table form to see descriptions:

nats-sink-metrics describe

Staleness Checks

Service scripts often need to know whether a process is still updating its snapshot. --stale-after-seconds fails when the snapshot is older than the provided limit:

nats-sink-metrics show .local/nats-sinks/metrics.json --stale-after-seconds 60

Example stale output:

Metrics snapshot is stale: age=141.8s limit=60.0s

Exit codes:

Exit code Meaning
0 Command succeeded.
2 Snapshot or display validation failed.
3 Snapshot is stale and --allow-stale was not used.
4 get could not find the metric and no default was provided.

Use --allow-stale when you want a warning but still want to print the snapshot:

nats-sink-metrics show .local/nats-sinks/metrics.json \
  --stale-after-seconds 60 \
  --allow-stale

Sorting And Filtering

The show command supports filters that are useful in shell scripts and operations dashboards:

nats-sink-metrics show .local/nats-sinks/metrics.json --kind counter
nats-sink-metrics show .local/nats-sinks/metrics.json --metric "*error*" --metric "*failed*"
nats-sink-metrics show .local/nats-sinks/metrics.json --sort value --reverse
nats-sink-metrics show .local/nats-sinks/metrics.json --format names --kind gauge

--metric uses shell-style glob matching against flattened metric row names. Observation rows include their statistic, for example sink_batch_write_seconds.count and sink_batch_write_seconds.sum.

Python Hooks

Applications can use the same metrics hooks directly:

from nats_sinks import JsonFileMetrics, JetStreamSinkRunner
from nats_sinks.file import FileSink

metrics = JsonFileMetrics(".local/nats-sinks/metrics.json", namespace="nats_sinks")
sink = FileSink(directory=".local/file-sink/events")

runner = JetStreamSinkRunner(
    nats_url="nats://localhost:4222",
    stream="ORDERS",
    consumer="orders-file-sink",
    subject="orders.*",
    sink=sink,
    metrics=metrics,
)

await runner.run()

When embedding OracleSink directly, pass the same recorder to both the sink and the runner so Oracle duplicate/conflict counters and core delivery counters land in one snapshot:

from nats_sinks import JsonFileMetrics, JetStreamSinkRunner
from nats_sinks.oracle import OracleSink

metrics = JsonFileMetrics(".local/nats-sinks/metrics.json", namespace="nats_sinks")
sink = OracleSink(
    dsn="localhost:1521/FREEPDB1",
    user="app_user",
    password_env="ORACLE_PASSWORD",
    table="NATS_SINK_EVENTS",
    mode="insert_ignore",
    metrics=metrics,
)

runner = JetStreamSinkRunner(
    nats_url="nats://localhost:4222",
    stream="ORDERS",
    consumer="orders-oracle-sink",
    subject="orders.*",
    sink=sink,
    metrics=metrics,
)

await runner.run()

Read a snapshot from another Python process:

from nats_sinks import load_metrics_snapshot, metric_rows_from_snapshot

snapshot = load_metrics_snapshot(".local/nats-sinks/metrics.json")
rows = metric_rows_from_snapshot(snapshot)

for row in rows:
    print(row.kind, row.name, row.value)

Use InMemoryMetrics for deterministic unit tests:

from nats_sinks import InMemoryMetrics, MetricNames

metrics = InMemoryMetrics()
metrics.increment(MetricNames.MESSAGES_FETCHED_TOTAL, 1)

assert metrics.counters[MetricNames.MESSAGES_FETCHED_TOTAL] == 1

Security And Privacy

Metrics snapshots intentionally do not include message bodies, decrypted payloads, passwords, tokens, private keys, Oracle connection strings, NATS credential files, or CA certificate contents. Keep it that way when adding new recorders or exporters.

Follow these rules:

  • do not add high-cardinality labels such as message IDs or stream sequence values to generic metrics,
  • do not emit classification values, free-form labels, subjects, or payload fields as unbounded metric labels,
  • keep snapshot files under an operator-controlled path,
  • keep snapshots out of git,
  • treat stale snapshots as an operational warning,
  • ensure metrics failures never cause early ACKs or skipped sink commits.

The snapshot reader validates JSON structure, rejects duplicate object keys, requires the expected schema, bounds file size to 1 MiB, and accepts only finite numeric metric values. Snapshot writing also rejects non-finite values such as NaN, Infinity, and -Infinity so local diagnostics and Prometheus exports always start from standards-compliant JSON. This keeps command-line inspection predictable even when the snapshot file is damaged or replaced by an unexpected file.

Metric Reference

The preferred metric suffixes are:

Metric suffix Type Meaning
messages_fetched_total counter Raw JetStream messages fetched by the pull consumer.
messages_prepared_total counter Messages converted into envelopes and transformed by core policies.
messages_written_total counter Messages reported durable by the destination sink.
messages_acked_total counter Messages acknowledged to JetStream after durable success or DLQ success.
messages_terminated_total counter Messages terminally acknowledged to JetStream after successful DLQ publication.
messages_nacked_total counter Messages negatively acknowledged after retryable failures.
messages_failed_total counter Messages that entered a failure path before ACK.
messages_dlq_total counter Messages published to a configured dead-letter subject.
batches_fetched_total counter Non-empty batches fetched from JetStream.
nats_fetch_seconds observation Seconds spent waiting for JetStream pull fetch calls.
message_mapping_seconds observation Seconds spent converting raw NATS messages into internal envelopes.
sink_batches_written_total counter Batches that returned durable success from sink.write_batch(...).
sink_batch_write_seconds observation Seconds spent inside sink.write_batch(...) for successful batches.
oracle_execute_seconds observation Seconds spent executing Oracle batch write statements before commit.
oracle_commit_seconds observation Seconds spent committing Oracle transactions.
message_ack_seconds observation Seconds spent ACKing JetStream messages after durable success.
message_term_seconds observation Seconds spent sending terminal acknowledgements after DLQ publication.
retry_backoff_delay_seconds observation Retry delay seconds selected before delayed NAK on retryable failures.
sink_write_errors_total counter Sink write failures raised before durable success.
message_normalization_errors_total counter Raw NATS messages that failed envelope normalization.
payload_encryption_errors_total counter Messages that failed core payload encryption before sink delivery.
dlq_publish_errors_total counter Messages whose DLQ publication failed before original ACK.
ack_errors_total counter Messages whose JetStream ACK failed after durable success.
ack_confirmation_attempts_total counter JetStream ACK confirmation attempts after durable success or DLQ success.
ack_confirmation_successes_total counter JetStream ACK confirmations accepted by the server after durable success or DLQ success.
ack_confirmation_timeouts_total counter JetStream ACK confirmation attempts that timed out after durable success or DLQ success.
ack_confirmation_failures_total counter JetStream ACK confirmation attempts that failed after durable success or DLQ success.
ack_confirmation_unsupported_total counter Messages where ACK confirmation was requested but unsupported by the client path.
ack_confirmation_seconds observation Elapsed seconds spent waiting for JetStream ACK confirmation responses.
term_errors_total counter Messages whose terminal acknowledgement failed after successful DLQ publication.
policy_messages_passed_total counter Messages accepted by pre-sink policy evaluation.
policy_messages_rejected_total counter Messages rejected by pre-sink policy evaluation before any sink write.
policy_batches_passed_total counter Batches whose messages all passed pre-sink policy evaluation.
policy_batches_rejected_total counter Batches with at least one message rejected by pre-sink policy evaluation.
policy_evaluation_errors_total counter Messages affected by unexpected pre-sink policy evaluation errors.
size_policy_messages_passed_total counter Messages accepted by the core size policy before sink delivery.
size_policy_messages_rejected_total counter Messages rejected by the core size policy before sink delivery.
size_policy_batches_passed_total counter Batches with at least one message accepted by the core size policy.
size_policy_batches_rejected_total counter Batches with at least one message rejected by the core size policy.
size_policy_evaluation_errors_total counter Messages affected by unexpected size-policy evaluation errors.
event_age_at_receive_seconds observation Event age in seconds when the runner received the message.
event_age_at_store_seconds observation Event age in seconds after the sink reported durable success.
events_stale_at_receive_total counter Events older than the configured stale threshold at receive time.
events_stale_at_store_total counter Events older than the configured stale threshold after durable sink success.
event_creation_timestamp_missing_total counter Messages without a usable publisher or JetStream creation timestamp.
event_creation_timestamp_malformed_total counter Messages with a malformed publisher creation timestamp header.
event_creation_timestamp_future_total counter Messages whose creation timestamp is beyond the configured future-skew tolerance.
event_source_clock_skew_seconds observation Positive source clock skew seconds observed for future-dated messages.
nats_connection_disconnected_total counter NATS client disconnect events observed by the runner.
nats_connection_reconnected_total counter NATS client reconnect events observed by the runner.
nats_connection_closed_total counter NATS client closed events observed by the runner.
nats_discovered_servers_total counter NATS discovered-server events observed by the runner.
nats_async_errors_total counter NATS asynchronous error callback events observed by the runner.
jetstream_advisories_received_total counter JetStream advisory messages accepted by the optional advisory monitor.
jetstream_advisories_filtered_total counter JetStream advisory messages ignored because they did not match allowed subjects.
jetstream_advisory_parse_errors_total counter JetStream advisory messages rejected by safe JSON parsing and validation.
jetstream_advisory_unsupported_total counter JetStream advisory messages observed with unsupported advisory kinds.
jetstream_advisory_max_deliver_total counter JetStream max-deliver advisories observed without exposing stream or consumer names.
jetstream_advisory_nak_total counter JetStream NAK advisories observed without exposing stream or consumer names.
jetstream_advisory_terminated_total counter JetStream terminal-ack advisories observed without exposing stream or consumer names.
jetstream_advisory_stream_quorum_lost_total counter JetStream stream quorum-lost advisories observed as aggregate events.
jetstream_advisory_consumer_quorum_lost_total counter JetStream consumer quorum-lost advisories observed as aggregate events.
jetstream_advisory_stream_leader_elected_total counter JetStream stream leader-election advisories observed as aggregate events.
jetstream_advisory_consumer_leader_elected_total counter JetStream consumer leader-election advisories observed as aggregate events.
jetstream_advisory_stream_action_total counter JetStream stream action advisories observed as aggregate events.
jetstream_advisory_consumer_action_total counter JetStream consumer action advisories observed as aggregate events.
jetstream_advisory_api_audit_total counter JetStream API audit advisories observed as aggregate events.
priority_lane_batches_total counter Batches ordered by enabled priority-lane scheduling.
priority_lane_messages_total counter Messages evaluated by priority-lane scheduling without exposing subjects.
priority_lane_defaulted_total counter Messages routed to the default priority lane because priority was missing or unknown.
priority_lane_rejected_total counter Messages rejected because priority metadata violated the priority-lane policy.
current_priority_lanes_active gauge Number of configured priority lanes represented in the active scheduled batch.
fanout_route_matches_total counter Route policy entries matched by fan-out delivery without exposing route names.
fanout_messages_routed_total counter Messages with at least one selected fan-out child sink target.
fanout_messages_no_route_total counter Messages rejected or ignored because routing selected no fan-out targets.
fanout_child_sinks_selected_total counter Selected fan-out child sink operations across routed messages.
current_fanout_child_sinks_selected gauge Child sink targets selected for the latest evaluated message.
fanout_required_child_success_total counter Required fan-out child sink operations that committed before ACK.
fanout_required_child_failure_total counter Required fan-out child sink operations that failed and blocked ACK.
fanout_optional_child_success_total counter Optional fan-out child sink operations that completed before the ACK gate released.
fanout_optional_child_failure_total counter Optional fan-out child sink operations that failed without blocking required ACK.
fanout_optional_child_timeout_total counter Optional fan-out child sink operations that exceeded their bounded ACK wait window.
fanout_messages_acked_total counter Fan-out messages whose original JetStream message became eligible for ACK.
fanout_messages_ack_blocked_total counter Fan-out messages whose original ACK was blocked by required failure.
fanout_ack_gate_wait_seconds observation Seconds spent waiting at the fan-out ACK gate.
fanout_batch_seconds observation Seconds spent processing a fan-out batch across selected child sinks.
oracle_conflicts_total counter Oracle write conflicts observed by OracleSink, such as duplicate-key conflicts.
oracle_duplicates_total counter Oracle rows identified as duplicate prior processing through idempotent handling.
oracle_duplicate_ignored_total counter Oracle duplicate rows safely ignored by insert_ignore mode.
oracle_duplicate_noop_total counter Oracle duplicate rows safely left unchanged by merge mode with no update columns.
oracle_merge_rows_total counter Oracle rows committed through merge mode.
oracle_merge_outcome_unknown_total counter Oracle merge rows where insert-versus-match outcome is not reliably exposed.
mysql_execute_seconds observation Seconds spent executing Oracle MySQL batch write statements before commit.
mysql_commit_seconds observation Seconds spent committing Oracle MySQL transactions.
mysql_conflicts_total counter Oracle MySQL duplicate-key conflicts observed while writing a batch.
mysql_duplicates_total counter Oracle MySQL rows identified as duplicate prior processing through idempotent handling.
mysql_duplicate_ignored_total counter Oracle MySQL duplicate rows safely ignored by insert_ignore mode.
mysql_duplicate_noop_total counter Oracle MySQL duplicate rows safely left unchanged by upsert mode with no update columns.
mysql_upsert_rows_total counter Oracle MySQL rows committed through upsert mode.
mysql_upsert_outcome_unknown_total counter Oracle MySQL upsert rows where insert-versus-match outcome is not reliably exposed.
last_sink_success_epoch_seconds gauge Unix epoch seconds for the latest durable sink success followed by ACK.
current_batch_messages gauge Number of messages in the active batch currently being processed.

Legacy aliases remain available with --include-legacy:

Legacy suffix Preferred suffix
messages_received_total messages_prepared_total
batches_written_total sink_batches_written_total
batch_write_seconds sink_batch_write_seconds
last_success_timestamp last_sink_success_epoch_seconds
current_batch_size current_batch_messages