Skip to content

Security

Security defaults are part of the project design. nats-sinks sits between a message broker and durable destinations, so a mistake can expose data or lose processing guarantees. The core runtime and sinks must avoid silent data loss, secret leakage, unsafe configuration parsing, unsafe SQL construction, and non-deterministic tests.

This page is written for both application developers and operators. Developers should use it when adding sinks or changing runtime behavior. Operators should use it when preparing production configuration and deployment environments.

The guidance is also intended for sensitive operational and defence-adjacent deployments where payloads, headers, subjects, logs, and database rows may carry mission, logistics, personnel, platform, or other protected information. The project does not replace an organization's accreditation or security engineering process, but it aims to make the secure default the easy default.

Secrets

Do not commit credentials. Use environment variables, secret stores, and password_env for Oracle passwords.

The CLI redacts fields with names containing:

  • password
  • token
  • secret
  • private_key
  • credentials
  • creds
  • key_b64
  • key_material

Resolved passwords are not printed.

JSON Configuration

Runtime config uses JSON. JSON avoids parser features that are not needed for this package and gives operators a direct mapping to the validated Pydantic model tree.

The loader treats configuration as hostile input until it has passed several checks:

  • the file must be UTF-8,
  • the file must stay within the documented size limit,
  • the root value must be a JSON object,
  • duplicate object keys are rejected instead of accepting the last value,
  • non-standard constants such as NaN, Infinity, and -Infinity are rejected instead of being accepted as Python-specific extensions,
  • allowed environment overrides are applied through an explicit allow list,
  • the final structure is validated with Pydantic models, and
  • unknown fields are rejected in core configuration sections.

This fail-closed behavior helps operators catch mistakes before a sink connects to NATS or acknowledges any JetStream message.

Sink Connector Supply Chain

Sink connectors are code, not data. Loading a Python entry point imports and executes code from an installed distribution, so optional connector discovery is treated as a supply-chain trust decision.

Secure defaults:

  • Oracle Database, Oracle MySQL, Oracle NoSQL Database, Oracle Coherence Community Edition, FileSink, SpoolSink, HttpSink, and S3Sink are first-party built-in connectors and do not require plugin discovery.
  • Optional third-party discovery is disabled by default.
  • When discovery is enabled, plugins.allowed_sinks must explicitly list each external connector name.
  • The runtime never accepts module paths, class paths, or dynamic import names from JSON configuration.
  • External connectors must return a typed SinkConnector descriptor and must match the allow-listed entry-point name.
  • Production deployments should keep plugins.require_production_ready=true.

Before enabling an external connector, review the package source, dependency tree, release history, license, maintainer posture, and certification evidence. Install it through normal package-management and change-control processes. Do not install connector packages dynamically at runtime, and do not grant the service account broader filesystem, network, database, or cloud permissions merely because a connector asks for them.

Future Oracle-family sinks such as OCI Object Storage, Oracle Berkeley DB, and OCI Streaming are expected to be first-party connectors in this repository unless project governance changes that posture. Palantir Foundry, Palantir Gotham, and other third-party platform connectors require the same connector certification evidence before production recommendation.

Production Secure Development Baseline

Every feature should be reviewed as a small threat model before implementation. For nats-sinks, the most important assets are message payloads, message metadata, destination rows or files, secrets, encryption keys, idempotency keys, DLQ records, logs, and acknowledgement state.

flowchart LR
    Input[External input] --> Validate[Validate and bound]
    Validate --> Normalize[Normalize and type]
    Normalize --> Policy[Apply explicit policy]
    Policy --> Work[Perform durable work]
    Work --> Log[Log sanitized outcome]
    Work --> Ack[ACK only after durable success]

Use this baseline for code review and future releases:

  • Treat NATS subjects, headers, payloads, config files, environment variables, database rows, files, DLQ messages, and third-party responses as untrusted.
  • Fail closed when validation, authorization, dependency loading, encryption setup, sink selection, or policy evaluation is ambiguous.
  • Keep security-sensitive logic centralized and documented: configuration loading, TLS setup, credential resolution, SQL identifier validation, payload encryption, message authenticity verification, pre-sink policy enforcement, log sanitization, redaction, DLQ shaping, and ACK decisions.
  • Use allow-list validation for enum values, formats, lengths, ranges, SQL identifiers, file extensions, URL schemes, sink names, route names, and NATS subject patterns.
  • Use real parsers for structured formats and reject malformed, oversized, duplicate-key, non-standard-constant, or ambiguous input early.
  • Keep data separate from code. Use SQL bind variables for values, never shell out with untrusted strings, and never use eval, exec, pickle, unsafe YAML loaders, or user-controlled dynamic imports.
  • Bound batches, payloads, config files, retries, queue depths, parser depth, file output, and any count that could trigger CPU, memory, database, or native-library exhaustion.
  • Treat logs as an injection surface. Sanitize control characters and redact secrets while retaining enough subject, stream, sequence, sink, and error category context for operators.
  • Treat observability output as an information-sharing boundary. Metrics, timestamps, event freshness observations, source clock-skew values, failure counters, duplicate counters, and write timings can reveal operational tempo even without payloads. Use disabled-by-default observability policies and allow lists before publishing to Prometheus, OpenTelemetry Collectors, Elastic, Grafana Alloy, Splunk HEC, OCI Monitoring, StatsD, Datadog, Amazon CloudWatch, Azure Monitor, syslog, NATS monitoring snapshots, or any future monitoring platform. OTLP collector endpoints must not contain credentials, non-loopback endpoints must use HTTPS, and optional collector headers must be sourced from environment variables rather than stored in policy JSON. OCI Monitoring export must use least-privilege OCI identity, protected SDK config files only when needed, and low-cardinality dimensions that do not expose subjects, classification values, labels, mission metadata, hostnames, usernames, table names, file paths, message IDs, tenancy details, or credential material. Datadog tags, CloudWatch dimensions, and Azure Monitor dimensions must stay explicitly reviewed, low-cardinality, and free of sensitive metadata. Azure Monitor resource IDs, locations, and bearer tokens must not appear in dry-run output, logs, issues, or release evidence. Syslog export must be treated as redistribution to an operational logging fabric, not as a private local debug stream. NATS server monitoring endpoint values must also be selected with explicit endpoint and field allow lists before they are stored locally or rendered for Prometheus. JetStream advisories are also treated as sensitive operational signals: the optional advisory observer is disabled by default, subscribes only to configured advisory subjects, parses bounded JSON payloads, and emits aggregate counters without exporting stream names, consumer names, sequence numbers, or advisory payload bodies. Fan-out metrics must remain aggregate by default: do not export route names, child sink names, classification values, labels, file paths, or destination identifiers without a reviewed allow-list policy. Subject-family observability must use the disabled-by-default subject_metrics policy and prepared labeled_metrics rows. Exporters must not derive labels directly from raw NATS subjects, payloads, message IDs, classifications, file paths, table names, or credentials. Run the subject-aware certification tests and follow the Subject-Aware Observability Runbook before enabling a subject-family sharing boundary.
  • Treat local spool directories as protected custody locations. Spool records are encrypted by default, but operators must still restrict filesystem permissions, exclude spool paths from source control, monitor disk usage, rotate keys with care, and avoid exposing directory listings because wrapper metadata can still reveal record counts and rough priority ordering.
  • Treat read-only lineage helpers as an information-access boundary. Even when they do not modify data, they can reveal operational relationships between mission identifiers, tracks, tasking identifiers, subjects, and event timing. Use least-privilege database accounts, bounded result limits, allow-listed query fields, and redacted output. Do not paste raw lineage output containing operational identifiers into public issues, tickets, CI logs, or shared documentation.
  • Prefer least privilege for NATS accounts, Oracle users, service accounts, CI jobs, containers, cloud identities, filesystems, and documentation examples.
  • Keep cryptography on mature libraries and authenticated modes. Keys must stay separate from encrypted data and should be loaded through secure runtime configuration or a managed secret store.
  • Treat native libraries, database drivers, compression libraries, and FFI boundaries as memory-safety boundaries. Validate sizes and file metadata before handing untrusted data to them.
  • Use bounded retries, backoff, jitter, timeouts, connection pooling, backpressure, graceful shutdown, and idempotency for external operations.
  • Add tests for normal paths, negative paths, malformed input, duplicate messages, boundary values, dependency failures, and abuse cases. Treat crashes, hangs, flaky tests, memory growth, and data corruption as security and reliability signals.
  • Keep local and CI guardrails active. The repository includes scripts/secret-scan.sh as a lightweight high-confidence secret scanner, scripts/security.sh for secret scanning plus Bandit, CodeQL for static security analysis, dependency review for pull requests, Dependabot for updates, CycloneDX SBOM generation for release evidence, and pre-commit hooks for local checks.

Release SBOM guidance is documented in SBOM And Release Evidence. Dependency Graph and manifest-maintenance guidance is documented in Dependency Management. High-trust deployment guidance for pinned, hash-verified installs is documented in Hash-Verified Installs. Policy-controlled metric and selected NATS monitoring export is documented in Observability, with Prometheus-specific connector guidance kept in the Prometheus Integration sub-page and OpenTelemetry-specific connector guidance kept in the OpenTelemetry OTLP Integration sub-page. Elastic, Grafana Alloy, Splunk HEC, OCI Monitoring, StatsD, Datadog, Amazon CloudWatch, Azure Monitor, and syslog platform guidance is kept in their respective observability sub-pages so each sharing boundary has its own configuration and security review. The NATS server monitoring connector and delivery-boundary decision, including /jsz and /healthz handling, are documented in NATS Server Monitoring Integration. JetStream advisory observation is documented in Configuration and Metrics. It is observational only and must not be granted broader NATS permissions than the configured advisory subjects require. NATS runtime account authorization templates are documented in NATS Least-Privilege Permissions. Kubernetes-specific deployment examples, including service accounts, security contexts, Secret references, NetworkPolicy guidance, and resource limits, are documented in Kubernetes Deployment. Container image hardening guidance, including Oracle Linux 9 slim FIPS image construction, non-root UID/GID 10001, read-only root filesystem behavior, writable mount boundaries, OCI labels, vulnerability scanning expectations, and defence-oriented accreditation caveats, is documented in Production Container Hardening.

The detailed maintainer-requested control review is tracked in Security Rule Review. That page records the current status of all 316 secure-development guidance points as applied, already covered, partially covered, roadmap, or not applicable for the current package surface.

Pre-Sink Policy Gate

The optional size_policy gate is the first general resource-control layer for message shape. When enabled, it bounds sink-bound payload bytes, header count, header lengths, label count, label lengths, mission metadata JSON size, standard metadata JSON size, approximate normalized record size, and accepted batch size before any sink write. It is intentionally sink-neutral so Oracle, file, and future sinks all receive the same protection. Rejections are permanent validation failures and follow the same DLQ-before-ACK rule as other pre-sink failures.

Use security_labels when deployments need a structured data-centric security label profile with releasability, handling caveats, owner, originator, policy identifier, and retention category. The profile is validated before sink delivery and stored as JSON in Oracle and file outputs. It can inform policy decisions, but it is not an access-control mechanism by itself. Keep authorization in Oracle, the application, IAM, compartment policies, database views, VPD policies, or the destination platform that actually controls access. See Data-Centric Security Label Profile.

Use size_policy for broad resource and abuse-case controls, then use pre_sink_policy for semantic requirements such as classification, required labels, encryption, or approved mission metadata keys. Detailed configuration is documented in Configuration, and operational tuning guidance is in Message Sizing.

The optional pre_sink_policy gate is a security and correctness control that runs after the core has built a validated NatsEnvelope, resolved generic metadata, validated mission metadata, and optionally encrypted the payload. It runs before any sink writes. This placement lets one destination-neutral policy protect Oracle, file, and future sinks without duplicating checks in every destination module.

The gate is disabled by default. When enabled, it is fail-closed by default: subjects that do not match any rule are rejected unless an operator explicitly sets unmatched_subject_action to allow. Rules are intentionally simple and auditable. They can require priority, classification, labels, mission metadata, encrypted payloads, mission metadata root-key allow lists, and maximum sink-bound payload size.

The policy engine does not execute Python code, templates, regular expressions, dynamic imports, or user-provided expressions. Rejections use sanitized reason codes and must not include payload bodies, credentials, table names, private network locators, or other sensitive values. A policy rejection is treated as a permanent validation failure: the message never reaches the sink, and if DLQ is configured the original JetStream message is ACKed or terminally acknowledged only after DLQ publication succeeds.

Use the policy gate for deployment-wide controls such as:

  • requiring encrypted payloads for selected subjects,
  • requiring classification for restricted operational streams,
  • requiring labels that identify an audit lane or mission-support workflow,
  • rejecting mission metadata fields outside an approved root-key vocabulary,
  • bounding payload size before handing data to destination drivers.

The full configuration reference is in Configuration.

Payload Privacy

Payload logging is disabled by default. Treat message payloads as sensitive unless a deployment explicitly proves otherwise.

In mission environments, treat subjects and headers as potentially sensitive as well. Even when the payload is encrypted, metadata can reveal operational tempo, routing, classification, source systems, or unit and platform naming conventions. Use subject design, header minimization, access controls, and retention policies accordingly.

Domain-specific metadata can be sensitive too. A field such as mission_metadata.f2t2ea.phase may reveal workflow stage or operational tempo even when the value is only used for custody and audit. Public examples must stay synthetic, and live deployments should protect mission metadata with the same access-control, retention, and release rules used for the surrounding event. The core mission metadata feature validates object structure, size, duplicate keys, non-standard JSON constants, and secret-looking field names before sinks see the message. Deployments can also enable operator-defined metadata profile registry checks so approved profile names fail closed on missing required fields, wrong JSON types, unsupported enum values, excessive string length, unexpected root fields, or nested object structures that exceed configured limits. See Mission Metadata and F2T2EA Event Phase Tagging for safe example wording and non-goals. The broader Defence And Mission Support blueprint set shows how to document mission-oriented deployments without placing live operational details, credentials, network locators, or sensitive payloads into public docs or GitHub Issues.

Optional core payload encryption can protect message bodies before they are stored by a sink. When enabled, the runner encrypts only the body bytes and leaves metadata clear for routing, idempotency, and troubleshooting. Supported algorithms are AES-256-GCM and AES-256-CCM through the optional nats-sinks[crypto] extra. Encryption can be enabled globally for every subject consumed by a runner or selectively through ordered encryption.rules that match NATS subjects.

Use encryption.key_b64_env for key material and keep the referenced environment variable in a secret manager, protected service environment file, or platform secret injection mechanism. Do not commit direct key_b64 values. Rule order is security-sensitive because the first matching rule wins and a disabled rule intentionally leaves matching payloads unchanged. See Payload Encryption for the full design and configuration reference.

Optional message authenticity verification can prove that an approved producer signed the message body and selected metadata before the core passes the envelope to any sink. This is not the same as NATS authentication or TLS. NATS controls who can connect and publish; message authenticity checks whether a specific delivered event matches a configured producer signature policy.

Authenticity rules are subject scoped, algorithm allow-listed, and fail closed by default. HMAC-SHA256 uses constant-time comparison for shared-secret deployments. Ed25519 lets the sink runtime hold only public verification key material while producers keep private signing keys. A verification rejection is a permanent pre-sink failure: the message never reaches Oracle Database, Oracle MySQL, Oracle Coherence Community Edition, file, spool, or future sinks, and the original JetStream message is ACKed only after successful DLQ publication when DLQ is configured. See Message Authenticity.

Optional tamper-evident custody metadata can help auditors detect unexpected changes to stored payload or metadata records. When enabled, the runner hashes the normalized payload and stable metadata before sink delivery and asks sinks to persist the resulting custody object next to the durable record. This is not encryption and not a digital signature. A hash can reveal that two messages carried the same payload or metadata shape, even when the payload body is encrypted.

Use custody metadata only after reviewing the privacy tradeoff for the deployment. Protect persisted custody objects with the same access controls, backup rules, retention rules, and issue-comment hygiene used for the surrounding event data. The optional key_id field is a non-secret policy or future key-version identifier; never place key material, tokens, locations, or private operational detail in it. See Tamper-Evident Custody Metadata.

Headers-only delivery can reduce payload exposure to a sink process, but it is not a complete confidentiality control. Subjects, headers, stream metadata, sequence numbers, message size, priority, classification, labels, mission metadata, and DLQ records can remain sensitive. The headers-only design is documented in Headers-Only Delivery so omitted bodies are never silently stored as if producers sent empty payloads.

Ordered-consumer inspection can expose sensitive stream content even when payloads are hidden, because subjects, headers, sequence numbers, priority, classification, labels, and timing can reveal operational context. The nats-sink inspect-ordered command is read-only, bounded, redacted by default, and separated from durable sink writes. Production replay into sinks should use durable pull consumers and commit-then-acknowledge rather than ordered inspection consumers. See Ordered Consumer Evaluation.

Push-consumer support is disabled by default and must be explicitly enabled. It is manual ACK only, bounded by explicit pending-message and pending-byte limits, and protected against unbounded callback intake. Flow-control errors, heartbeat events, callback exceptions, and queue saturation must be logged without payloads, credentials, private subject families, or sensitive metadata values. The certification tests prove these controls without exposing real subjects or payloads. See Push Consumer Evaluation.

Subject-aware observability has a disabled-by-default policy model, bounded prepared labeled_metrics rows, and a focused certification suite. NATS subjects can reveal mission, tenant, platform, environment, or routing structure, and per-subject metrics can create high-cardinality series. Any connector that renders subject-family export must use explicit allow lists, stable low-cardinality labels, caps, deterministic overflow behavior, and fail-closed export decisions without changing ACK behavior. See Subject-Aware Observability Evaluation and the Subject-Aware Observability Runbook.

Route target ACK-gating also follows fail-closed defaults. Every selected target is required unless configuration explicitly marks it optional. Optional targets require a known sink type, bounded wait values, and sanitized logging. Those logs contain sink names and outcome categories only. They must not include payloads, credentials, connection strings, raw file paths, private subjects, or sensitive classification and label values.

Oracle NoSQL Database configuration is also a trust boundary. Endpoints, deployment modes, auth modes, table names, row field names, key prefixes, table creation controls, and size limits are allow-list validated before SDK handle creation. The sink stores the full normalized event in a configured JSON value field and must not log payloads, OCI private-key material, passphrases, or table row contents by default.

S3-compatible object sink configuration is also a trust boundary. Bucket names, prefixes, endpoints, credential-source references, key strategies, object suffixes, duplicate policies, metadata modes, and retry budgets are allow-list validated before any SDK client is created. Object metadata must remain low-cardinality and non-secret. Do not put raw subjects, classification values, labels, message IDs, payload fields, private endpoints, or credentials into object metadata, issue evidence, or public reports.

Key rotation should use explicit key_id values. New runtime configuration encrypts with the active key, while authorized verification, replay, or migration tooling can use PayloadKeyRegistry to decrypt records written with old and new keys during the same retention window. nats-sinks does not import cloud secret-manager SDKs in the core package; retrieve key material through your approved platform mechanism and pass only the resolved base64 key or environment-variable reference into the configuration. Treat key identifiers as clear operational metadata and avoid names that disclose sensitive missions, networks, or customer details.

flowchart LR
    Plain[Payload bytes] --> Encrypt[Core payload encryption]
    Encrypt --> Stored[Encrypted payload stored by sink]
    Metadata[Subject, headers, stream sequence] --> Stored

NATS Security

Production deployments should use:

  • TLS,
  • authenticated connections,
  • TLS verification enabled.

Do not disable TLS verification outside controlled local development.

Private defence or mission networks often use internal certificate authorities. Use nats.tls_ca_file to trust the local CA rather than disabling verification. Disabling verification can turn a protected transport into an unauthenticated channel and should not be used for deployed services.

Supported NATS client authentication modes in this release are documented in NATS Connections And Authentication. In short:

  • use nats.token_env for token authentication,
  • use nats.user and nats.password_env for username/password authentication,
  • use the same client-side username/password configuration when the NATS server stores a bcrypted password hash,
  • use nats.creds_file for NATS credentials-file and decentralized JWT user workflows,
  • use nats.nkey_seed_file for NKEY challenge authentication,
  • use nats.tls_ca_file to trust a local CA certificate for private or self-signed NATS server certificates, and
  • use nats.tls_cert_file with nats.tls_key_file when the deployment requires a client certificate during the TLS handshake.

Bcrypt is a server-side storage control. The client still needs the clear-text password to authenticate, so username/password and token authentication should use TLS in production.

Credentials files, NKEY seed files, TLS private keys, and client certificate paths are treated as identity material. They must be mounted from a secret location, kept out of Git, excluded from issue comments, and protected with least-privilege file permissions. nats-sinks redacts those fields in effective configuration output and validates that path strings are not empty, not surrounded by whitespace, and do not contain control characters.

WebSocket transport is supported with explicit guardrails. Prefer wss://, verify certificates, use tls_ca_file for private CAs, and reserve ws:// for local labs or explicitly approved controlled networks. Do not place credentials in WebSocket URLs. Optional WebSocket headers must be bounded, validated, and redacted; sensitive header values such as Authorization or Cookie must come from environment variables through nats.websocket_headers_env. Assume that reverse proxies may log paths, headers, and client metadata, and review proxy retention before using WebSocket transport for sensitive event custody. See WebSocket Connection Evaluation.

Use least-privilege NATS subject permissions for the runtime account. A standard worker should be able to request from its configured pull consumer, receive inbox responses, ACK messages it has received, and publish to the configured DLQ subject only when DLQ is enabled. It should not need broad publish, broad subscribe, stream administration, or source-subject publish rights. See NATS Least-Privilege Permissions for templates and validation checklists.

Durable consumer creation and reconciliation are explicitly controlled by consumer_management.mode. The safest production posture is bind_only with a separate administrative identity creating or updating consumers. If create_if_missing or reconcile is enabled, grant only the narrow JetStream consumer-management permissions required for the configured stream and durable consumer. Do not give the sink worker account broad stream-management or account-management authority.

Stream creation and retention planning are separate administrative concerns. The nats-sink stream-plan helper is safe to run with ordinary local access because it is offline: it does not connect to NATS, does not resolve credentials, and does not mutate streams. Treat its output as a review artifact for a separate NATS administrator or platform automation identity. That administrative identity may need permissions such as $JS.API.STREAM.CREATE.<STREAM> or $JS.API.STREAM.UPDATE.<STREAM>, but the long-running sink worker should not receive those grants by default. See JetStream Stream Management Planning.

Richer consumer policy fields are also security-relevant. Multiple filter_subjects can widen what the worker receives, so nats-sinks requires each configured filter to stay within nats.subject and documents the NATS permission tradeoff for plural filters. Consumer metadata is bounded and rejects secret-looking keys, but it is still visible to operators with JetStream management access. Use it only for low-sensitivity labels such as component ownership or deployment purpose, never for credentials, payload fragments, mission details, private endpoints, or compartment names.

flowchart LR
    Env[Secret environment variable] --> CLI[nats-sink]
    CLI --> TLS[TLS connection]
    TLS --> NATS[NATS server]
    NATS --> Hash[Server-side token or bcrypt verification]

TLS certificate files, NKEY seed files, and NATS credentials files are supported client-side authentication inputs. The NATS operator must still certify server-side account resolvers, trust anchors, subject permissions, and identity-to-authorization policy for the target deployment.

Oracle Security

Use a least-privilege Oracle user. The user should need only the permissions required to write to the configured table.

For classified, restricted, or compartmented event stores, keep schema ownership separate from runtime ingestion. The sink runtime account should be able to insert or merge into the approved table shape, not administer the database or remove evidence of prior ingestion.

When Oracle staging-table mode is enabled, apply the same least-privilege rule to both database objects. The runtime account needs only the permissions required to insert rows into the staging table, read the active batch from the staging table for the set-based merge, delete only its staged batch when cleanup="delete_on_success" is used, and insert or merge into the approved target table. It should not receive broad schema administration, drop-table, or unrelated data access permissions merely because staging is enabled.

For Oracle Autonomous Database wallet/mTLS connections, treat the wallet files as secret runtime material. Do not commit Wallet_*.zip, ewallet.pem, cwallet.sso, ewallet.p12, tnsnames.ora from private environments, or wallet passwords. Store wallet files in an ignored local directory, a protected host path, or a secret volume. Use sink.wallet_password_env instead of embedding wallet passwords in JSON.

SQL security controls:

  • identifiers are allow-list validated,
  • values use bind variables,
  • bind values are not logged by default,
  • schema creation is disabled unless explicitly enabled.

File Sink Security

The file sink writes local JSON files and therefore depends on operating-system filesystem controls. Run it as a dedicated service user and make the output directory writable only by that user and trusted operators.

For operational audit, disconnected transfer, or air-gapped handoff patterns, consider the file output directory part of the mission data boundary. Apply the same classification handling, retention, backup, and media control rules you would apply to any other repository of event records.

The sink sanitizes subject names, stream names, and message IDs before they become path components, and it verifies that resolved output paths remain under the configured root directory. Operators should still treat the configured directory as sensitive because generated files may contain payloads, headers, and metadata.

If gzip compression is enabled, the compressed files may still contain sensitive payloads and metadata. Compression is not encryption. Protect .json.gz files with the same filesystem permissions, retention policy, and backup controls as uncompressed .json files.

Do not point the file sink at a source-code directory, shared temporary directory, or path served directly by a web server. Keep generated output under an application data path such as /var/lib/nats-sinks/events, and apply your normal backup, retention, and access-control policies.

For cross-domain handoff preparation, nats-sinks can help create reviewable records and package-shaped evidence, but it is not a cross-domain guard, certification boundary, release authority, data diode, or sanitizer. Keep transfer decisions in approved systems outside this package. If you use the Cross-Domain Handoff Package blueprint, enforce path normalization, bounded file counts, bounded payload sizes, and secret-free manifests before committing the package.

Palantir Foundry Sink Security

The Palantir Foundry sink is experimental and should be treated as a controlled integration boundary. Configure only reviewed Foundry stream push endpoints, use endpoint_allowed_hosts, and keep tokens, OAuth2 client identifiers, and client secrets in environment variables or a protected service environment file.

The connector validates endpoint schemes, host allow-lists, field names, batch sizes, record sizes, response sizes, timeout values, and authentication mode combinations before attempting a write. It rejects private URL query strings and URL userinfo so tokens do not move through the endpoint field.

Runtime errors intentionally avoid printing the Foundry endpoint, token values, client identifiers, response bodies, stream resource identifiers, subjects, or payloads. Treat live Foundry responses as untrusted input: malformed, oversized, rejected, partial, or ambiguous responses fail closed and prevent ACK until the normal retry, redelivery, or DLQ policy completes.

Local fake-client certification is not production certification. Before using the connector in production, validate it against an approved Foundry test environment with least-privilege application permissions and sanitized evidence.

Palantir Gotham Sink Security

The Palantir Gotham sink is experimental and should be treated as a controlled RevDB object handoff boundary. Configure only reviewed Gotham base URLs, use endpoint_allowed_hosts, and keep tokens, OAuth2 client identifiers, and client secrets in environment variables or a protected service environment file.

The connector validates endpoint schemes, host allow-lists, object type names, property type names, security portion markings, batch sizes, object sizes, response sizes, timeout values, and authentication mode combinations before attempting a write. It builds the object-create endpoint from a base URL and a validated object type so configuration cannot supply arbitrary API paths.

Runtime errors intentionally avoid printing the Gotham endpoint, token values, client identifiers, response bodies, object types, property types, primary keys, subjects, or payloads. Treat live Gotham responses as untrusted input: malformed, oversized, missing-primary-key, rejected, partial, timed-out, or ambiguous responses fail closed and prevent ACK until the normal retry, redelivery, or DLQ policy completes.

Local fake-client certification is not production certification. Before using the connector in production, validate it against an approved Gotham test environment with least-privilege application permissions, an approved object model, and sanitized evidence.

Secure Failure Flow

flowchart TD
    Failure[Processing failure] --> Classify{Temporary or permanent?}
    Classify -->|Temporary| Redeliver[No ACK, redelivery allowed]
    Classify -->|Permanent with DLQ| Publish[Publish DLQ JSON]
    Publish -->|success| Ack[ACK original]
    Publish -->|failure| Redeliver

Dependency Hygiene

Dependencies are intentionally limited. CI includes formatting, linting, type checking, unit tests, package build checks, dependency review, CodeQL, and Bandit.

Release builds also generate SHA256SUMS so operators can verify GitHub Release assets before promoting them into controlled mirrors or wheelhouses. Hash verification is a deployment control rather than a runtime feature: it does not change ACK behavior, sink writes, payload encryption, or idempotency.