Skip to main content
Central control plane is the topology for organisations that need more than one data-plane cluster but only one place to administer them. You run the control plane, the console, and the analytics stack; data planes in other clusters enrol into your control plane instead of into NeuralTrust SaaS. Select it with global.deploymentMode: saas.
Do not confuse this with the hosted SaaS model, where NeuralTrust operates both planes and there is nothing to install. Here the chart renders a control plane that behaves like the hosted one, but it is yours and it runs in your environment.
Choose it when a single External install cannot work because data has to stay where it was produced — separate business units, jurisdictions, or environments — but the console, alerting, and cross-cluster reporting have to be in one place. If every workload fits in one cluster, External is simpler. If NeuralTrust hosts the control plane, use Hybrid.

​
Architecture

Central control plane architecture: remote Hybrid clusters each run the request path — TrustGate on :8081 and :8082, TrustGuard on :8081, the Firewall on :8000 — with their own recommended PostgreSQL and Redis; raw payloads stay there. Each remote cluster opens four outbound connections on 443 to your central cluster: one config-sync endpoint each for AgentGateway and TrustGuard, the telemetry ingest gateway, and DataBridge. The central cluster runs the console and API on :8000, both product control planes on :8080, a collector on :4317 and :4318, ClickHouse, DataCore, and AlertEngine, plus recommended PostgreSQL and Redis of its own. The central cluster never dials into a remote one.

​
What runs in your central cluster

This mode is a superset of External: everything External deploys, plus three components and one behaviour change. PostgreSQL and Redis are recommended managed, outside the cluster; ClickHouse stays in-cluster on the central side only. Each remote cluster has its own recommended PostgreSQL and Redis — raw payloads never leave the cluster that produced them.

​
External baseline (also here)

​
Additions that make it a control plane for other clusters

The behaviour change is in DataCore: it runs with a hybrid residency backend rather than reading the local ClickHouse for everything, so entitled reads go out through DataBridge to the cluster that holds the data. The console, ClickHouse, AlertEngine, the bootstrap administrator, and the datastore split all behave exactly as in External — this page covers only what a control plane for other clusters adds on top.

​
Select the topology

Two values, on the central cluster:
global.controlPlane.domain is what remote clusters dial. Leave it empty to keep using NeuralTrust SaaS through global.saasRegion.

​
Cross-cluster endpoints

The chart derives all four endpoints from that one suffix: One value drives all four deliberately. A remote cluster that reached DataBridge on your domain but still dialled NeuralTrust for config-sync would half-work, and the half that broke would be silent. Set the same global.controlPlane.domain on the central cluster and on every remote cluster. DNS, certificates, and load-balancer provisioning for these names are operator prerequisites. There are no NeuralTrust hostnames or IPs anywhere in this topology, and no NeuralTrust inbound source IP — you own both ends.

​
Network rules

From each remote cluster (egress): allow TCP 443 to all four endpoints above on your own domain, plus the container registry (or mirror), the LLM upstreams, and that cluster’s own PostgreSQL and Redis. Remote planes keep raw payloads in that cluster’s PostgreSQL and export metadata over OTLP to https://telemetry.<domain> (TELEMETRY_EXPORTERS_METADATA=otlp, TELEMETRY_EXPORTERS_RAW=postgres), so the central ClickHouse holds metadata and the console reaches raw payloads on demand through DataBridge. Into the central cluster (ingress): it publishes those four endpoints, so it accepts inbound TCP 443 on each. Restrict them to the egress ranges of your data-plane clusters rather than leaving them open, and prefer an internal load-balancer scheme when the callers are on a private network — see Cloud notes. Where a link is private, you can skip publishing a config-sync Service at all with <product>.configSync.expose.enabled: false. Nothing here has to traverse the internet. VPC peering, a Transit Gateway or Direct Connect all work, and a private path is what makes the chart’s own certificates a sound choice — see TLS. Two of these carry long-lived streams. If an egress proxy or middlebox in the path reaps idle connections, config-sync and DataBridge drop and reconnect on that interval — check its idle timeout. A TLS-intercepting proxy breaks certificate verification unless its CA reaches the client: put it in the same bundle you point dataagent.databridge.tlsCa, <product>.configSync.tlsCa and global.clickstack.egress.tlsCaSecretName at, since each of those replaces the system roots rather than adding to them.

​
Prerequisites

Have these ready before installing. Everything else the chart does for you. The central cluster is also a full External install, so the External prerequisites apply to it: capacity for roughly 4–5 workers at 8 vCPU / 16–32 GiB, the gcr-secret image pull Secret, an onprem-superadmin Secret for the first console administrator, and a decision on the console hostname before install. Each is covered in External → Prerequisites.

​
Images

A central cluster pulls two images beyond the External set, both from the NeuralTrust registry and both covered by the gcr-secret pull secret you already have — databridge, and opentelemetry-collector-contrib for the ingest gateway. The collector is the same image and tag your Hybrid clusters run as their egress sidecar, so a mirror that already carries it needs nothing new. Mirroring for an air-gapped install works the same as everywhere else, and global.imageRegistry covers both of these. See Images and registries.

​
TLS

All four endpoints are dialled from other clusters, and each terminates TLS itself. So each needs a certificate covering its hostname, and each remote cluster needs to accept it. There are three ways to get there, and you can mix them per endpoint.
The chart refuses to render an endpoint with no certificate at all, rather than publishing one that nothing outside the cluster can verify. If you see a validation error naming DataBridge or one of the config-sync listeners, this is why, and the message lists the options above.
Which option fits depends on how your remote clusters reach the control plane.

​
A certificate the data planes already trust

If the data planes traverse the public internet, or you run an internal PKI whose root is already in their trust stores, supply the certificates:
Each certificate must cover its own hostname from the table above. Nothing further is needed on the remote side — the default trust store already accepts them. On AWS, an ACM certificate cannot serve the first three: TLS terminates inside the pod and ACM does not export private keys. It can serve the telemetry endpoint, which terminates at the ALB. Use cert-manager or your own PKI for the rest.

​
Chart-generated, for a control plane on a private network

When remote clusters arrive over VPC peering, Direct Connect or a private link, no public trust store is involved and there is nothing to buy. Let the chart mint every certificate and distribute the CA as configuration:
Each component mints its own CA, so a remote cluster needs all four. Export them as a single bundle from the central cluster:
Apply that Secret in every remote cluster and point the dialling legs at it, as in Remote data planes.
Minting a certificate does not make anyone trust it. Until the CA bundle is installed on a remote cluster, every handshake from it fails.
Keypairs survive upgrades, because agents hold long-lived streams that a reissue would drop, and are reissued only when the names they cover change — which is how retargeting global.controlPlane.domain reaches the certificates. Rerun the export script after any such change.

​
Keeping an endpoint off a load balancer

To reach a config-sync listener over peering without publishing a Service at all, set <product>.configSync.expose.enabled: false and route to the ClusterIP yourself. For endpoints that do get a load balancer, prefer an internal scheme when the callers are on a private network — see Cloud notes.

​
Telling data planes apart

DataBridge has to know which data plane is which. Two authentication modes do that:
token and dev are rejected in this mode. Both authenticate every data plane with one shared credential and then trust whichever tenant an agent claims for itself, which would let any enrolled data plane read another’s data.
Mint one enrolment token per remote data plane, each with its own instance ID. Reusing a single token across clusters collapses them into one identity in every query result and audit trail, and nothing later signals that it happened.

​
Secrets

Five credentials come from the shared platform Secret and are generated for you when the chart owns secrets: If you pre-provision secrets yourself — global.preserveExistingSecrets, global.autoGenerateSecrets: false, or global.platformSecret.existingSecret — all five must be present, and the two token keys must hold one identical value. When they drift, every agent connection returns 401 with nothing visibly wrong on either side. Omitting CONFIG_SYNC_SIGNING_SECRET fails later and further away: DataCore answers POST /v1/admin/credentials with HTTP 501 not implemented, so the console’s private-gateway wizard breaks after it has already created the gateway row. Running ./create-secrets.sh with DEPLOYMENT_MODE=saas writes all five correctly, including the alias.
Taking the platform Secret out of play in this mode is rejected at render time rather than at runtime, so a mistake here surfaces during helm upgrade instead of on first traffic.
See Secrets for how the shared platform Secret works generally.

​
Remote data planes

Each remote cluster is an ordinary Hybrid install pointed at your domain instead of at NeuralTrust:
Its enrolment and config-sync tokens are issued by your console, not by app.neuraltrust.ai. Everything else — the four operator-supplied Secrets, the values file, verification, exposing both TrustGate entry points — is the ordinary Hybrid install, with global.controlPlane.domain set and no NeuralTrust hostnames to allow.

​
Trusting a chart-generated control plane

Only needed when the central cluster serves chart-generated certificates. Apply the bundle produced by scripts/export-controlplane-ca.sh, then point all three dialling legs at it:
Three separate settings because the clients read their trust store differently. DataAgent and the config-sync clients take a file path, so they use the mount that global.customCaCert provides. The telemetry collector configures TLS from its own config file and ignores that mount, so it takes a Secret name instead.
tlsCa replaces the system roots on that connection rather than adding to them. If a leg also has to trust something else — a TLS-intercepting corporate proxy, for instance — put every CA in one bundle.
Before rolling out, confirm from inside each remote cluster that it can reach all four central endpoints. A data plane whose egress is blocked does not crash — it starts cleanly, serves its last-known-good configuration, and quietly stops receiving updates. See Network rules.

​
Cloud notes

Nothing in the chart is cloud-specific; the LoadBalancer Services take free-form annotations. On EKS with the AWS Load Balancer Controller:
Use the same shape for trustguard.configSync.expose. GKE already gives a layer 4 passthrough load balancer for a plain LoadBalancer Service; its private equivalent is networking.gke.io/load-balancer-type: "Internal", and on AKS it is service.beta.kubernetes.io/azure-load-balancer-internal: "true". Three things worth knowing on any cloud:
  • Use a layer 4 load balancer for DataBridge and config-sync. Both carry long-lived gRPC streams with TLS terminated in the pod. A layer 7 load balancer would have to re-terminate, and its idle timeout will cut streams that are healthy but quiet.
  • The ingest gateway is plain HTTP, so it goes through an Ingress and a layer 7 load balancer is fine. It is the only one of the four that is not layer 4.
  • An internal scheme keeps the whole topology off the public internet, which is what makes chart-generated certificates a sound production choice rather than a shortcut.
Central datastores — managed PostgreSQL, Redis, and ClickHouse — follow the normal External guidance.

​
Install

Start from the maintained neuraltrust-platform chart, then layer your platform, domain, ingress, TLS, and datastore choices:
Install and verify the central cluster first, then bring up one remote cluster as a canary before the rest.

​
Verify

Verify the central cluster on its own before any remote cluster dials it. A remote plane that enrols against a half-configured control plane fails in ways that look like network faults.
Everything in the External baseline should be running, plus the two workloads that make this a control plane for other clusters: databridge and clickstack-ingest-gateway. If either is absent, global.deploymentMode is not saas — and nothing else about the install will look wrong, because every other component is shared with External. Then confirm the four cross-cluster endpoints have addresses and that DNS resolves to them. Three are layer-4 Services and one is an Ingress, so they appear in different places:
The Service names are not the hostnames. Those three Services must each report an external address, and the DNS records from Cross-cluster endpoints — databridge.<domain>, agentgateway-configsync.<domain>, and trustguard-configsync.<domain> — are what point at them. Those DNS records are an operator prerequisite the chart cannot check. Until each name resolves, remote clusters cannot enrol; and because config-sync and DataBridge retry, the symptom is a remote data plane that stays Running but never becomes Ready, with nothing wrong on the central side. Sign in to the console and create the organisation exactly as in External → Verify; the console is the same install here.

​
Then enrol one remote cluster

Bring up a single remote data plane as a canary. Two log lines on the remote side prove both directions of the topology, and both are worth reading before the rest of the clusters follow:
Check the addr in that first line names your domain. A remote cluster that still reads databridge.<saas region> has an empty global.controlPlane.domain and has enrolled into NeuralTrust SaaS instead of into your control plane.
Finish with the validation checklist, which covers the parts no kubectl command reaches: TLS from the remote clusters, one enrolment identity per data plane, LLM and MCP traffic, and telemetry arriving in the central ClickHouse.

​
High availability

This topology has two availability questions, and they are not equally urgent. The data planes matter most, because they are on the request path. Each remote cluster climbs the same ladder as in Hybrid → High availability: one cluster across zones first, then twin clusters, and only for regional loss an active/passive pair on one writable PostgreSQL primary with region-local Redis, with just the active cluster running DataAgent. The three availability tiers side by side. Tier 1, one cluster multi-AZ: node pools in three availability zones, two or more replicas of TrustGate, TrustGuard and Firewall, and managed PostgreSQL and Redis with automatic failover inside the region; it survives node and zone loss and needs no promotion procedure or DNS work. Tier 2, twin clusters in one region: a serving cluster plus a second cluster to roll upgrades through, both against one PostgreSQL primary and one Redis primary shared inside the region; it survives cluster loss and bad upgrades. Tier 3, two regions active/passive: an active region running the only DataAgent, a warm passive region with no traffic, one writable PostgreSQL primary with a cross-region read replica, and region-local Redis per cluster; it survives regional loss and adds DNS promotion and a datastore runbook. Two invariants hold at every tier: exactly one writable PostgreSQL primary, and exactly one active DataAgent per gateway scope. The central cluster is not on the request path. If it goes down, every remote data plane keeps serving traffic from its last synchronized configuration. What stops is configuration updates, telemetry ingest, cross-cluster reporting, and the console. That is an availability target measured in minutes of operator inconvenience rather than dropped requests — as long as remote clusters keep their last-known-good configuration on persistent storage.

​
The central cluster

Most central clusters should sit on tier 1: one cluster across three zones with managed stores. It is not on the request path, so a promotion runbook is rarely worth its own operational cost. If a regional requirement forces tier 3, run it as an External active/passive pair on one writable PostgreSQL primary with a cross-region read replica and region-local Redis — see External → High availability for the shape, the ClickHouse caveat, and the promotion steps. Two things are specific to a central cluster: What a central-cluster promotion moves. Remote data planes are unaffected: they keep serving on their last-known-good configuration, and config sync and DataBridge retry until DNS converges — provided the snapshot cache is on persistent storage. DNS holds the four published hostnames: agentgateway-configsync, trustguard-configsync, telemetry, and databridge. They resolve to the active central cluster and must be repointed at the standby after a promotion. Both central clusters are identical installs, each with four L4 Services and its own CA, a console and API on :8000, and its own ClickHouse whose history does not follow the promotion. Both read and write one writable PostgreSQL primary with a cross-region read replica, promoted only if the failed region held the primary. Certificates must be valid from both central clusters: chart-generated CAs are per cluster, so install both bundles in every remote cluster or issue the four certificates from your own PKI. The four published endpoints have to follow the promotion. Remote clusters dial databridge.<domain>, telemetry.<domain>, and the two config-sync hostnames. Those names must resolve to the promoted cluster’s load balancers, so plan the DNS switch as part of the runbook. DataBridge streams and config-sync reconnect on their own once DNS converges; there is nothing to restart in the remote clusters. Certificates must be valid from both central clusters. Chart-generated certificates are minted per cluster with a per-cluster CA, so a remote cluster that trusts only the primary’s bundle fails every handshake after a promotion. Either export and install both clusters’ CA bundles in every remote cluster, or issue the four certificates from your own PKI or cert-manager so one bundle covers either cluster. The second option is much easier to operate — see TLS.
The ENROLMENT_SIGNING_SECRET and TELEMETRY_JWT_PRIVATE_KEY_PEM in the platform Secret must be identical in both central clusters. If each cluster generates its own, enrolment tokens and OTLP tokens minted by one are rejected by the other, and every remote agent returns 401 after a promotion. Pre-provision the platform Secret and apply the same one to both clusters — see Secrets.

​
Before you rely on it

  • Every remote cluster keeps its last-known-good configuration on persistent storage, so a central outage cannot take the request path down.
  • Central PostgreSQL is managed, with one primary and a cross-region read replica; each central region has its own Redis.
  • The four endpoint DNS records can be repointed, and you know how long convergence takes.
  • Both central clusters serve certificates every remote cluster trusts.
  • Both central clusters hold the same platform Secret values.
  • You have rehearsed a central promotion and confirmed remote data planes reconnect without intervention.

​
Next steps

Hybrid

The install every remote data-plane cluster runs.

External

The baseline your central cluster is built on.

Config sync

How remote products pull configuration from your control plane.

Secrets

Credential handling, including the four cross-cluster keys.

Validation checklist

What to prove before the install counts as done.