Skip to main content

Generated at build time from charts/racora-monitoring/README.md (helm-docs: values.yaml + README.md.gotmpl); edit the source, not this page.

racora-monitoring

The observability stack for Racora. Four components, one wire contract — every producer sends OTLP to the collector, which fans out to ClickHouse (logs) and Tempo (traces); Grafana reads from both with pre-provisioned datasources.

What this installs​

ComponentImageRole
OTel Collectormirrored-opentelemetry-collector-contrib:0.115.1Single OTLP ingress (gRPC :4317 + HTTP :4318), fans out logs→ClickHouse, traces→Tempo; its own metrics on :8888 (exposed, not scraped)
ClickHousemirrored-clickhouse-server:24.10Log SQL store. The otel_logs table is auto-created by the exporter on first connect (the exporter is on the logs pipeline only — no trace or metric tables)
Tempomirrored-tempo:2.6.1Distributed-trace store, single-binary monolithic mode; metrics-generator enabled with no remote-write target
Grafanamirrored-grafana:11.4.0UI with Tempo + ClickHouse datasources and the dashboards under files/dashboards/ pre-provisioned (folder "Racora"), NodePort 30300

The images are the upstream ones, mirrored unmodified into the racora registry under these mirrored-* names (versions.yaml → third_party); the distribution re-roots them with global.systemDefaultRegistry.

All four are independently togglable (<component>.enabled defaults to true).

Requires​

  • racora-base (monitoring namespace)

No CRD dependency — this stack is RAN-agnostic and could be reused unchanged for any cluster.

Install​

helm install racora-monitoring ./charts/racora-monitoring

Values​

KeyTypeDefaultDescription
namespacestring"monitoring"Namespace the monitoring stack deploys into. Must already exist (racora-base creates it).
clusterLabelstring"racora"Cluster identity attached to every record by the OTel collector. Useful when federating multiple gNB clusters into a single observability backend (each cluster's records are disambiguated by this label). Matches the OTel resource attribute set by the python services (cns_logging.configure_otel).
imageRegistrystring""Image registry prefix. The distribution sets global.systemDefaultRegistry (the racora registry) and the third-party images below are mirrored into it under mirrored-* names — one re-rootable namespace, no Docker Hub pulls at runtime. Left empty here; the umbrella's global value drives it.
clickhouse.enabledbooltrueRender ClickHouse.
clickhouse.image.repositorystring"mirrored-clickhouse-server"ClickHouse image name.
clickhouse.image.tagstring"24.10"ClickHouse image tag; the upstream image, mirrored unmodified into the racora registry; pinned in versions.yaml (third_party).
clickhouse.image.pullPolicystring"IfNotPresent"Image pull policy — IfNotPresent, because the images ship with the distribution.
clickhouse.dbstring"otel"Database the collector's ClickHouse exporter writes otel_logs into.
clickhouse.userstring"default"ClickHouse user; no password — the store is in-cluster only.
clickhouse.resourcesobject{"limits":{"cpu":2,"memory":"4Gi"},"requests":{"cpu":"200m","memory":"512Mi"}}Pod resources for ClickHouse.
clickhouse.storage.persistentboolfalseUse a PersistentVolumeClaim instead of emptyDir, so the data survives pod restarts.
clickhouse.storage.sizestring"50Gi"Claim size when persistent.
clickhouse.storage.storageClassNamestring""StorageClass for the claim; empty = the cluster default.
tempo.enabledbooltrueRender Tempo.
tempo.image.repositorystring"mirrored-tempo"Tempo image name.
tempo.image.tagstring"2.6.1"Tempo image tag; the upstream image, mirrored unmodified into the racora registry; pinned in versions.yaml (third_party).
tempo.image.pullPolicystring"IfNotPresent"Image pull policy — IfNotPresent, because the images ship with the distribution.
tempo.blockRetentionstring"24h"Trace retention (Tempo compactor block_retention).
tempo.resourcesobject{"limits":{"cpu":1,"memory":"2Gi"},"requests":{"cpu":"100m","memory":"256Mi"}}Pod resources for Tempo.
tempo.storage.persistentboolfalseUse a PersistentVolumeClaim instead of emptyDir, so the data survives pod restarts.
tempo.storage.sizestring"20Gi"Claim size when persistent.
tempo.storage.storageClassNamestring""StorageClass for the claim; empty = the cluster default.
otelCollector.enabledbooltrueRender OTel Collector.
otelCollector.image.repositorystring"mirrored-opentelemetry-collector-contrib"OTel Collector image name.
otelCollector.image.tagstring"0.115.1"OTel Collector image tag; the upstream image, mirrored unmodified into the racora registry; pinned in versions.yaml (third_party).
otelCollector.image.pullPolicystring"IfNotPresent"Image pull policy — IfNotPresent, because the images ship with the distribution.
otelCollector.resourcesobject{"limits":{"cpu":1,"memory":"1Gi"},"requests":{"cpu":"100m","memory":"256Mi"}}Pod resources for OTel Collector.
otelCollector.logsTtlstring"24h"Retention for logs in ClickHouse — independent of trace retention (which lives in tempo.blockRetention).
grafana.enabledbooltrueRender Grafana.
grafana.image.repositorystring"mirrored-grafana"Grafana image name.
grafana.image.tagstring"11.4.0"Grafana image tag; the upstream image, mirrored unmodified into the racora registry; pinned in versions.yaml (third_party).
grafana.image.pullPolicystring"IfNotPresent"Image pull policy — IfNotPresent, because the images ship with the distribution.
grafana.service.typestring"NodePort"Service type: NodePort for browser access from outside the cluster, ClusterIP to port-forward instead.
grafana.service.nodePortint30300NodePort used when service.type is NodePort.
grafana.anonymousAccessbooltrueAnonymous access to the UI (no login) with the role below. Viewer can browse dashboards and query; Editor could also change them. Tighten (anonymousAccess: false) when exposing Grafana beyond a lab.
grafana.anonymousRolestring"Viewer"Role anonymous users get: Viewer (browse and query) or Editor (also change dashboards).
nodeSelectorobject{}Node selector for the monitoring workloads (applied per workload unless overridden).
tolerationslist[]Tolerations for the monitoring workloads (applied per workload unless overridden).
affinityobject{}Affinity rules for the monitoring workloads (applied per workload unless overridden).

Producer wiring​

Every Racora/OCUDU service writes OTLP to the collector at otel-collector.monitoring.svc.cluster.local. The collector accepts both OTLP transports, and the producers split by SDK:

  • gRPC :4317 — the Python components (controller, CU-IP) via cns_logging.configure_otel()
  • HTTP :4318 — the gNB/DU (its C++ OTel exporter posts to /v1/traces / /v1/logs; the controller injects the endpoint into each DU Deployment)

Configured per-service via the OTEL_EXPORTER_OTLP_ENDPOINT env var. The Python bootstrap soft-imports OTel packages so absent collector = stdout logging, no spans, no failure.

Storage caveat​

ClickHouse + Tempo both default to emptyDir. Pod restart wipes the data. Fine for the iteration-mode reality where the system is reset frequently. For real distribution use, flip <component>.storage.persistent: true in values; the chart provisions a PVC automatically.

Verify​

kubectl get pods -n monitoring
# Expected:
# clickhouse-xxx 1/1 Running
# tempo-xxx 1/1 Running
# otel-collector-xxx 1/1 Running
# grafana-xxx 1/1 Running

# Grafana (if NodePort):
curl -s http://localhost:30300/api/health

Once installed, the "Failed to export logs to otel-collector..." warnings in every other service's logs should immediately stop.