Skip to main content

Alertmanager

Alertmanager handles notification routing for Prometheus alerts. Lumie deploys Alertmanager and the Karma UI together from a single ArgoCD manifest file.

Source paths​

  • lumie-infra/observability/alertmanager/argocd.yaml
  • lumie-infra/observability/alertmanager/helm-values.yaml
  • lumie-infra/observability/alertmanager/common-values.yaml
  • lumie-infra/security/teleport/agent/helm-values.yaml

Runtime contract​

  • Alertmanager runs as one replica with persistence disabled.
  • The active configuration is mounted from the Vault-rendered alertmanager-config secret.
  • Karma runs as a separate application in the same namespace and points at http://alertmanager.alertmanager.svc.cluster.local:9093.

Routing behavior​

The Vault template in common-values.yaml defines:

  • one main email receiver
  • explicit null routing for Watchdog and InfoInhibitor
  • faster repeat intervals for severity: critical
  • inhibition from critical to warning on matching labels

That makes Vault, not the chart default config, the source of truth for routing behavior.

Access boundary​

  • Alertmanager itself is published as a Teleport app.
  • Karma is also published separately as the Teleport app karma.
  • Helm ingress is disabled for both.

Operational boundaries​

  • config.enabled: false in the Helm chart disables the chart's stock config and hands control to the mounted secret.
  • Persistence is off, so silences and notification-log state are not durable across pod replacement.
  • SMTP credentials and configuration come from the same Vault-backed secret path.

Failure modes​

  • If the VaultStaticSecret refresh breaks, Alertmanager may keep serving stale configuration or fail to reload new credentials.
  • Because persistence is disabled, restarting the pod clears in-memory silence state.
  • Karma health does not prove Alertmanager delivery health; it only proves the UI can reach the Alertmanager API.

Verification​

kubectl get applications.argoproj.io -n argocd alertmanager karma
kubectl get pods -n alertmanager
kubectl get secret -n alertmanager alertmanager-config
kubectl get secret -n alertmanager alertmanager-smtp
kubectl describe statefulset -n alertmanager alertmanager