Passa al contenuto principale

Postgres Exporter Monitoring

Scope​

postgres_exporter runs on db_nodes and binds to the Tailscale address on TCP 9187. Prometheus on akira-mgmt-01-staging scrapes the exporters and loads Postgres alert rules from /etc/prometheus/rules/postgres.yml.

Vault Secrets​

Add these variables to infra/group_vars/all/vault.yml and encrypt the file with Ansible Vault:

vault_postgres_exporter_user: "akira_monitoring"
vault_postgres_exporter_password: "REPLACE_ME_openssl_rand_hex_16"

Generate the password before editing Vault:

openssl rand -hex 16

The role creates the database login on the primary and grants pg_monitor. The replica receives the same DATA_SOURCE_NAME through replication of the role/user metadata and connects locally to akira.

Deploy​

cd /home/devcomm/akira/infra
ansible-playbook -i inventory/staging.yml playbooks/deploy_stateful.yml \
  --vault-password-file ~/.akira-vault-pass.txt \
  --limit db_nodes \
  --tags observability

ansible-playbook -i inventory/staging.yml playbooks/deploy_management.yml \
  --vault-password-file ~/.akira-vault-pass.txt \
  --limit management \
  --tags observability,grafana,prometheus,alertmanager

Verify​

curl -fsS http://akira-db-01-staging:9187/metrics | grep '^pg_up'
curl -fsS http://akira-db-02-staging:9187/metrics | grep '^pg_up'
curl -fsS http://127.0.0.1:9090/api/v1/rules | jq '.data.groups[].name'

Expected alerts:

  • PostgresDown
  • PostgresReplicationLagHigh
  • PostgresReplicationLagCritical
  • PostgresReplicationDisconnected
  • PostgresConnectionsHigh
  • PostgresLongRunningQuery