Vault Deployment

Declarative, GitOps deployment of HashiCorp Vault on an OCI ARM VM.

• Terraform — VCN/subnet/security list, A1.Flex VM (2 OCPU / 12 GB, Ubuntu 24.04), OCI KMS vault + key for auto-unseal, instance-principal IAM, Cloudflare DNS, Object Storage bucket for backups. State lives on OCI Object Storage (S3-compatible).
• cloud-init — OS layer only: Docker, OCI CLI, fail2ban, key-only SSH, /opt/vault.
• Ansible — renders vault.hcl + Caddyfile from Terraform outputs, installs the systemd-managed compose stack and the daily snapshot timer. Gates on cloud-init status --wait, so the OS and app layers never race.
• App — vault (raft storage, KMS auto-unseal, mlock) behind caddy (Let's Encrypt).

Prerequisites

1. State backend: OCI bucket vault-tfstate, a Customer Secret Key.
2. GitHub secrets

Setup

1. Initialise Vault

Auto-unseal handles every reboot after this, but init is manual so recovery material never touches CI logs.

ssh ubuntu@vault.cloud.lippok.dev
cd /opt/vault
docker compose exec vault vault operator init

Store the recovery keys and initial root token securely.

2. Enable audit logging

cd /opt/vault
sudo docker compose exec vault vault login <root-token>

sudo docker compose exec vault vault audit enable file file_path=/vault/file/audit.log

3. Backups

The vault-snapshot.timer runs daily and uploads a raft snapshot to the vault-backups bucket via the instance principal. It needs a token first:

# create a scoped backup policy + a periodic token, then store it on the host
cd /opt/vault
sudo docker compose exec -T vault vault policy write backup - <<'EOF'
path "sys/storage/raft/snapshot" { capabilities = ["read"] }
EOF

sudo docker compose exec -T vault \
  vault token create -policy=backup -period=768h -orphan -field=token \
  | sudo install -m600 /dev/stdin /etc/vault-backup/token

# verify
sudo systemctl start vault-snapshot.service
oci os object list --bucket-name vault-backups --auth instance_principal

-orphan matters: token revocation in Vault cascades to children, so a backup token created as a child of the root token dies the moment you do step 4 — and the snapshot timer then fails silently every night. An orphan token has no parent to be revoked with. It still needs renewing within its 768h period, which vault-snapshot.sh does on every run.

This token is not in Git or GitHub Actions — Ansible never touches /etc/vault-backup/token. Rotating it is a manual step on the host.

Restore: docker compose exec vault vault operator raft snapshot restore <file> into a node sealed by the same KMS key.

4. Root token hygiene

After creating a real auth method + policies, revoke the root token. Confirm the backup token from step 3 is an orphan first, or this revokes it too:

sudo docker compose exec -T -e VAULT_TOKEN="$(sudo cat /etc/vault-backup/token)" \
  vault vault token lookup -format=json | jq '.data.orphan, .data.ttl'

cd /opt/vault
sudo docker compose exec vault vault token revoke -self

5. Upgrades

Renovate opens PRs bumping the pinned image tags in app/docker-compose.yml. Merging to main re-runs the pipeline; Ansible's ExecReload recreates only the changed container, which reloads raft and auto-unseals.