Secure Secret Management with SOPS, Age, and Bitwarden (Day 26)

When working with infrastructure as code and Kubernetes, you inevitably face the challenge of managing secrets securely.

API tokens, and other sensitive information shouldn't be stored in plain text in your Git repositories, but they still need to be accessible for deployments.

Enter SOPS and Age

SOPS (Secrets OPerationS) is a powerful tool that supports multiple encryption providers including AWS KMS, GCP KMS, Azure Key Vault, age, and PGP.

But for those of us without cloud provider resources, age offers a lightweight, modern alternative for encryption.

Key Management with Bitwarden

The first question with age is: where do you store your keys securely, especially when you might need to access them across multiple machines? or what happens you reset or loose your machine and loose the keys.

I went looking for options and found Bitwarden Secrets Manager, it offers an elegant solution to securely store cryptographic keys and access them, if you're also already using Bitwarden for password management then why not try it.

Setting Up the Infrastructure

Install the Required Tools

First, install age and SOPS:

# Install age and SOPS (commands will vary by OS)
# on mac you can use brew

Generate Your Age Key

age-keygen -o key.txt

The generated file contains two important pieces:

• A public key (starts with age1...)
• A secret key (starts with AGE-SECRET-KEY-...)

Store Keys in Bitwarden Secrets Manager

Follow the Bitwarden Secrets Manager guide to set up your account and store both keys.

Use the Bitwarden CLI

Install the Bitwarden Secrets CLI (bws) and set up your access token:

export BWS_ACCESS_TOKEN=<your token>

Load keys as env variables

Add this function to your shell profile (e.g., .zshrc) to easily load keys when needed:

load_age_secrets() {
    export SOPS_AGE_KEY=$(bws secret get <secret id> | jq .value | xargs)
    export AGE_PUBLIC_KEY=$(bws secret get <secret id> | jq ".value" | xargs)
    echo "export SOPS_AGE_KEY='$SOPS_AGE_KEY'" > /tmp/.secrets_exports
    echo "export AGE_PUBLIC_KEY='$AGE_PUBLIC_KEY'" >> /tmp/.secrets_exports
    echo "Secrets loaded"
}

# this loads the keys as env variables if the values exist
[[ -f /tmp/.secrets_exports ]] && source /tmp/.secrets_exports 

the <secret id> is the secret uuid that can be found by running bws secret list.

Terragrunt and Proxmox

An example of using encrypted secrets with Terragrunt for infra provisioning (in this case, for Proxmox):

1. Create a YAML file with your secrets

# auth.yaml
proxmox_token: "your_super_secret_token_that_no_one_should_know"
proxmox_user_id: "your_user_id_which_we_also_encrypted_because_why_not"

Using YAML because we will use Terragrunt's yamldecode function for parsing decrypted secret later.

2. Encrypt it

sops --encrypt --age $AGE_PUBLIC_KEY --in-place auth.yaml

--in-place overwrites the file with a new encrypted file.

3. Reference it in your Terragrunt file

secret_vars = yamldecode(sops_decrypt_file(find_in_parent_folders("sample.yaml")))

# ...

pm_api_token_secret = local.secret_vars.proxmox_token
pm_api_token_id = local.secret_vars.proxmox_user_id

4. Use the variables in your provider block

generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "proxmox" {
  pm_api_url          = "${local.pm_api_url}"
  pm_api_token_id     = "${local.pm_api_token_id}"
  pm_api_token_secret = "${local.pm_api_token_secret}"
  pm_tls_insecure     = false
  pm_parallel         = 10
}
EOF
}

Terragrunt commands should work as usual i.e e.g when running terragrunt apply the secret gets decrypted and used to authenticate with the proxmox API.

Kubernetes Secrets

You can also encrypt Kubernetes secrets

1. Create a secret file

# secret.yaml
apiVersion: v1
data:
  key1: c3VwZXJzZWNyZXQ=
  key2: dG9wc2VjcmV0
kind: Secret
metadata:
  name: my-secret

2. Encrypt it, specifying only certain fields to encrypt

sops --encrypt --age $AGE_PUBLIC_KEY --encrypted-regex '^(data|stringData)$' secret.yaml

To apply it you can pipe the decrypted output to kubectl e.g

sops --decrypt --encrypted-regex '^(data|stringData)$' sample.yaml | k apply -f -

Benefits of This Approach

• Security: Secrets never appear in plain text in your repositories
• Convenience: Access to secrets across multiple machines or if something happens to the data locally
• Auditability: Changes to encrypted files are tracked in git

So this setup gives you a nice foundation for secret management.