OpenBao Production Architecture

Raft Consensus Protocol

Distributed consensus made understandable — the backbone of OpenBao's integrated storage

What is Raft?

Raft is a distributed consensus protocol designed as a more understandable alternative to Paxos. It was introduced in a 2014 paper by Diego Ongaro and John Ousterhout with the explicit goal of being easier to implement and reason about. Raft guarantees that a cluster of nodes agrees on a sequence of state changes, even when some nodes fail or network partitions occur.

In OpenBao, Raft serves as the recommended integrated storage backend, requiring no external dependencies. Every secret, policy, auth configuration, and audit entry is stored as a replicated log entry in the Raft cluster.

How Raft works

Raft divides consensus into three subproblems: leader election, log replication, and safety. At any point, each node is in one of three states:

• Leader — handles all client writes, replicates log entries to followers, sends periodic heartbeats
• Follower — passive; responds to RPCs from the leader and candidates
• Candidate — transitional state during leader election; a follower becomes a candidate when it hasn't received a heartbeat within the election timeout

Time is divided into terms (monotonically increasing integers). Each term begins with an election. If a candidate wins, it serves as leader for the rest of the term. If the election splits (no majority), a new term begins.

Log replication is the core mechanism: the leader appends client requests to its log, then replicates them to followers. Once a majority (quorum) of nodes have written the entry, it is considered committed and can be applied to the state machine. This is why write latency depends on how quickly a majority of nodes acknowledge the entry.

Raft in OpenBao

When you configure storage "raft" in bao.hcl, OpenBao uses an embedded Raft implementation (based on HashiCorp's raft library) for all persistent state. Key operational details:

• Cluster sizing: always use an odd number of nodes — 3 (tolerates 1 failure) or 5 (tolerates 2 failures). Even numbers provide no additional fault tolerance but increase the risk of split-brain scenarios.
• Quorum: majority of nodes must be healthy. For 3 nodes, quorum = 2. For 5 nodes, quorum = 3. Losing quorum means the cluster is read-only (no writes, no leader election).
• Data path: stored in the directory specified by path in the storage stanza (e.g., /opt/bao/data). All data is encrypted at rest by the master key.
• Cluster port: Raft communication happens on port 8201 (the cluster_addr), separate from the API port 8200.

Peer management

# List all Raft peers and their state
bao operator raft list-peers

# Join a new node to an existing cluster
bao operator raft join https://bao-1.example.com:8200

# Remove a dead node from the cluster
bao operator raft remove-peer bao-3

# Take a snapshot for backup
bao operator raft snapshot save /backup/bao-$(date +%Y%m%d).snap

# Restore from snapshot (replaces ALL data)
bao operator raft snapshot restore -force backup.snap

Performance considerations

• Write latency is bounded by the slowest node in the quorum — if one of three nodes has slow disks, every write waits for it
• Use SSDs. Raft is write-intensive; every API operation that changes state generates a log entry that must be fsync'd to disk
• Network latency matters. Cross-AZ is fine (< 2ms typical). Cross-region is not — Raft performance degrades badly above 10ms RTT
• Autopilot (built-in) automatically manages dead server cleanup, stable server promotion, and server health tracking

Shamir's Secret Sharing

Cryptographic key splitting for separation of duties — the foundation of OpenBao's seal model

The algorithm

Shamir's Secret Sharing is a cryptographic algorithm invented by Adi Shamir in 1979. It splits a secret (in this case, the master key) into N shares, such that any K shares (the threshold) can reconstruct the original secret, but K-1 or fewer shares reveal absolutely nothing about the secret. This is a mathematically provable property, not just obfuscation.

The algorithm works using polynomial interpolation over a finite field. The secret is the constant term of a random polynomial of degree K-1. Each share is a point on that polynomial. With K points, you can reconstruct the polynomial (and thus the secret) via Lagrange interpolation. With fewer than K points, there are infinitely many valid polynomials — the secret could be anything.

How OpenBao uses Shamir

During bao operator init, OpenBao:

1. Generates a random master key
2. Uses the master key to encrypt the encryption key (which encrypts all data at rest)
3. Splits the master key into N shares using Shamir's algorithm
4. Outputs the shares — each share goes to a different key holder
5. Discards the master key from memory

# Initialize with 5 shares, threshold of 3
bao operator init -key-shares=5 -key-threshold=3

# Output:
# Unseal Key 1: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Unseal Key 2: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Unseal Key 3: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Unseal Key 4: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Unseal Key 5: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Initial Root Token: hvs.xxxxxxxxxxxxxxxxxxxx

# Unseal process (run 3 times with different keys)
bao operator unseal   # provide key share 1
bao operator unseal   # provide key share 2
bao operator unseal   # provide key share 3
# Sealed: false — master key reconstructed in memory

Key shares vs. recovery keys

When using auto-unseal (cloud KMS), the master key is encrypted by the KMS rather than split via Shamir. In this mode:

• Key shares are not generated — the KMS handles sealing/unsealing automatically
• Recovery keys are generated instead (also using Shamir's algorithm) — these are used for operations like bao operator generate-root
• Recovery keys cannot unseal the instance — only the KMS key can do that

Migration between seal types

# Migrate from Shamir to auto-unseal:
# 1. Add the seal stanza to bao.hcl
# 2. Restart Bao — it will start sealed
# 3. Provide existing Shamir keys with -migrate flag
bao operator unseal -migrate

# Migrate from auto-unseal back to Shamir:
# 1. Remove the seal stanza, add disabled = "true"
# 2. Restart and provide recovery keys with -migrate
bao operator unseal -migrate

Security considerations

• Distribute shares to different people in different physical locations — the entire point is separation of duties
• Never store multiple shares together — that defeats the threshold property
• Consider using PGP encryption during init (-pgp-keys) so each share is encrypted to its holder's PGP key
• If you lose more than N-K shares, the master key is permanently irrecoverable — all data is lost
• For most production deployments, auto-unseal is preferred over Shamir — it eliminates the operational burden of coordinating key holders for every restart

AppRole Authentication

Machine-to-machine auth method — solving the "secret zero" problem for applications

What is AppRole?

AppRole is an authentication method designed for machines and applications (no human interaction). It uses a two-part credential system: a RoleID (analogous to a username) and a SecretID (analogous to a password). The separation of these two components is the key security property — they should be delivered through different channels.

How it works

1. An administrator creates a role: bao write auth/approle/role/my-app policies="app-policy" token_ttl=1h
2. The RoleID is embedded in the application's configuration or image — it's relatively static and identifies which role to authenticate as
3. A SecretID is generated on-demand and delivered to the application through a trusted channel (CI/CD pipeline, orchestrator, configuration management)
4. The application combines RoleID + SecretID to authenticate: bao write auth/approle/login role_id=xxx secret_id=yyy
5. Bao returns a token with the policies attached to the role

Configuration

# Enable AppRole auth method
bao auth enable approle

# Create a role
bao write auth/approle/role/my-app \
  token_policies="app-readonly" \
  token_ttl=1h \
  token_max_ttl=4h \
  secret_id_ttl=10m \
  secret_id_num_uses=1

# Read the RoleID (embed this in app config)
bao read auth/approle/role/my-app/role-id

# Generate a SecretID (deliver via trusted channel)
bao write -f auth/approle/role/my-app/secret-id

# Application authenticates
bao write auth/approle/login \
  role_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" \
  secret_id="yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"

Response wrapping for SecretID delivery

The recommended pattern uses response wrapping to deliver the SecretID securely:

# Generate a wrapped SecretID (returns a wrapping token)
bao write -wrap-ttl=120s -f auth/approle/role/my-app/secret-id

# The wrapping token is a single-use token that can only be
# unwrapped once — if someone intercepts it, the app's unwrap
# will fail (alerting you to compromise)
bao unwrap hvs.wrapping-token-here

This ensures that even if the delivery channel is compromised, the attacker gets a single-use wrapping token. If the attacker unwraps it first, the legitimate application's unwrap fails — providing a tamper-detection mechanism.

Common deployment pattern

• CI/CD pipeline: The CI system (Jenkins, GitLab CI, GitHub Actions) has a long-lived token with permission to generate SecretIDs. It generates a wrapped SecretID, passes the wrapping token to the deployment. The application unwraps it, combines with its RoleID, and authenticates.
• Kubernetes: For K8s workloads, prefer Kubernetes auth over AppRole — it uses the pod's service account token directly, eliminating the secret zero problem entirely.
• VMs: Configuration management (Ansible, Puppet) delivers the SecretID during provisioning. The application authenticates on startup and renews its token periodically.

Comparison with other machine auth methods

• vs. Kubernetes auth: K8s auth is simpler (no SecretID management) but only works for K8s workloads. AppRole works everywhere.
• vs. TLS certificate auth: Cert auth requires PKI infrastructure and certificate distribution. More secure (mutual TLS) but higher operational overhead.
• vs. Cloud IAM auth (AWS, Azure, GCP): Cloud auth uses the instance's IAM identity — zero secret management, but only works on that cloud provider.

Transit Secrets Engine

Encryption-as-a-service — manage cryptographic keys without exposing them to applications

What is Transit?

The Transit secrets engine provides encryption, decryption, signing, verification, and hashing as an API service. Applications send data to Bao, Bao performs the cryptographic operation using a key that never leaves its boundary, and returns the result. The application never sees or manages the encryption key.

This is fundamentally different from KV secrets (where Bao stores your secrets) — with Transit, Bao performs cryptographic operations on your behalf. Your application stores the ciphertext in its own database; Bao just handles the crypto.

Core operations

# Enable the Transit engine
bao secrets enable transit

# Create an encryption key
bao write -f transit/keys/my-app-key

# Encrypt data (plaintext must be base64-encoded)
bao write transit/encrypt/my-app-key \
  plaintext=$(echo -n "sensitive-data" | base64)
# => ciphertext: vault:v1:xxxxxxxxxxxxxxxxxxxxxxxxxxx

# Decrypt data
bao write transit/decrypt/my-app-key \
  ciphertext="vault:v1:xxxxxxxxxxxxxxxxxxxxxxxxxxx"
# => plaintext: c2Vuc2l0aXZlLWRhdGE= (base64)

# Sign data
bao write transit/sign/my-signing-key \
  input=$(echo -n "message" | base64)

# Verify a signature
bao write transit/verify/my-signing-key \
  input=$(echo -n "message" | base64) \
  signature="vault:v1:yyyyyyyyyyy"

Key types

• aes256-gcm96 (default) — symmetric encryption. Fast, suitable for bulk data encryption. 96-bit nonce for authenticated encryption. Also available: aes128-gcm96.
• chacha20-poly1305 — symmetric encryption. 256-bit key. Good alternative to AES on platforms without hardware AES acceleration.
• rsa-2048 / rsa-3072 / rsa-4096 — asymmetric. Encrypt, decrypt, sign, verify. Useful when external systems need to encrypt data that only Bao can decrypt.
• ecdsa-p256 / ecdsa-p384 / ecdsa-p521 — asymmetric, signing only. Smaller keys and signatures than RSA, widely supported.
• ed25519 — asymmetric, signing only. High performance, modern. Recommended for new signing use cases.
• hmac — HMAC key for message authentication. Supports generation and verification.

Key rotation and versioning

Transit supports key rotation without re-encrypting existing data:

# Rotate the key (creates a new version)
bao write -f transit/keys/my-app-key/rotate

# New encryptions use version 2, but version 1
# ciphertext can still be decrypted

# Enforce minimum encryption version (force new key usage)
bao write transit/keys/my-app-key \
  min_encryption_version=2

# Enforce minimum decryption version (retire old keys)
bao write transit/keys/my-app-key \
  min_decryption_version=2
# WARNING: ciphertext encrypted with v1 becomes undecryptable

# Re-encrypt old ciphertext to the latest key version
bao write transit/rewrap/my-app-key \
  ciphertext="vault:v1:xxxxxxxxxxx"
# => ciphertext: vault:v2:yyyyyyyyyyy

The rewrap endpoint is key: it decrypts with the old version and re-encrypts with the current version, all server-side. Run a batch rewrap job after rotation to migrate all ciphertext to the latest key version, then advance min_decryption_version to retire old keys.

Convergent encryption

By default, encrypting the same plaintext twice produces different ciphertext (random nonce). Convergent encryption produces the same ciphertext for the same plaintext, enabling equality searches on encrypted data:

# Create a key with convergent encryption
bao write transit/keys/searchable-key \
  type=aes256-gcm96 \
  convergent_encryption=true \
  derived=true

Use this when you need to search encrypted database fields (e.g., "find all records for this email address") without decrypting everything. The trade-off: convergent encryption leaks the fact that two ciphertexts correspond to the same plaintext.

Real-world patterns

• Database field encryption: Encrypt PII (SSN, email, payment info) before storing in the application database. The database never sees plaintext.
• API token encryption: Encrypt API keys and tokens at rest. Decrypt only when needed for outbound API calls.
• Envelope encryption: Use Transit to encrypt a local data encryption key (DEK). Encrypt bulk data locally with the DEK. Store the encrypted DEK alongside the data. This reduces round-trips to Bao for large datasets.
• Tokenization: Combine Transit with format-preserving encryption to replace sensitive values with tokens that maintain the same format (e.g., credit card numbers).

PKI Secrets Engine

Built-in certificate authority — issue, renew, and revoke X.509 certificates on demand

What is the PKI engine?

The PKI secrets engine turns OpenBao into a full X.509 certificate authority. It can generate root and intermediate CAs, issue leaf certificates with configurable TTLs, manage CRLs, and serve OCSP responses. This eliminates the need for manual CSR workflows, expensive commercial CAs for internal services, or hand-managed OpenSSL scripts.

The key insight: certificates become short-lived and dynamic, just like database credentials. Instead of issuing a cert that lasts a year (and forgetting to rotate it), you issue certs that last 24-72 hours and let automation handle renewal.

Architecture: Root vs. Intermediate

The recommended pattern is a two-tier CA hierarchy:

• Root CA — generated offline or in a separate, tightly locked-down Bao mount. Long-lived (10-20 years). Signs only intermediate CA certificates. Ideally kept offline after signing the intermediate.
• Intermediate CA — the one applications actually use. Shorter-lived (1-5 years). Signed by the root. If compromised, revoke and re-issue a new intermediate without touching the root.

Setting up a PKI hierarchy

# 1. Enable the root CA mount
bao secrets enable -path=pki pki
bao secrets tune -max-lease-ttl=87600h pki   # 10 years

# 2. Generate a root certificate
bao write pki/root/generate/internal \
  common_name="My Org Root CA" \
  ttl=87600h
# => Save the certificate — this is your trust anchor

# 3. Configure CRL and issuing URLs
bao write pki/config/urls \
  issuing_certificates="https://bao.example.com:8200/v1/pki/ca" \
  crl_distribution_points="https://bao.example.com:8200/v1/pki/crl"

# 4. Enable the intermediate CA mount
bao secrets enable -path=pki_int pki
bao secrets tune -max-lease-ttl=43800h pki_int   # 5 years

# 5. Generate an intermediate CSR
bao write pki_int/intermediate/generate/internal \
  common_name="My Org Intermediate CA" \
  | jq -r '.data.csr' > pki_int.csr

# 6. Sign the intermediate with the root
bao write pki/root/sign-intermediate \
  csr=@pki_int.csr \
  format=pem_bundle \
  ttl=43800h \
  | jq -r '.data.certificate' > signed_int.pem

# 7. Import the signed intermediate
bao write pki_int/intermediate/set-signed \
  certificate=@signed_int.pem

Issuing certificates

# Create a role (defines what certs can be issued)
bao write pki_int/roles/web-server \
  allowed_domains="example.com" \
  allow_subdomains=true \
  max_ttl=72h

# Issue a certificate
bao write pki_int/issue/web-server \
  common_name="app.example.com" \
  alt_names="app2.example.com" \
  ttl=24h
# => Returns: certificate, private_key, ca_chain, serial_number

# Revoke a certificate
bao write pki_int/revoke \
  serial_number="xx:xx:xx:xx:..."

Roles and constraints

PKI roles define the boundaries of what certificates can be issued. This is where you enforce policy:

• allowed_domains — which domains the role can issue for
• allow_subdomains — whether subdomains of allowed_domains are permitted
• allow_glob_domains — pattern-based domain matching
• max_ttl — maximum certificate lifetime
• key_type / key_bits — RSA (2048/4096) or ECDSA (256/384)
• require_cn — whether a common name is mandatory
• server_flag / client_flag — restrict to server-auth, client-auth, or both

Automated renewal patterns

• Bao Agent — runs as a sidecar, auto-renews certs via templating and writes them to disk. Services reload on file change (e.g., Nginx, HAProxy).
• cert-manager (K8s) — integrates with OpenBao's PKI engine as an issuer. Automatically provisions and rotates TLS secrets for Kubernetes workloads.
• Consul-template / envconsul — watches the PKI lease and re-renders config files on renewal.
• Short TTLs + automation — the core philosophy. A 24h cert that auto-renews is far safer than a 1-year cert that everyone forgets about.

Database Secrets Engine

Dynamic, short-lived database credentials — eliminate shared passwords and credential sprawl

What is the Database engine?

The Database secrets engine generates dynamic, short-lived database credentials on demand. Instead of applications sharing a static db_password from a config file, each application instance requests its own unique credentials from Bao. Those credentials are automatically revoked when the lease expires.

This is arguably the highest-value engine in OpenBao. It solves the "who has the database password?" problem completely — every credential is unique, traceable, and automatically expires.

Supported databases

• PostgreSQL — full support including static roles, rotation, and custom SQL statements
• MySQL / MariaDB — dynamic and static credential management
• MongoDB — dynamic credentials with custom roles
• Microsoft SQL Server — Windows and SQL auth support
• Oracle — dynamic credentials via custom creation statements
• Redis — ACL-based dynamic credentials
• Elasticsearch — dynamic user management
• Snowflake, Cassandra, InfluxDB, Couchbase — community and built-in plugins

Setting up dynamic PostgreSQL credentials

# 1. Enable the database engine
bao secrets enable database

# 2. Configure the PostgreSQL connection
bao write database/config/my-postgres \
  plugin_name=postgresql-database-plugin \
  allowed_roles="readonly,readwrite" \
  connection_url="postgresql://{{username}}:{{password}}@pg.example.com:5432/mydb?sslmode=verify-full" \
  username="bao_admin" \
  password="initial-password"

# 3. Rotate the root credentials (Bao takes ownership)
bao write -f database/rotate-root/my-postgres
# WARNING: after this, only Bao knows the password

# 4. Create a role (defines what credentials look like)
bao write database/roles/readonly \
  db_name=my-postgres \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
    GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
  revocation_statements="DROP ROLE IF EXISTS \"{{name}}\";" \
  default_ttl=1h \
  max_ttl=24h

# 5. Generate credentials
bao read database/creds/readonly
# => username: v-approle-readonly-xxxxxxxxxxxxx
# => password: A1B2-C3D4-E5F6-randomized
# => lease_duration: 1h
# => lease_id: database/creds/readonly/xxxx

Lease management

Every dynamic credential comes with a lease. When the lease expires, Bao runs the revocation statement (e.g., DROP ROLE). Applications can renew leases to extend the credential lifetime up to the max_ttl:

# Renew a lease (extend by 1 hour)
bao lease renew database/creds/readonly/xxxx

# Revoke a specific credential immediately
bao lease revoke database/creds/readonly/xxxx

# Revoke ALL credentials for a role
bao lease revoke -prefix database/creds/readonly/

Static roles

For cases where you can't use dynamic usernames (legacy apps that hardcode a username), static roles rotate the password of an existing database user on a schedule:

# Create a static role that rotates every 24 hours
bao write database/static-roles/my-static-user \
  db_name=my-postgres \
  username="app_user" \
  rotation_period=86400

# Get the current password
bao read database/static-creds/my-static-user
# => username: app_user
# => password: (current rotated password)

Application integration patterns

• Bao Agent + templating — Agent fetches credentials and renders them into config files or environment variables. Handles renewal automatically.
• Direct API calls — Application requests credentials at startup, renews them periodically, and handles reconnection on rotation.
• Kubernetes sidecar — Init container or sidecar fetches creds via K8s auth, writes to a shared volume. The app reads creds from a file.
• Connection pooling — Use PgBouncer or ProxySQL as an intermediary. Bao rotates creds in the pooler config; applications connect through the pooler with a stable address.

Operational considerations

• Lease explosion: If applications request new credentials on every HTTP request instead of reusing them, you'll get thousands of orphaned DB roles. Set max_ttl aggressively and monitor vault.expire.num_leases.
• Root rotation: After rotate-root, only Bao knows the admin password. If Bao is down, you can't manage the database directly. Keep a break-glass superuser account separate from the one Bao manages.
• Schema changes: Dynamic users need permissions granted via the creation statement. If you add new tables, update the role's creation_statements or use GRANT ... ON ALL TABLES with default privileges.

SSH Secrets Engine

Signed certificates and dynamic keys — eliminate authorized_keys management and static SSH credentials

What is the SSH engine?

The SSH secrets engine provides two mechanisms for securing SSH access: signed SSH certificates (recommended) and dynamic SSH keys (OTP-based). Both eliminate the need to manage authorized_keys files, distribute SSH keys manually, or maintain static credentials across servers.

The signed certificate approach is particularly powerful: the SSH server trusts a CA public key, and Bao signs user certificates on demand. No per-user configuration on the server side. A new engineer gets access by requesting a signed cert from Bao — no one touches the server.

Signed SSH Certificates (recommended)

This is the preferred approach. OpenBao acts as an SSH Certificate Authority:

# 1. Enable the SSH engine
bao secrets enable -path=ssh-client-signer ssh

# 2. Generate or import a CA signing key
bao write ssh-client-signer/config/ca \
  generate_signing_key=true
# => Returns the CA public key

# 3. Get the CA public key (configure this on SSH servers)
bao read -field=public_key ssh-client-signer/config/ca \
  > /etc/ssh/trusted-user-ca-keys.pem

# 4. On each SSH server, add to /etc/ssh/sshd_config:
#    TrustedUserCAKeys /etc/ssh/trusted-user-ca-keys.pem
#    Then: systemctl restart sshd

# 5. Create a role for signing user keys
bao write ssh-client-signer/roles/admin \
  key_type=ca \
  default_user=ubuntu \
  allowed_users="ubuntu,admin,deploy" \
  allowed_extensions="permit-pty,permit-port-forwarding" \
  ttl=30m \
  max_ttl=4h

# 6. Sign a user's public key
bao write ssh-client-signer/sign/admin \
  public_key=@$HOME/.ssh/id_ed25519.pub \
  valid_principals=ubuntu \
  ttl=2h
# => Returns: signed_key (the certificate)

# 7. Save and use the certificate
# Save the signed key to ~/.ssh/id_ed25519-cert.pub
ssh -i ~/.ssh/id_ed25519 user@server
# SSH automatically uses the matching *-cert.pub file

How SSH certificates work

SSH certificates are different from X.509/TLS certificates. They are a built-in feature of OpenSSH:

• The server trusts a CA public key (one line in sshd_config)
• Users present a certificate signed by that CA along with their private key
• The server validates the signature, checks the principals (allowed usernames), and checks the expiry time
• No authorized_keys entry needed for the user — the CA signature is sufficient
• Certificates have a TTL — expired certs are automatically rejected

This means adding or removing user access is entirely managed through Bao policies, not through server configuration.

One-Time Password (OTP) mode

An alternative for environments where SSH certificate support is limited:

# Enable the SSH engine for OTP
bao secrets enable -path=ssh ssh

# Configure a role
bao write ssh/roles/otp_role \
  key_type=otp \
  default_user=ubuntu \
  cidr_list="10.0.0.0/8"

# Generate an OTP for a specific host
bao write ssh/creds/otp_role \
  ip=10.0.1.50
# => key: xxxxxxxx-xxxx-xxxx  (one-time password)
# => username: ubuntu

# SSH using the OTP
ssh ubuntu@10.0.1.50
# Enter the OTP as the password — it works exactly once

OTP mode requires the bao-ssh-helper PAM module installed on target servers. The helper validates the OTP with Bao and invalidates it immediately after use.

Host key signing

Bao can also sign host keys, solving the "do you trust this host?" TOFU (Trust On First Use) problem:

# Enable a separate mount for host signing
bao secrets enable -path=ssh-host-signer ssh

# Generate a host CA key
bao write ssh-host-signer/config/ca generate_signing_key=true

# Sign a server's host key
bao write ssh-host-signer/sign/host-role \
  cert_type=host \
  public_key=@/etc/ssh/ssh_host_ed25519_key.pub

# On clients, add the host CA to known hosts:
# @cert-authority *.example.com 
# No more "The authenticity of host ... can't be established" prompts

Role configuration and security

• allowed_users — restrict which Linux users can be specified as principals. Use "*" with caution.
• allowed_extensions — control SSH features: permit-pty, permit-port-forwarding, permit-agent-forwarding. Omit extensions to restrict capabilities.
• ttl / max_ttl — keep certificate lifetimes short. 30 minutes to 4 hours is typical for interactive sessions. CI/CD pipelines might use 15-minute certs.
• allowed_critical_options — e.g., force-command to restrict what the user can execute via SSH.
• cidr_list (OTP mode) — restrict which IP ranges OTPs can be generated for.

Real-world deployment

• Interactive access: Engineers authenticate to Bao (OIDC/LDAP), request a signed cert with a 2h TTL, SSH into servers. Cert expires automatically — no cleanup needed.
• CI/CD pipelines: Pipeline authenticates via AppRole, gets a 15-minute cert, deploys, cert expires. No long-lived deploy keys sitting in CI secrets.
• Emergency access: Break-glass procedure: generate a signed cert via Bao with a specific "emergency" principal, triggering audit alerts. Full traceability.
• Offboarding: Remove the user's Bao policy. Their existing cert expires within hours. No servers to touch, no keys to remove.