feat(phase-2): workstream 1 — HashiCorp Vault credential storage

Vault is optional — server falls back to bcrypt (Phase 1 behaviour) when VAULT_ADDR is not set. Full coexistence: existing bcrypt credentials continue to work until rotated. Changes: - src/vault/VaultClient.ts — wraps node-vault KV v2; writeSecret, readSecret, verifySecret (constant-time), deleteSecret - src/db/migrations/005_add_vault_path.sql — vault_path column on credentials - CredentialRepository — createWithVaultPath, updateVaultPath methods - CredentialService — routes generate/rotate through Vault when configured; bcrypt path unchanged - OAuth2Service — verifies via Vault when vaultPath set, bcrypt otherwise - src/app.ts — createVaultClientFromEnv() wired into service layer - ICredentialRow — vaultPath field added - docs/devops/environment-variables.md — VAULT_ADDR, VAULT_TOKEN, VAULT_MOUNT - docs/devops/vault-setup.md — dev quickstart, production config, migration guide - tests: 33/33 unit tests pass (VaultClient + CredentialService Vault path) - node-vault + @types/node-vault installed Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 15:02:33 +00:00
parent 7593bfe1c1
commit 90a4addb21
14 changed files with 1064 additions and 36 deletions
--- a/docs/devops/vault-setup.md
+++ b/docs/devops/vault-setup.md
@@ -0,0 +1,197 @@
+# HashiCorp Vault Setup
+
+Phase 2 of AgentIdP optionally stores credential secrets in [HashiCorp Vault](https://www.vaultproject.io/) KV v2 instead of bcrypt hashes in PostgreSQL. This guide covers:
+
+- Dev mode quickstart
+- Production Vault configuration
+- Migration from bcrypt to Vault
+
+Vault is **entirely optional**. If `VAULT_ADDR` is not set, AgentIdP operates in bcrypt mode (identical to Phase 1 behaviour).
+
+---
+
+## How Vault integration works
+
+When enabled:
+
+1. `POST /api/v1/agents/{agentId}/credentials` — the plain-text secret is written to Vault at `{mount}/data/agentidp/agents/{agentId}/credentials/{credentialId}`. Only the Vault path is stored in PostgreSQL (`credentials.vault_path`). No bcrypt hash is written.
+2. `POST /api/v1/token` — the submitted `client_secret` is compared against the value read from Vault (constant-time comparison). No bcrypt is involved.
+3. `POST /api/v1/agents/{agentId}/credentials/{credentialId}/rotate` — a new Vault version is written (KV v2 versioning). The path is unchanged; the old version is retained in Vault history.
+4. `DELETE /api/v1/agents/{agentId}/credentials/{credentialId}` — all versions of the secret are permanently deleted from Vault.
+
+**Coexistence**: Credentials created before Vault was enabled keep their bcrypt hash and continue to work. New credentials use Vault. Both paths coexist until all pre-Vault credentials are rotated.
+
+---
+
+## Dev mode quickstart
+
+The fastest way to get Vault running locally:
+
+```bash
+# Pull and start Vault in dev mode (in-memory, auto-unsealed)
+docker run --rm -d \
+  --name vault-dev \
+  -p 8200:8200 \
+  -e VAULT_DEV_ROOT_TOKEN_ID=dev-root-token \
+  hashicorp/vault:1.15 server -dev
+
+# Verify it is running
+curl http://127.0.0.1:8200/v1/sys/health | jq .
+```
+
+Add to your `.env`:
+
+```
+VAULT_ADDR=http://127.0.0.1:8200
+VAULT_TOKEN=dev-root-token
+VAULT_MOUNT=secret
+```
+
+The KV v2 secrets engine is automatically enabled at `secret/` in dev mode. No further configuration is needed.
+
+> **Warning**: Dev mode stores everything in memory. Data is lost when the container stops. Do not use dev mode in production.
+
+---
+
+## Production Vault configuration
+
+### 1. Enable KV v2 secrets engine
+
+```bash
+vault secrets enable -path=secret kv-v2
+```
+
+Or use a custom mount path:
+
+```bash
+vault secrets enable -path=agentidp kv-v2
+# Set VAULT_MOUNT=agentidp in your .env
+```
+
+### 2. Create a policy for AgentIdP
+
+```hcl
+# agentidp-policy.hcl
+path "secret/data/agentidp/*" {
+  capabilities = ["create", "read", "update", "delete"]
+}
+
+path "secret/metadata/agentidp/*" {
+  capabilities = ["delete"]
+}
+```
+
+Apply the policy:
+
+```bash
+vault policy write agentidp agentidp-policy.hcl
+```
+
+### 3. Create a service token
+
+```bash
+vault token create \
+  -policy=agentidp \
+  -ttl=8760h \
+  -renewable=true \
+  -display-name="agentidp-service"
+```
+
+Copy the `token` field from the output and set it as `VAULT_TOKEN` in your environment.
+
+### 4. Token renewal
+
+Service tokens expire unless renewed. Set up a scheduled renewal before the TTL expires:
+
+```bash
+# Renew with a new 720-hour (30-day) lease
+vault token renew -increment=720h <token>
+```
+
+In Kubernetes, use Vault Agent Injector or the Vault Secrets Operator to handle renewal automatically.
+
+---
+
+## Running migration 005
+
+After configuring Vault, run the migration to add the `vault_path` column:
+
+```bash
+npm run db:migrate
+```
+
+Verify the migration:
+
+```sql
+SELECT column_name, data_type, is_nullable
+FROM information_schema.columns
+WHERE table_name = 'credentials'
+ORDER BY ordinal_position;
+```
+
+You should see a `vault_path` column with `data_type = text` and `is_nullable = YES`.
+
+---
+
+## Migrating existing credentials to Vault
+
+Existing credentials (with `vault_path IS NULL`) continue to work via bcrypt until they are rotated. To migrate a credential:
+
+```bash
+# Rotate the credential — this writes the new secret to Vault
+curl -s -X POST http://localhost:3000/api/v1/agents/$AGENT_ID/credentials/$CRED_ID/rotate \
+  -H "Authorization: Bearer $TOKEN" | jq .
+```
+
+The response includes the new `clientSecret` (store it immediately). After rotation, `vault_path` is set and the bcrypt hash is cleared.
+
+To migrate all credentials for an agent in bulk, rotate them one by one using the API.
+
+---
+
+## Verifying Vault secrets
+
+After generating a credential with Vault enabled, verify the secret was written:
+
+```bash
+vault kv get secret/agentidp/agents/$AGENT_ID/credentials/$CRED_ID
+```
+
+Expected output:
+
+```
+====== Secret Path ======
+secret/data/agentidp/agents/<agentId>/credentials/<credentialId>
+
+======= Metadata =======
+Key                Value
+---                -----
+created_time       2026-03-28T...
+version            1
+
+====== Data ======
+Key             Value
+---             -----
+clientSecret    <the secret>
+```
+
+---
+
+## Troubleshooting
+
+### `VAULT_WRITE_ERROR` on credential generation
+
+- Verify Vault is running: `curl $VAULT_ADDR/v1/sys/health`
+- Verify the token has write access: `vault token capabilities $VAULT_TOKEN secret/data/agentidp/test`
+- Check Vault audit logs: `vault audit list`
+
+### `VAULT_READ_ERROR` on token issuance
+
+- Verify the `vault_path` stored in PostgreSQL matches the actual Vault path
+- Check the token has read access to `secret/data/agentidp/*`
+
+### Vault is down — what happens?
+
+If Vault is unreachable, credential generation and token issuance for Vault-backed credentials will fail with a `500` error. Credentials created before Vault was enabled (bcrypt mode) continue to work.
+
+For high availability, run Vault in HA mode with an integrated Raft storage backend. See [Vault HA documentation](https://developer.hashicorp.com/vault/docs/concepts/ha).