fd90b2acd1
Implements all 22 WS6 tasks completing Phase 3 Enterprise. Column-level encryption (AES-256-CBC, Vault-backed key) via EncryptionService applied to credentials.secret_hash, credentials.vault_path, webhook_subscriptions.vault_secret_path, and agent_did_keys.vault_key_path. Backward-compatible: isEncrypted() guard skips decryption for existing plaintext rows until next read-write cycle. Audit chain integrity (CC7.2): AuditRepository computes SHA-256 Merkle hash on every INSERT (hash = SHA-256(eventId+timestamp+action+outcome+agentId+orgId+prevHash)). AuditVerificationService walks the full chain verifying hash continuity. AuditChainVerificationJob runs hourly; sets agentidp_audit_chain_integrity Prometheus gauge to 1 (pass) or 0 (fail). TLS enforcement (CC6.7): TLSEnforcementMiddleware registered as first middleware in Express stack; 301 redirect on non-https X-Forwarded-Proto in production. SecretsRotationJob (CC9.2): hourly scan for credentials expiring within 7 days; increments agentidp_credentials_expiring_soon_total. ComplianceController + routes: GET /audit/verify (auth+audit:read scope, 30/min rate-limit); GET /compliance/controls (public, Cache-Control 60s). ComplianceStatusStore: module-level map updated by jobs, consumed by controller. Prometheus: 2 new metrics (agentidp_credentials_expiring_soon_total, agentidp_audit_chain_integrity); 6 alerting rules in alerts.yml. Compliance docs: soc2-controls-matrix.md, encryption-runbook.md, audit-log-runbook.md, incident-response.md, secrets-rotation.md. Tests: 557 unit tests passing (35 suites); 26 new tests (EncryptionService, AuditVerificationService); 19 compliance integration tests. TypeScript clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
160 lines
4.5 KiB
Markdown
160 lines
4.5 KiB
Markdown
# Encryption Key Rotation Runbook — SentryAgent.ai AgentIdP
|
|
|
|
**Control:** SOC 2 CC6.1 — Encryption at Rest
|
|
**Service:** `src/services/EncryptionService.ts`
|
|
**Vault path:** Configured via `ENCRYPTION_KEY_VAULT_PATH` env var (default: `secret/data/agentidp/encryption-key`)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
AgentIdP uses AES-256-CBC column-level encryption for sensitive PostgreSQL columns.
|
|
The encryption key is a 64-character hex string (32 bytes) stored in HashiCorp Vault.
|
|
The `EncryptionService` fetches the key once and caches it in process memory.
|
|
|
|
Encrypted format: `base64(IV):base64(ciphertext)` where IV is 16 random bytes per encryption call.
|
|
|
|
---
|
|
|
|
## Key Rotation Procedure
|
|
|
|
### Prerequisites
|
|
|
|
- Access to HashiCorp Vault with write permissions to the encryption key path
|
|
- Access to the production application environment (to trigger restart)
|
|
- At least one backup of the current key stored securely offline
|
|
|
|
### Step 1: Generate a New Key
|
|
|
|
Generate a cryptographically strong 32-byte (64-character hex) key:
|
|
|
|
```bash
|
|
openssl rand -hex 32
|
|
# Example output: a1b2c3d4e5f6... (64 hex chars)
|
|
```
|
|
|
|
Record the new key securely.
|
|
|
|
### Step 2: Backup the Current Key
|
|
|
|
Before overwriting, read and securely store the current key:
|
|
|
|
```bash
|
|
vault kv get -field=encryptionKey secret/agentidp/encryption-key > /secure/backup/encryption-key-$(date +%Y%m%d).txt
|
|
```
|
|
|
|
Store in a hardware security module (HSM) or offline key store.
|
|
|
|
### Step 3: Write the New Key to Vault
|
|
|
|
```bash
|
|
vault kv put secret/agentidp/encryption-key encryptionKey="<new-64-char-hex-key>"
|
|
```
|
|
|
|
Verify the write:
|
|
|
|
```bash
|
|
vault kv get secret/agentidp/encryption-key
|
|
```
|
|
|
|
Confirm the `encryptionKey` field contains exactly 64 hex characters.
|
|
|
|
### Step 4: Restart the Application
|
|
|
|
The `EncryptionService` caches the key in process memory. A restart forces a re-fetch from Vault:
|
|
|
|
```bash
|
|
# Kubernetes rolling restart
|
|
kubectl rollout restart deployment/agentidp
|
|
|
|
# Docker Compose
|
|
docker-compose restart agentidp
|
|
|
|
# PM2
|
|
pm2 restart agentidp
|
|
```
|
|
|
|
### Step 5: Verify Key Pick-Up
|
|
|
|
Check the application logs for:
|
|
|
|
```
|
|
[AgentIdP] EncryptionService enabled — sensitive columns encrypted at rest (SOC 2 CC6.1)
|
|
```
|
|
|
|
Call the compliance controls endpoint to confirm the control is passing:
|
|
|
|
```bash
|
|
curl -s https://api.sentryagent.ai/v1/compliance/controls | jq '.controls[] | select(.id == "CC6.1")'
|
|
```
|
|
|
|
Expected output:
|
|
```json
|
|
{ "id": "CC6.1", "name": "Encryption at Rest", "status": "passing", "lastChecked": "..." }
|
|
```
|
|
|
|
### Step 6: Re-encryption of Existing Rows
|
|
|
|
Existing rows encrypted with the old key will fail to decrypt after key rotation.
|
|
Re-encryption happens lazily: the next time each row is read and re-written (e.g. credential rotation,
|
|
webhook update), the application will decrypt with the old key and re-encrypt with the new one.
|
|
|
|
For immediate full re-encryption, use the re-encryption script:
|
|
|
|
```bash
|
|
# Run the re-encryption migration script (reads old key from backup, encrypts with new key)
|
|
# Note: This script requires both old and new keys to be available
|
|
ts-node scripts/reencrypt-columns.ts --old-key-file /secure/backup/encryption-key-<date>.txt
|
|
```
|
|
|
|
---
|
|
|
|
## Emergency Rollback
|
|
|
|
If the new key causes issues (e.g. test failures, decryption errors), roll back:
|
|
|
|
### Step 1: Restore Old Key to Vault
|
|
|
|
```bash
|
|
vault kv put secret/agentidp/encryption-key encryptionKey="<old-64-char-hex-key-from-backup>"
|
|
```
|
|
|
|
### Step 2: Restart the Application
|
|
|
|
```bash
|
|
kubectl rollout restart deployment/agentidp
|
|
```
|
|
|
|
### Step 3: Verify Recovery
|
|
|
|
```bash
|
|
curl -s https://api.sentryagent.ai/v1/compliance/controls | jq '.controls[] | select(.id == "CC6.1")'
|
|
```
|
|
|
|
### Step 4: Investigate Root Cause
|
|
|
|
Review application logs for `AES-256-CBC decryption failed` errors and audit the cause before
|
|
reattempting rotation.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
| Symptom | Likely Cause | Resolution |
|
|
|---|---|---|
|
|
| `Invalid encryption key ... expected a 64-character hex string` | Key in Vault is wrong length or encoding | Re-write correct key to Vault, restart |
|
|
| `AES-256-CBC decryption failed — possible key mismatch` | Key rotated but rows still encrypted with old key | Rollback to old key, then migrate properly |
|
|
| `CC6.1` status shows `unknown` | Vault unreachable, key fetch failed | Check Vault connectivity, `VAULT_ADDR`, `VAULT_TOKEN` |
|
|
|
|
---
|
|
|
|
## Audit Evidence
|
|
|
|
After rotation, record the following for SOC 2 evidence:
|
|
|
|
- Date of rotation
|
|
- Who performed the rotation (approver + executor)
|
|
- Vault audit log entry confirming the key write
|
|
- Application log confirming EncryptionService initialised with new key
|
|
- `GET /compliance/controls` response showing CC6.1 = passing
|