vijay_admin/sentryagent-idp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(devops): update all documentation for DockerSpec compliance

Browse Source 
- Replace all docker-compose.yml/docker-compose.monitoring.yml references with
  compose.yaml/compose.monitoring.yaml (modern Compose Spec naming)
- Replace all `docker-compose` CLI commands with `docker compose` (plugin syntax)
- Update Dockerfile stage descriptions: node:18-alpine → node:20.11-bookworm-slim,
  built-in node user → explicit nodeapp:1001 non-root user
- Update image version references: postgres:14-alpine → postgres:14.12-alpine3.19,
  redis:7-alpine → redis:7.2-alpine3.19
- Externalize postgres credentials: hardcoded values → POSTGRES_USER/PASSWORD/DB env vars
- Externalize Grafana admin password: hardcoded 'agentidp' → GF_ADMIN_PASSWORD env var
- Add Docker Compose Variables section to environment-variables.md (POSTGRES_*, GF_ADMIN_PASSWORD)
- Update local-development.md Step 3: cp .env.example .env, document POSTGRES_* purpose
- Update quick-start.md: cp .env.example .env, use awk/sed for JWT key injection
- Update 07-dev-setup.md: remove 'no .env.example' claim, reference cp .env.example
- Update docker-compose.yml key file description in 04-codebase-structure.md
- Update monitoring overlay launch commands across all docs (compose.yaml + compose.monitoring.yaml)
- Update volume names to kebab-case: postgres_data → postgres-data, redis_data → redis-data
- Fix compliance encryption-runbook: docker-compose restart agentidp → docker compose restart app

All docs now consistent with compose.yaml in repo root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
SentryAgent.ai Developer
2026-04-08 08:27:37 +00:00
parent 6fada694bb
commit f9a6a8aafb
12 changed files with 160 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/devops/README.md
View File

@@ -19,7 +19,7 @@ SentryAgent.ai AgentIdP is a Node.js REST API backed by PostgreSQL and Redis. It
| [Architecture](architecture.md) | All engineers | Components, ports, data flow, Redis key patterns |
| [Environment Variables](environment-variables.md) | All engineers | Every env var — required, optional, format, examples |
| [Database](database.md) | Backend, DevOps | Schema (26 tables/migrations), how to apply and verify |
| [Local Development](local-development.md) | All engineers | docker-compose setup, startup, health checks |
| [Local Development](local-development.md) | All engineers | Docker Compose setup (`compose.yaml`), startup, health checks |
| [Security](security.md) | All engineers | JWT key generation and rotation, CORS, secret storage |
| [Operations](operations.md) | DevOps | Startup order, graceful shutdown, log interpretation, troubleshooting |
| [field-trial.md](field-trial.md) | DevOps engineers, QA | In-house Docker Compose field trial execution playbook |

62
docs/devops/environment-variables.md
View File

@@ -6,6 +6,62 @@ Variables are loaded from a `.env` file at startup via `dotenv`. In production,
---
## Docker Compose Variables
These variables are read by `compose.yaml` — not by the application itself. They are required when running the stack via `docker compose up`.
### `POSTGRES_USER`
PostgreSQL superuser name — used to configure the `postgres` container and construct `DATABASE_URL`.
| | |
|-|-|
| **Required for Compose** | Yes |
| **Default in `.env.example`** | `sentryagent` |
| **Example** | `POSTGRES_USER=sentryagent` |
---
### `POSTGRES_PASSWORD`
PostgreSQL superuser password.
| | |
|-|-|
| **Required for Compose** | Yes |
| **Default in `.env.example`** | `change-me-in-production` |
| **Example** | `POSTGRES_PASSWORD=strongpassword` |
> Never use the default value in production. Generate a strong random password.
---
### `POSTGRES_DB`
PostgreSQL database name to create on first startup.
| | |
|-|-|
| **Required for Compose** | Yes |
| **Default in `.env.example`** | `sentryagent_idp` |
| **Example** | `POSTGRES_DB=sentryagent_idp` |
---
### `GF_ADMIN_PASSWORD`
Grafana admin panel password — used by `compose.monitoring.yaml`.
| | |
|-|-|
| **Required for monitoring stack** | Yes |
| **Default in `.env.example`** | `change-me-in-production` |
| **Example** | `GF_ADMIN_PASSWORD=strongpassword` |
> Never use the default value in production.
---
## Required Variables
These variables must be set. The server will throw and exit immediately if any are missing.
@@ -438,6 +494,12 @@ NODE_ENV=development
PORT=3000
CORS_ORIGIN=http://localhost:3001
# ── Docker Compose (postgres container + monitoring) ─────────────────────────
POSTGRES_USER=sentryagent
POSTGRES_PASSWORD=change-me-in-production
POSTGRES_DB=sentryagent_idp
GF_ADMIN_PASSWORD=change-me-in-production
# ── Database ─────────────────────────────────────────────────────────────────
DATABASE_URL=postgresql://sentryagent:sentryagent@localhost:5432/sentryagent_idp
DB_POOL_MAX=20

17
docs/devops/field-trial.md
View File

@@ -152,7 +152,10 @@ grep -E "^(DATABASE_URL|REDIS_URL|JWT_PRIVATE_KEY|JWT_PUBLIC_KEY|BILLING_ENABLED
Expected output (values abbreviated):
```
DATABASE_URL=postgresql://agentidp:password@localhost:5432/agentidp
POSTGRES_USER=sentryagent
POSTGRES_PASSWORD=sentryagent
POSTGRES_DB=sentryagent_idp
DATABASE_URL=postgresql://sentryagent:sentryagent@localhost:5432/sentryagent_idp
REDIS_URL=redis://localhost:6379
JWT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n...
JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\n...
@@ -185,10 +188,10 @@ docker compose ps
Expected output — all three services must show `healthy`:
```
NAME IMAGE STATUS
sentryagent-idp-app-1 sentryagent-idp-app running (healthy)
sentryagent-idp-postgres-1 postgres:14-alpine running (healthy)
sentryagent-idp-redis-1 redis:7-alpine running (healthy)
NAME IMAGE STATUS
sentryagent-idp-app-1 sentryagent-idp-app running (healthy)
sentryagent-idp-postgres-1 postgres:14.12-alpine3.19 running (healthy)
sentryagent-idp-redis-1 redis:7.2-alpine3.19 running (healthy)
```
If any service shows `starting` or `unhealthy`, wait 15 seconds and run `docker compose ps`
@@ -787,7 +790,7 @@ Common causes:
| Service | Cause | Fix |
|---------|-------|-----|
| `postgres` | Wrong database credentials | Verify `DATABASE_URL` in `.env` matches `docker-compose.yml` credentials |
| `postgres` | Wrong database credentials | Verify `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` in `.env` match values in `compose.yaml` |
| `redis` | Port conflict | Check `lsof -ti:6379` and kill occupying process |
| `app` | Missing env var | Check `docker compose logs app` for `Failed to start server` message |
@@ -825,7 +828,7 @@ Cause: A previous partial migration run left the database in an inconsistent sta
Fix: Check which migrations have been applied:
```bash
docker compose exec postgres psql -U agentidp -d agentidp \
docker compose exec postgres psql -U sentryagent -d sentryagent_idp \
-c "SELECT name FROM schema_migrations ORDER BY name;"
```

49
docs/devops/local-development.md
View File

@@ -17,7 +17,7 @@ Verify versions:
```bash
docker --version
docker-compose --version
docker compose version
node --version
npm --version
```
@@ -57,18 +57,29 @@ Keep these files in the project root. They are used only locally and should not
## Step 3 — Configure environment
Create a `.env` file in the project root:
Copy the template and fill in your values:
```bash
cat > .env << 'ENVEOF'
cp .env.example .env
```
The template already includes all required variables. At minimum, verify these are set correctly for local development:
```
POSTGRES_USER=sentryagent
POSTGRES_PASSWORD=sentryagent
POSTGRES_DB=sentryagent_idp
DATABASE_URL=postgresql://sentryagent:sentryagent@localhost:5432/sentryagent_idp
REDIS_URL=redis://localhost:6379
PORT=3000
NODE_ENV=development
CORS_ORIGIN=*
ENVEOF
```
> **Note:** `POSTGRES_USER`, `POSTGRES_PASSWORD`, and `POSTGRES_DB` are used by `compose.yaml`
> to configure the PostgreSQL container and construct `DATABASE_URL`. They are not read by
> the application directly — only `DATABASE_URL` is.
Append the JWT keys to `.env`:
```bash
@@ -86,10 +97,10 @@ grep -E "^(DATABASE_URL|REDIS_URL|JWT_PRIVATE_KEY|JWT_PUBLIC_KEY)" .env
## Step 4 — Start infrastructure services
The `docker-compose.yml` defines three services: `postgres`, `redis`, and `app`. For local development, start only the infrastructure services — the application runs directly via Node.js.
The `compose.yaml` defines three services: `postgres`, `redis`, and `app`. For local development, start only the infrastructure services — the application runs directly via Node.js.
```bash
docker-compose up -d postgres redis
docker compose up -d postgres redis
```
Expected output:
@@ -100,7 +111,7 @@ Expected output:
✔ Container sentryagent-idp-redis-1 Healthy
```
Both services must show `Healthy` before proceeding. If they show `Starting`, wait a few seconds and run `docker-compose ps` to recheck.
Both services must show `Healthy` before proceeding. If they show `Starting`, wait a few seconds and run `docker compose ps` to recheck.
### Service ports
@@ -112,18 +123,18 @@ Both services must show `Healthy` before proceeding. If they show `Starting`, wa
Verify manually:
```bash
docker-compose exec postgres pg_isready -U sentryagent -d sentryagent_idp
docker-compose exec redis redis-cli ping
docker compose exec postgres pg_isready -U sentryagent -d sentryagent_idp
docker compose exec redis redis-cli ping
```
### Docker volumes
Data is persisted in named Docker volumes:
Data is persisted in named Docker volumes (kebab-case per Compose Spec standard):
| Volume | Service | Contents |
|--------|---------|---------|
| `sentryagent-idp_postgres_data` | PostgreSQL | All database data |
| `sentryagent-idp_redis_data` | Redis | Redis persistence (if enabled) |
| `sentryagent-idp_postgres-data` | PostgreSQL | All database data |
| `sentryagent-idp_redis-data` | Redis | Redis persistence (if enabled) |
---
@@ -222,15 +233,13 @@ CORS_ORIGIN=http://localhost:3001
> deployments — see the [field trial guide](field-trial.md). For day-to-day development, start
> only the infrastructure services and run the application directly.
When the Dockerfile is available, the entire stack (infrastructure + application) can be started with:
The entire stack (infrastructure + application) can be started with:
```bash
docker-compose up -d
docker compose up --build -d
```
The `app` service depends on `postgres` and `redis` with health check conditions, so it will not start until both services are healthy.
Environment variables for the container are loaded from `.env` via the `env_file` directive in `docker-compose.yml`.
The `app` service depends on `postgres` and `redis` with health check conditions, so it will not start until both services are healthy. Environment variables are loaded from `.env` via the `env_file` directive in `compose.yaml` (`required: false` — the file is optional if env vars are injected directly).
---
@@ -239,19 +248,19 @@ Environment variables for the container are loaded from `.env` via the `env_file
Stop infrastructure only (preserves volumes):
```bash
docker-compose stop postgres redis
docker compose stop postgres redis
```
Stop and remove containers (preserves volumes):
```bash
docker-compose down
docker compose down
```
Stop and remove containers AND volumes (destroys all data):
```bash
docker-compose down -v
docker compose down -v
```
> Use `-v` only when you want a clean slate. This deletes all PostgreSQL data and Redis data permanently.

18
docs/devops/operations.md
View File

@@ -111,7 +111,7 @@ Three key patterns are used in Redis. Useful for debugging and manual inspection
```bash
# Connect to Redis CLI
docker-compose exec redis redis-cli
docker compose exec redis redis-cli
```
| Key pattern | Example | Purpose | TTL |
@@ -192,10 +192,10 @@ Error: connect ECONNREFUSED 127.0.0.1:5432
| Cause | Fix |
|-------|-----|
| PostgreSQL container not started | Run `docker-compose up -d postgres` |
| PostgreSQL container not yet healthy | Wait and run `docker-compose ps` — wait for `healthy` |
| PostgreSQL container not started | Run `docker compose up -d postgres` |
| PostgreSQL container not yet healthy | Wait and run `docker compose ps` — wait for `healthy` |
| Wrong `DATABASE_URL` host/port | Check `DATABASE_URL` matches the PostgreSQL port (5432) |
| PostgreSQL container exited | Run `docker-compose logs postgres` to see why it exited |
| PostgreSQL container exited | Run `docker compose logs postgres` to see why it exited |
---
@@ -210,8 +210,8 @@ Redis client error Error: connect ECONNREFUSED 127.0.0.1:6379
| Cause | Fix |
|-------|-----|
| Redis container not started | Run `docker-compose up -d redis` |
| Redis container not yet healthy | Run `docker-compose ps` — wait for `healthy` |
| Redis container not started | Run `docker compose up -d redis` |
| Redis container not yet healthy | Run `docker compose ps` — wait for `healthy` |
| Wrong `REDIS_URL` | Check `REDIS_URL` matches the Redis port (6379) |
---
@@ -257,7 +257,7 @@ If a migration is listed there but the table is inconsistent, manually inspect a
# Find the current window key
WINDOW=$(node -e "console.log(Math.floor(Date.now() / 60000))")
# Check count for a specific client
docker-compose exec redis redis-cli GET "rate:<client_id>:$WINDOW"
docker compose exec redis redis-cli GET "rate:<client_id>:$WINDOW"
```
**Fix:** Wait until `X-RateLimit-Reset` (Unix timestamp in the response header) before retrying. The window resets every 60 seconds.
@@ -296,10 +296,10 @@ AgentIdP exposes a Prometheus metrics endpoint at `GET /metrics` (unauthenticate
```bash
# Start the full stack with monitoring
docker compose -f docker-compose.yml -f docker-compose.monitoring.yml up -d
docker compose -f compose.yaml -f compose.monitoring.yaml up -d
# Prometheus: http://localhost:9090
# Grafana: http://localhost:3001 (admin / agentidp)
# Grafana: http://localhost:3001 (admin / <GF_ADMIN_PASSWORD from .env>)
```
The Grafana dashboard auto-provisions on first start. Navigate to **Dashboards → AgentIdP → SentryAgent.ai — AgentIdP**.