eced5f8699
Complete docs/engineering/ suite — 12 documents covering company overview, system architecture, tech stack ADRs, codebase structure, service deep dives, annotated code walkthroughs, dev setup, engineering workflow, testing strategy, deployment/ops, SDK guide, and README index. All content verified against source files. All 82 tasks in openspec/changes/engineering-docs/tasks.md marked complete. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
405 lines
14 KiB
Markdown
405 lines
14 KiB
Markdown
# 07 — Development Environment Setup
|
||
|
||
This guide takes you from a fresh machine to a running AgentIdP server with a
|
||
passing smoke test. Estimated time: 15–20 minutes.
|
||
|
||
---
|
||
|
||
## 8.1 Prerequisites
|
||
|
||
Install all of these before proceeding.
|
||
|
||
| Prerequisite | Minimum version | Install link |
|
||
|-------------|----------------|--------------|
|
||
| Node.js | 18.x LTS | https://nodejs.org/en/download — use the LTS version |
|
||
| npm | 9.x (ships with Node.js 18) | Included with Node.js |
|
||
| Docker Desktop | Latest stable | https://docs.docker.com/get-docker/ |
|
||
| Git | 2.x | https://git-scm.com/downloads |
|
||
|
||
**Verify your versions:**
|
||
```bash
|
||
node --version # Should print v18.x.x or higher
|
||
npm --version # Should print 9.x or higher
|
||
docker --version # Should print Docker version 24.x or higher
|
||
git --version # Should print git version 2.x
|
||
```
|
||
|
||
---
|
||
|
||
## 8.2 Clone and Install
|
||
|
||
```bash
|
||
# Clone the repository
|
||
git clone https://github.com/sentryagent-ai/sentryagent-idp.git
|
||
cd sentryagent-idp
|
||
|
||
# Install Node.js dependencies
|
||
npm install
|
||
```
|
||
|
||
This installs all production dependencies (Express, pg, Redis, etc.) and
|
||
development dependencies (TypeScript, Jest, ts-jest, eslint).
|
||
|
||
---
|
||
|
||
## 8.3 Environment Variables Setup
|
||
|
||
The server requires a `.env` file at the project root. There is no `.env.example`
|
||
file — create it from scratch using the template below.
|
||
|
||
```bash
|
||
touch .env
|
||
```
|
||
|
||
Add the following content to `.env`. Every variable is documented below.
|
||
|
||
```bash
|
||
# ─────────────────────────────────────────────────────────────
|
||
# PostgreSQL connection
|
||
# ─────────────────────────────────────────────────────────────
|
||
DATABASE_URL=postgresql://sentryagent:sentryagent@localhost:5432/sentryagent_idp
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# Redis connection
|
||
# ─────────────────────────────────────────────────────────────
|
||
REDIS_URL=redis://localhost:6379
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# HTTP server port
|
||
# ─────────────────────────────────────────────────────────────
|
||
PORT=3000
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# JWT RSA keys (generate these below)
|
||
# ─────────────────────────────────────────────────────────────
|
||
JWT_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----"
|
||
JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----"
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# CORS (optional — defaults to '*' which allows all origins)
|
||
# ─────────────────────────────────────────────────────────────
|
||
CORS_ORIGIN=*
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# Node environment
|
||
# ─────────────────────────────────────────────────────────────
|
||
NODE_ENV=development
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# OPA policy directory (optional — defaults to ./policies)
|
||
# ─────────────────────────────────────────────────────────────
|
||
# POLICY_DIR=/path/to/policies
|
||
|
||
# ─────────────────────────────────────────────────────────────
|
||
# HashiCorp Vault (optional — omit to use bcrypt mode)
|
||
# ─────────────────────────────────────────────────────────────
|
||
# VAULT_ADDR=http://127.0.0.1:8200
|
||
# VAULT_TOKEN=your-vault-token
|
||
# VAULT_MOUNT=secret
|
||
```
|
||
|
||
**Complete environment variable reference:**
|
||
|
||
| Variable | Required | Default | Description |
|
||
|----------|----------|---------|-------------|
|
||
| `DATABASE_URL` | Yes | — | PostgreSQL connection string. Format: `postgresql://user:password@host:port/dbname` |
|
||
| `REDIS_URL` | Yes | — | Redis connection URL. Format: `redis://host:port[/db]` |
|
||
| `PORT` | No | `3000` | TCP port the HTTP server listens on |
|
||
| `JWT_PRIVATE_KEY` | Yes | — | PEM-encoded RSA private key (2048-bit minimum) for signing tokens |
|
||
| `JWT_PUBLIC_KEY` | Yes | — | PEM-encoded RSA public key (matching the private key above) for verifying tokens |
|
||
| `CORS_ORIGIN` | No | `*` | CORS allowed origin. Use `*` for development, set to your dashboard domain in production |
|
||
| `NODE_ENV` | No | `development` | Set to `test` to suppress Morgan HTTP logging during tests |
|
||
| `POLICY_DIR` | No | `./policies` | Absolute path to the directory containing `authz.wasm` or `data/scopes.json` |
|
||
| `VAULT_ADDR` | No | — | HashiCorp Vault server address (e.g. `http://127.0.0.1:8200`). When omitted, bcrypt mode is used |
|
||
| `VAULT_TOKEN` | No | — | Vault authentication token. Required when `VAULT_ADDR` is set |
|
||
| `VAULT_MOUNT` | No | `secret` | Vault KV v2 mount path. Only used when Vault is configured |
|
||
|
||
**Generating JWT keys:**
|
||
```bash
|
||
# Generate RSA 2048-bit private key
|
||
openssl genrsa -out private.pem 2048
|
||
|
||
# Extract public key
|
||
openssl rsa -in private.pem -pubout -out public.pem
|
||
|
||
# Print private key as single-line for .env (replace newlines with \n)
|
||
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' private.pem
|
||
|
||
# Print public key as single-line for .env
|
||
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' public.pem
|
||
```
|
||
|
||
Paste the output (including the `-----BEGIN/END-----` lines) as the value for
|
||
`JWT_PRIVATE_KEY` and `JWT_PUBLIC_KEY` in your `.env` file, surrounded by double
|
||
quotes.
|
||
|
||
---
|
||
|
||
## 8.4 Docker Compose Startup and Health Checks
|
||
|
||
Docker Compose starts PostgreSQL 14 and Redis 7. The application reads the
|
||
`DATABASE_URL` and `REDIS_URL` from your `.env` file to connect to them.
|
||
|
||
```bash
|
||
# Start PostgreSQL and Redis in the background
|
||
docker compose up postgres redis -d
|
||
|
||
# Wait for health checks to pass (usually 10-15 seconds)
|
||
docker compose ps
|
||
```
|
||
|
||
Expected output when both services are healthy:
|
||
```
|
||
NAME STATUS PORTS
|
||
sentryagent-idp-postgres-1 Up (healthy) 0.0.0.0:5432->5432/tcp
|
||
sentryagent-idp-redis-1 Up (healthy) 0.0.0.0:6379->6379/tcp
|
||
```
|
||
|
||
**Manual health check:**
|
||
```bash
|
||
# Test PostgreSQL connection
|
||
docker exec sentryagent-idp-postgres-1 pg_isready -U sentryagent -d sentryagent_idp
|
||
# Expected: /var/run/postgresql:5432 - accepting connections
|
||
|
||
# Test Redis connection
|
||
docker exec sentryagent-idp-redis-1 redis-cli ping
|
||
# Expected: PONG
|
||
```
|
||
|
||
---
|
||
|
||
## 8.5 Database Migrations
|
||
|
||
Run the migration script to create all required tables:
|
||
|
||
```bash
|
||
npm run db:migrate
|
||
```
|
||
|
||
Expected output:
|
||
```
|
||
Running database migrations...
|
||
✓ Applied: 001_create_agents.sql
|
||
✓ Applied: 002_create_credentials.sql
|
||
✓ Applied: 003_create_audit_events.sql
|
||
✓ Applied: 004_create_tokens.sql
|
||
✓ Applied: 005_add_vault_path.sql
|
||
|
||
Migrations complete. 5 migration(s) applied.
|
||
```
|
||
|
||
Running `npm run db:migrate` a second time is safe — it skips already-applied migrations:
|
||
```
|
||
- Skipped (already applied): 001_create_agents.sql
|
||
...
|
||
Migrations complete. 0 migration(s) applied.
|
||
```
|
||
|
||
**Migration internals:**
|
||
The migration runner (`scripts/migrate.ts`) reads `.sql` files from `src/db/migrations/`
|
||
in alphabetical order, wraps each in a transaction, and records the filename in the
|
||
`schema_migrations` table. If a migration fails, the transaction rolls back and
|
||
the runner exits with code 1.
|
||
|
||
---
|
||
|
||
## 8.6 Start the Server
|
||
|
||
```bash
|
||
npm run dev
|
||
# Expected: SentryAgent.ai AgentIdP listening on port 3000
|
||
```
|
||
|
||
`npm run dev` uses `ts-node` to execute `src/server.ts` directly without compiling.
|
||
This is faster for development. For a production-style start, compile first:
|
||
|
||
```bash
|
||
npm run build
|
||
npm start
|
||
```
|
||
|
||
---
|
||
|
||
## 8.7 Smoke Test
|
||
|
||
Verify the server is working with these three curl commands.
|
||
|
||
**1. Health check:**
|
||
```bash
|
||
curl http://localhost:3000/health
|
||
```
|
||
Expected response (200 OK):
|
||
```json
|
||
{
|
||
"status": "healthy",
|
||
"checks": {
|
||
"database": "healthy",
|
||
"redis": "healthy"
|
||
}
|
||
}
|
||
```
|
||
|
||
**2. Register an agent:**
|
||
First, you need a token to authenticate. But to get a token, you need credentials.
|
||
And to get credentials, you need an agent. The chicken-and-egg is resolved by the
|
||
fact that agent registration requires an `agents:write` scoped token — which means
|
||
you need to bootstrap the first agent another way.
|
||
|
||
For local development, temporarily test without auth by using the `/api/v1` prefix
|
||
directly (the server accepts requests; OPA will enforce scope).
|
||
|
||
The easiest approach: generate a test token programmatically:
|
||
|
||
```bash
|
||
# Generate test keys
|
||
openssl genrsa -out /tmp/test_private.pem 2048 2>/dev/null
|
||
openssl rsa -in /tmp/test_private.pem -pubout -out /tmp/test_public.pem 2>/dev/null
|
||
|
||
# Set them in your environment temporarily
|
||
export JWT_PRIVATE_KEY="$(cat /tmp/test_private.pem)"
|
||
export JWT_PUBLIC_KEY="$(cat /tmp/test_public.pem)"
|
||
|
||
# Start server with these keys and use a tool or short Node.js script to mint a test token
|
||
```
|
||
|
||
**3. Token endpoint with seeded credentials:**
|
||
|
||
Once you have an agent with credentials (e.g. created via the API or seeded in
|
||
development), issue a token:
|
||
|
||
```bash
|
||
curl -X POST http://localhost:3000/api/v1/token \
|
||
-H "Content-Type: application/x-www-form-urlencoded" \
|
||
-d "grant_type=client_credentials" \
|
||
-d "client_id=YOUR_AGENT_ID" \
|
||
-d "client_secret=sk_live_YOUR_SECRET" \
|
||
-d "scope=agents:read agents:write"
|
||
```
|
||
|
||
Expected response (200 OK):
|
||
```json
|
||
{
|
||
"access_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...",
|
||
"token_type": "Bearer",
|
||
"expires_in": 3600,
|
||
"scope": "agents:read agents:write"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 8.8 Troubleshooting
|
||
|
||
### Error: `connection refused` on PostgreSQL or Redis
|
||
**Cause:** Docker services not running or not yet healthy.
|
||
**Fix:**
|
||
```bash
|
||
docker compose ps # Check status
|
||
docker compose up postgres redis -d # Start if not running
|
||
docker compose logs postgres # Check for startup errors
|
||
```
|
||
|
||
### Error: `DATABASE_URL environment variable is required`
|
||
**Cause:** `.env` file missing or not being loaded.
|
||
**Fix:** Ensure `.env` exists at the project root. `npm run dev` loads it via `dotenv.config()` in `src/server.ts`.
|
||
|
||
### Error: `JWT_PRIVATE_KEY and JWT_PUBLIC_KEY environment variables are required`
|
||
**Cause:** JWT keys not in `.env`, or newlines in the PEM keys are not properly escaped.
|
||
**Fix:** Ensure the keys are wrapped in double quotes and newlines are represented as `\n`. Use the `awk` command from section 8.3 to format them correctly.
|
||
|
||
### Error: `Cannot find module 'ts-node'`
|
||
**Cause:** `npm install` was not run, or ran against a different Node.js version.
|
||
**Fix:**
|
||
```bash
|
||
node --version # Confirm Node.js 18+
|
||
rm -rf node_modules package-lock.json
|
||
npm install
|
||
```
|
||
|
||
### Error: `Cannot connect to Redis` (during migration or server start)
|
||
**Cause:** Redis container not running or `REDIS_URL` is incorrect.
|
||
**Fix:**
|
||
```bash
|
||
docker exec sentryagent-idp-redis-1 redis-cli ping
|
||
# If container is not running:
|
||
docker compose up redis -d
|
||
```
|
||
|
||
### Port 3000 already in use
|
||
**Cause:** Another process is listening on port 3000.
|
||
**Fix:**
|
||
```bash
|
||
lsof -i :3000 # Find the process
|
||
kill <PID> # Kill it
|
||
# Or: set PORT=3001 in .env and restart
|
||
```
|
||
|
||
---
|
||
|
||
## 8.9 Running Tests Locally
|
||
|
||
```bash
|
||
# Run all tests (unit + integration)
|
||
npm test
|
||
|
||
# Run tests with coverage report
|
||
npm run test:unit -- --coverage
|
||
# Coverage report: coverage/lcov-report/index.html
|
||
|
||
# Run only unit tests
|
||
npm run test:unit
|
||
|
||
# Run only integration tests (requires running PostgreSQL and Redis)
|
||
npm run test:integration
|
||
|
||
# Run a single test file
|
||
npx jest tests/unit/services/AgentService.test.ts
|
||
|
||
# Run tests matching a pattern
|
||
npx jest --testNamePattern="registerAgent"
|
||
|
||
# Watch mode (re-runs on file changes)
|
||
npx jest --watch
|
||
```
|
||
|
||
**Integration test requirements:**
|
||
Integration tests connect to real PostgreSQL and Redis. Set these environment
|
||
variables before running integration tests:
|
||
```bash
|
||
TEST_DATABASE_URL=postgresql://sentryagent:sentryagent@localhost:5432/sentryagent_idp_test
|
||
TEST_REDIS_URL=redis://localhost:6379/1
|
||
```
|
||
|
||
The integration tests create their own tables (using `CREATE TABLE IF NOT EXISTS`)
|
||
and clean up after themselves with `DELETE FROM` statements in `afterAll`.
|
||
|
||
---
|
||
|
||
## 8.10 Web Dashboard Local Development
|
||
|
||
The web dashboard is a separate Vite project in the `dashboard/` directory.
|
||
|
||
```bash
|
||
# From the project root
|
||
cd dashboard
|
||
npm install
|
||
|
||
# Start the Vite development server
|
||
npm run dev
|
||
# Dashboard available at http://localhost:5173
|
||
```
|
||
|
||
The Vite dev server proxies all `/api/*` requests to `http://localhost:3000`,
|
||
so the API server must be running concurrently (in a separate terminal).
|
||
|
||
**Build for production:**
|
||
```bash
|
||
cd dashboard
|
||
npm run build
|
||
# Output: dashboard/dist/ (served by Express at /dashboard)
|
||
```
|
||
|
||
After building, the Express server serves the built dashboard at
|
||
`http://localhost:3000/dashboard`. You do not need to run the Vite dev server
|
||
for this — the static files are served directly by Express.
|