vijay_admin/sentryagent-idp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: commit all Phase 6 documentation updates and OpenSpec archives

Browse Source 
- devops docs: 8 files updated for Phase 6 state; field-trial.md added (946-line runbook)
- developer docs: api-reference (50+ endpoints), quick-start, 5 existing guides updated, 5 new guides added
- engineering docs: all 12 files updated (services, architecture, SDK guide, testing, overview)
- OpenSpec archives: phase-7-devops-field-trial, developer-docs-phase6-update, engineering-docs-phase6-update
- VALIDATOR.md + scripts/start-validator.sh: V&V Architect tooling added
- .gitignore: exclude session artifacts, build artifacts, and agent workspaces

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
SentryAgent.ai Developer
2026-04-07 02:24:24 +00:00
parent 0fb00256b4
commit 8cabc0191c
56 changed files with 12780 additions and 446 deletions
Download Patch File Download Diff File Expand all files Collapse all files

946
docs/devops/field-trial.md Normal file
View File

@@ -0,0 +1,946 @@
# SentryAgent.ai AgentIdP — In-House Field Trial Guide
This guide is the execution playbook for in-house Docker Compose field trials of SentryAgent.ai
AgentIdP. Follow each phase in order. All commands are exact — copy and paste them directly.
Estimated time to complete all phases: 45–60 minutes.
Prerequisites must be satisfied before Section 0.
## Prerequisites
**Docker 24+ and Docker Compose 2.20+**
```bash
docker --version
# Expected: Docker version 24.x.x or higher
docker compose version
# Expected: Docker Compose version v2.20.x or higher
```
**Node.js 18+ via nvm**
```bash
export NVM_DIR="$HOME/.nvm" && source "$NVM_DIR/nvm.sh"
node --version
# Expected: v18.x.x or higher
```
**openssl**
```bash
openssl version
# Expected: OpenSSL 1.1.x or higher (any version)
```
**Git repo cloned**
```bash
git clone https://git.sentryagent.ai/vijay_admin/sentryagent-idp.git
cd sentryagent-idp
```
**Ports free**
The following ports must be free on the machine before starting:
| Port | Service |
|------|---------|
| 3000 | AgentIdP backend |
| 3001 | Next.js portal |
| 5432 | PostgreSQL |
| 6379 | Redis |
Check all ports:
```bash
lsof -i :3000 -i :3001 -i :5432 -i :6379
# Expected: no output (all ports free)
```
If any port is in use, kill the occupying process:
```bash
lsof -ti:<port> | xargs kill
```
---
## Section 0 — Environment Setup
This section guides the engineer through creating a valid `.env` file for field trial use.
**Step 0.1 — Copy `.env.example`**
```bash
cp .env.example .env
```
**Step 0.2 — Generate RSA-2048 keypair**
Generate the JWT signing keys:
```bash
openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -pubout -out public.pem
```
Verify the keys are valid:
```bash
openssl rsa -in private.pem -check -noout
# Expected: RSA key ok
openssl rsa -in public.pem -pubin -noout -text 2>&1 | head -3
# Expected: Public-Key: (2048 bit)
```
**Step 0.3 — Write keys into `.env`**
Write the private key as a single-line PEM with `\n` separators:
```bash
PRIVATE_KEY_LINE=$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' private.pem)
sed -i "s|JWT_PRIVATE_KEY=.*|JWT_PRIVATE_KEY=\"${PRIVATE_KEY_LINE}\"|" .env
```
Write the public key:
```bash
PUBLIC_KEY_LINE=$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' public.pem)
sed -i "s|JWT_PUBLIC_KEY=.*|JWT_PUBLIC_KEY=\"${PUBLIC_KEY_LINE}\"|" .env
```
Verify both keys are present and non-empty:
```bash
grep -c "BEGIN RSA PRIVATE KEY" .env
# Expected: 1
grep -c "BEGIN PUBLIC KEY" .env
# Expected: 1
```
**Step 0.4 — Configure field trial values**
Set the following values in `.env`. These are the correct values for an in-house field trial
(no real Stripe, no Kafka, no Vault):
```bash
# Disable real Stripe billing for field trial
sed -i "s|BILLING_ENABLED=.*|BILLING_ENABLED=false|" .env
sed -i "s|STRIPE_SECRET_KEY=.*|STRIPE_SECRET_KEY=sk_test_placeholder|" .env
sed -i "s|STRIPE_WEBHOOK_SECRET=.*|STRIPE_WEBHOOK_SECRET=whsec_placeholder|" .env
sed -i "s|STRIPE_PRICE_ID=.*|STRIPE_PRICE_ID=price_placeholder|" .env
# Keep feature flags at defaults
sed -i "s|ANALYTICS_ENABLED=.*|ANALYTICS_ENABLED=true|" .env
sed -i "s|TIER_ENFORCEMENT=.*|TIER_ENFORCEMENT=true|" .env
sed -i "s|COMPLIANCE_ENABLED=.*|COMPLIANCE_ENABLED=true|" .env
# Allow portal CORS
sed -i "s|CORS_ORIGIN=.*|CORS_ORIGIN=http://localhost:3001|" .env
```
**Step 0.5 — Verify final `.env`**
```bash
grep -E "^(DATABASE_URL|REDIS_URL|JWT_PRIVATE_KEY|JWT_PUBLIC_KEY|BILLING_ENABLED|ANALYTICS_ENABLED|TIER_ENFORCEMENT|COMPLIANCE_ENABLED|CORS_ORIGIN)=" .env
```
Expected output (values abbreviated):
```
DATABASE_URL=postgresql://agentidp:password@localhost:5432/agentidp
REDIS_URL=redis://localhost:6379
JWT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n...
JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\n...
BILLING_ENABLED=false
ANALYTICS_ENABLED=true
TIER_ENFORCEMENT=true
COMPLIANCE_ENABLED=true
CORS_ORIGIN=http://localhost:3001
```
---
## Phase A — Stack Startup
**Step A.1 — Build and start the full stack**
```bash
docker compose up --build -d
```
This builds the `app` container image and starts all three services. The `app` service waits
for `postgres` and `redis` to pass their health checks before starting.
**Step A.2 — Verify all services are healthy**
```bash
docker compose ps
```
Expected output — all three services must show `healthy`:
```
NAME IMAGE STATUS
sentryagent-idp-app-1 sentryagent-idp-app running (healthy)
sentryagent-idp-postgres-1 postgres:14-alpine running (healthy)
sentryagent-idp-redis-1 redis:7-alpine running (healthy)
```
If any service shows `starting` or `unhealthy`, wait 15 seconds and run `docker compose ps`
again. If a service remains unhealthy after 60 seconds, see Troubleshooting.
**Step A.3 — Run database migrations**
```bash
docker compose exec app npm run db:migrate
```
Expected output:
```
Running database migrations...
✓ Applied: 001_create_agents.sql
✓ Applied: 002_create_credentials.sql
...
✓ Applied: 025_add_analytics_events.sql
✓ Applied: 026_add_tenant_tiers.sql
Migrations complete. 26 migration(s) applied.
```
All 26 migrations must apply without error before proceeding.
**Step A.4 — Verify application health**
```bash
curl -s http://localhost:3000/health | jq .
```
Expected response:
```json
{"status":"ok"}
```
**Step A.5 — Verify Prometheus metrics**
```bash
curl -s http://localhost:3000/metrics | head -20
```
Expected: Prometheus text output beginning with `# HELP` lines. Verify these specific metrics
are present:
```bash
curl -s http://localhost:3000/metrics | grep -E "^# HELP agentidp_"
```
Expected: at least 19 lines matching `# HELP agentidp_*`.
---
## Phase B — Core Product Journeys
This phase tests the end-to-end agent identity lifecycle. Run each step in order. Each step
depends on the output of the previous step.
> **Note on tokens:** The steps below use shell variables to pass values between commands. Run
> all commands in the same terminal session.
**Step B.1 — Create an organisation**
```bash
ORG_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/organizations \
-H "Content-Type: application/json" \
-d '{"name":"Field Trial Org","slug":"field-trial"}')
echo $ORG_RESPONSE | jq .
ORG_ID=$(echo $ORG_RESPONSE | jq -r '.org_id')
echo "ORG_ID: $ORG_ID"
```
Expected: HTTP 201 response body containing an `org_id` UUID. `ORG_ID` must be a non-empty UUID.
**Step B.2 — Register an agent**
```bash
AGENT_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/agents \
-H "Content-Type: application/json" \
-d "{
\"email\": \"trial-agent@field-trial.sentryagent.ai\",
\"agent_type\": \"classifier\",
\"version\": \"1.0.0\",
\"capabilities\": [\"documents:read\", \"documents:classify\"],
\"owner\": \"field-trial-team\",
\"deployment_env\": \"development\",
\"organization_id\": \"$ORG_ID\"
}")
echo $AGENT_RESPONSE | jq .
AGENT_ID=$(echo $AGENT_RESPONSE | jq -r '.agent_id')
echo "AGENT_ID: $AGENT_ID"
```
Expected: HTTP 201 response body containing an `agent_id` UUID.
**Step B.3 — Generate credentials**
```bash
CRED_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/credentials \
-H "Content-Type: application/json" \
-d "{\"agent_id\": \"$AGENT_ID\"}")
echo $CRED_RESPONSE | jq .
CLIENT_ID=$(echo $CRED_RESPONSE | jq -r '.client_id')
CLIENT_SECRET=$(echo $CRED_RESPONSE | jq -r '.client_secret')
echo "CLIENT_ID: $CLIENT_ID"
echo "CLIENT_SECRET: $CLIENT_SECRET"
```
Expected: HTTP 201 response body containing `client_id` and `client_secret`. The `client_secret`
is only returned once — save it now.
**Step B.4 — Issue an OAuth 2.0 access token**
```bash
TOKEN_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials&client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&scope=read")
echo $TOKEN_RESPONSE | jq .
ACCESS_TOKEN=$(echo $TOKEN_RESPONSE | jq -r '.access_token')
echo "ACCESS_TOKEN obtained: ${ACCESS_TOKEN:0:30}..."
```
Expected: HTTP 200 response body with `access_token`, `token_type: "Bearer"`, `expires_in: 3600`,
`scope: "read"`.
**Step B.5 — Use the token on a protected endpoint**
```bash
curl -s -H "Authorization: Bearer $ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents | jq .
```
Expected: HTTP 200 with a JSON array of agents including the agent registered in Step B.2.
**Step B.6 — Inspect JWT claims**
Decode and inspect the access token structure (without verifying signature):
```bash
echo $ACCESS_TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq .
```
Expected claims:
```json
{
"sub": "<client_id>",
"iss": "https://sentryagent.ai",
"aud": "sentryagent-api",
"scope": "read",
"agent_id": "<agent_id>",
"organization_id": "<org_id>",
"iat": "<issued-at-timestamp>",
"exp": "<expiry-timestamp>",
"jti": "<unique-jwt-id>"
}
```
Verify `exp - iat = 3600` (1 hour TTL).
**Step B.7 — Rotate credentials and verify old token is rejected**
Rotate the credentials (generates a new client_secret, revokes the old one):
```bash
ROTATE_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/credentials \
-H "Content-Type: application/json" \
-d "{\"agent_id\": \"$AGENT_ID\"}")
NEW_CLIENT_ID=$(echo $ROTATE_RESPONSE | jq -r '.client_id')
NEW_CLIENT_SECRET=$(echo $ROTATE_RESPONSE | jq -r '.client_secret')
echo "New credential: $NEW_CLIENT_ID"
```
Attempt to use the old token (must be rejected):
```bash
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents
# Expected: 401
```
Issue a new token with the new credentials:
```bash
NEW_TOKEN_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials&client_id=$NEW_CLIENT_ID&client_secret=$NEW_CLIENT_SECRET&scope=read")
NEW_ACCESS_TOKEN=$(echo $NEW_TOKEN_RESPONSE | jq -r '.access_token')
echo "New token obtained."
```
Verify the new token works:
```bash
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents
# Expected: 200
```
**Step B.8 — Check audit log**
```bash
curl -s -H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
"http://localhost:3000/api/v1/audit?limit=10" | jq .
```
Expected: JSON array of audit events. Verify these action types are present from Steps B.1–B.7:
`agent.created`, `credential.generated`, `token.issued`, `credential.rotated`, `token.revoked`.
---
## Phase C — Guardrails
This phase tests security boundaries. Each test case must be run with the exact command shown
and must produce the specified HTTP status code.
> **Setup:** Ensure `$NEW_ACCESS_TOKEN` is still set from Phase B. Use `export NEW_ACCESS_TOKEN`
> if switching terminals.
**Test C.1 — No Authorization header → 401**
```bash
curl -s -o /dev/null -w "%{http_code}" \
http://localhost:3000/api/v1/agents
```
Expected HTTP status: `401`
**Test C.2 — Malformed JWT → 401**
```bash
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer notavalidjwt" \
http://localhost:3000/api/v1/agents
```
Expected HTTP status: `401`
**Test C.3 — Expired JWT → 401**
Use a known-expired token. Generate one with a 1-second TTL (requires a test helper or
manually craft an expired JWT). For field trial purposes, use this pre-constructed expired token
(signed with a different key — will fail signature verification and return 401):
```bash
EXPIRED_TOKEN="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0ZXN0IiwiZXhwIjoxfQ.invalid"
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $EXPIRED_TOKEN" \
http://localhost:3000/api/v1/agents
```
Expected HTTP status: `401`
**Test C.4 — Valid JWT, wrong scope → 403**
Issue a token with scope `read`, then attempt to access an endpoint requiring scope `write`:
```bash
# The NEW_ACCESS_TOKEN has scope "read"
# Attempt an action requiring "write" scope (create agent)
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-X POST http://localhost:3000/api/v1/agents \
-d '{"email":"scope-test@example.com","agent_type":"custom","version":"1.0.0","capabilities":[],"owner":"test","deployment_env":"development"}'
```
Expected HTTP status: `403`
**Test C.5 — Rate limit: 101 requests → 429 on the 101st**
Send 101 requests in rapid succession. The 101st must return 429.
```bash
for i in $(seq 1 101); do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents)
if [ "$STATUS" = "429" ]; then
echo "Request $i returned 429 (PASS)"
break
fi
done
```
Expected: Output shows `Request 101 returned 429 (PASS)` (or earlier if previous requests in
the session have already counted toward the window).
After this test, wait 60 seconds for the rate limit window to reset, or use a fresh
`client_id` for subsequent tests.
**Test C.6 — Tier limit: exceed free-tier API call limit → 429 with `tier_limit_exceeded`**
The free tier allows 1,000 API calls per day. For field trial, manually set the counter to the
limit value to trigger the guard without making 1,000 real requests:
```bash
# Get the org_id from the token
ORG_ID=$(echo $NEW_ACCESS_TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq -r '.organization_id')
# Force the counter to the limit via Redis CLI
docker compose exec redis redis-cli SET "rate:tier:calls:$ORG_ID" 1001 EX 86400
# The next API call must be rejected
TIER_RESPONSE=$(curl -s -w "\n%{http_code}" \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents)
echo "$TIER_RESPONSE"
```
Expected: HTTP status `429`. Response body must contain `"code":"tier_limit_exceeded"`.
Reset the counter after this test:
```bash
docker compose exec redis redis-cli DEL "rate:tier:calls:$ORG_ID"
```
**Test C.7 — Tenant isolation: Org A token cannot access Org B agents → 403**
Create a second organisation and agent:
```bash
ORG_B_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/organizations \
-H "Content-Type: application/json" \
-d '{"name":"Org B","slug":"org-b"}')
ORG_B_ID=$(echo $ORG_B_RESPONSE | jq -r '.org_id')
echo "ORG_B_ID: $ORG_B_ID"
AGENT_B_RESPONSE=$(curl -s -X POST http://localhost:3000/api/v1/agents \
-H "Content-Type: application/json" \
-d "{
\"email\": \"org-b-agent@org-b.sentryagent.ai\",
\"agent_type\": \"monitor\",
\"version\": \"1.0.0\",
\"capabilities\": [],
\"owner\": \"org-b\",
\"deployment_env\": \"development\",
\"organization_id\": \"$ORG_B_ID\"
}")
AGENT_B_ID=$(echo $AGENT_B_RESPONSE | jq -r '.agent_id')
echo "AGENT_B_ID: $AGENT_B_ID"
```
Attempt to access Org B's agent using Org A's token:
```bash
curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents/$AGENT_B_ID
```
Expected HTTP status: `403`
---
## Phase D — Portal
**Step D.1 — Install portal dependencies**
```bash
cd portal && npm install && cd ..
```
**Step D.2 — Start the portal development server**
```bash
cd portal && npm run dev &
```
Wait 5 seconds for Next.js to compile, then verify it is listening:
```bash
curl -s -o /dev/null -w "%{http_code}" http://localhost:3001
# Expected: 200 or 307 (redirect to /login)
```
**Step D.3 — Verify each portal route loads**
Open a browser and navigate to each of the following URLs. Each must load without a JavaScript
error in the browser console:
| URL | Expected |
|-----|---------|
| `http://localhost:3001/login` | Login page renders |
| `http://localhost:3001/agents` | Agent list renders (may be empty or show auth redirect) |
| `http://localhost:3001/credentials` | Credentials page renders |
| `http://localhost:3001/audit` | Audit log page renders |
| `http://localhost:3001/analytics` | Analytics dashboard renders |
| `http://localhost:3001/settings/tier` | Tier status page renders |
| `http://localhost:3001/compliance` | Compliance report page renders |
| `http://localhost:3001/webhooks` | Webhooks page renders |
| `http://localhost:3001/marketplace` | Marketplace page renders |
All 9 routes must load without a blank page or unhandled error.
**Step D.4 — Verify analytics charts render**
Navigate to `http://localhost:3001/analytics`.
Verify both of the following chart components are present in the page DOM:
```bash
curl -s http://localhost:3001/analytics | grep -c "recharts"
# Expected: 1 or more (recharts is used for TokenTrendChart and AgentHeatmap)
```
**Step D.5 — Verify tier status page**
Navigate to `http://localhost:3001/settings/tier`.
The page must display the current tier (expected: `free` for a new organisation).
**Step D.6 — Stop the portal**
```bash
kill $(lsof -ti:3001)
```
---
## Phase E — AGNTCY Conformance
**Step E.1 — Activate nvm**
```bash
export NVM_DIR="$HOME/.nvm" && source "$NVM_DIR/nvm.sh"
```
**Step E.2 — Run the AGNTCY conformance suite**
```bash
npm run test:agntcy-conformance
```
**Step E.3 — Expected output**
```
AGNTCY Conformance Suite
Agent Card Export
✓ exports valid AGNTCY agent card format
✓ agent card contains required identity fields
Compliance Report
✓ generates SOC2-aligned compliance report
✓ compliance report includes all required control domains
4 passing (Xs)
```
All 4 tests must pass. A failure indicates a regression in AGNTCY conformance.
**What each test validates:**
| Test | What it validates |
|------|------------------|
| `exports valid AGNTCY agent card format` | The `/api/v1/compliance/agent-cards` endpoint returns an array where each card has `id`, `name`, `version`, `capabilities`, `did` fields in AGNTCY format |
| `agent card contains required identity fields` | Each agent card's `identity` block includes `agent_id`, `organization_id`, `did`, and `deployment_env` |
| `generates SOC2-aligned compliance report` | The `/api/v1/compliance/report` endpoint returns a report with `generated_at`, `controls`, `summary` top-level keys |
| `compliance report includes all required control domains` | The `controls` array in the report includes entries for `access_control`, `audit_logging`, `credential_management`, and `tenant_isolation` |
---
## Phase F — Performance Baseline
> **Prerequisite:** Apache Bench (`ab`) must be installed. On Ubuntu: `sudo apt install apache2-utils`.
> Verify: `ab -V`
**Step F.1 — Create a token payload file**
```bash
cat > /tmp/token_payload.json << 'EOF'
grant_type=client_credentials&client_id=REPLACE_CLIENT_ID&client_secret=REPLACE_CLIENT_SECRET&scope=read
EOF
```
Replace `REPLACE_CLIENT_ID` and `REPLACE_CLIENT_SECRET` with `$NEW_CLIENT_ID` and
`$NEW_CLIENT_SECRET` from Phase B:
```bash
cat > /tmp/token_payload.txt << EOF
grant_type=client_credentials&client_id=${NEW_CLIENT_ID}&client_secret=${NEW_CLIENT_SECRET}&scope=read
EOF
```
**Step F.2 — Benchmark token endpoint**
```bash
ab -n 100 -c 10 \
-p /tmp/token_payload.txt \
-T "application/x-www-form-urlencoded" \
http://localhost:3000/api/v1/token
```
**Pass criteria for token endpoint:**
- `Requests per second` > 10
- `Time per request (mean)` < 100 ms
- p95 (95th percentile, shown as `95%` in the `Percentage of requests` table) < 100 ms
- Zero non-2xx responses
**Step F.3 — Benchmark agent list endpoint**
Ensure `$NEW_ACCESS_TOKEN` is still set and valid. Issue a fresh token if needed:
```bash
NEW_ACCESS_TOKEN=$(curl -s -X POST http://localhost:3000/api/v1/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials&client_id=${NEW_CLIENT_ID}&client_secret=${NEW_CLIENT_SECRET}&scope=read" \
| jq -r '.access_token')
```
Run the benchmark:
```bash
ab -n 100 -c 10 \
-H "Authorization: Bearer $NEW_ACCESS_TOKEN" \
http://localhost:3000/api/v1/agents
```
**Pass criteria for agent list endpoint:**
- `Time per request (mean)` < 200 ms
- p95 (`95%` row in the `Percentage of requests` table) < 200 ms
- Zero non-2xx responses
**Step F.4 — Record results**
Record the following values from each `ab` output for the field trial report:
| Endpoint | Metric | Value |
|----------|--------|-------|
| `/api/v1/token` | Requests per second | |
| `/api/v1/token` | Mean time per request (ms) | |
| `/api/v1/token` | p95 (ms) | |
| `/api/v1/agents` | Requests per second | |
| `/api/v1/agents` | Mean time per request (ms) | |
| `/api/v1/agents` | p95 (ms) | |
A field trial passes Phase F if all p95 values are within the pass criteria above.
---
## Troubleshooting
Each entry follows the pattern: **Symptom** → **Cause** → **Fix** with exact commands.
---
**Port already in use**
Symptom:
```
Error response from daemon: driver failed programming external connectivity on endpoint
sentryagent-idp-app-1: Bind for 0.0.0.0:3000 failed: port is already allocated
```
Fix: Kill the process occupying the port, then restart:
```bash
lsof -ti:3000 | xargs kill
lsof -ti:5432 | xargs kill
lsof -ti:6379 | xargs kill
docker compose up --build -d
```
---
**Container shows `unhealthy`**
Symptom: `docker compose ps` shows `unhealthy` for a service.
Fix: Check logs for the unhealthy service:
```bash
docker compose logs postgres
docker compose logs redis
docker compose logs app
```
Common causes:
| Service | Cause | Fix |
|---------|-------|-----|
| `postgres` | Wrong database credentials | Verify `DATABASE_URL` in `.env` matches `docker-compose.yml` credentials |
| `redis` | Port conflict | Check `lsof -ti:6379` and kill occupying process |
| `app` | Missing env var | Check `docker compose logs app` for `Failed to start server` message |
---
**Migration fails — connection refused**
Symptom:
```
Migration failed: Error: connect ECONNREFUSED 127.0.0.1:5432
```
Cause: Running `npm run db:migrate` directly on the host (not inside the container) while
PostgreSQL is running inside Docker.
Fix: Always run migrations inside the container during a field trial:
```bash
docker compose exec app npm run db:migrate
```
---
**Migration fails — relation already exists**
Symptom:
```
Migration failed: Error: relation "agents" already exists
```
Cause: A previous partial migration run left the database in an inconsistent state.
Fix: Check which migrations have been applied:
```bash
docker compose exec postgres psql -U agentidp -d agentidp \
-c "SELECT name FROM schema_migrations ORDER BY name;"
```
If the database state cannot be repaired, reset it:
```bash
docker compose down -v
docker compose up --build -d
docker compose exec app npm run db:migrate
```
> `docker compose down -v` destroys all data. Use only when a clean slate is acceptable.
---
**JWT error — invalid signature or key format**
Symptom:
```
Failed to start server: Error: JWT_PRIVATE_KEY and JWT_PUBLIC_KEY environment variables are required
```
Or: All tokens return `401 Token signature is invalid`.
Cause: JWT keys in `.env` have incorrect PEM format — literal newlines instead of `\n`
sequences, or trailing whitespace.
Fix: Regenerate the keys and re-write them using the exact commands from Step 0.2 and 0.3.
Verify the key format in `.env`:
```bash
grep "JWT_PRIVATE_KEY" .env | head -c 100
# Expected: JWT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nMII...
# NOT: JWT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
# MII...
```
The entire key must be on a single line with `\n` as literal backslash-n characters, not
actual newlines.
---
**Portal CORS error**
Symptom: Browser console shows:
```
Access to XMLHttpRequest at 'http://localhost:3000/api/v1/...' from origin 'http://localhost:3001'
has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present
```
Cause: `CORS_ORIGIN` in `.env` does not include `http://localhost:3001`, or is set to a
different value.
Fix:
```bash
sed -i "s|CORS_ORIGIN=.*|CORS_ORIGIN=http://localhost:3001|" .env
docker compose up --build -d
```
Wait for the `app` container to become healthy before retrying.
---
**Tier counter not resetting**
Symptom: All API calls return 429 `tier_limit_exceeded` even after waiting.
Cause: The Redis tier counter was manually set in Test C.6 and not deleted.
Fix:
```bash
# Get your org_id from the token
ORG_ID=$(echo $NEW_ACCESS_TOKEN | cut -d. -f2 | base64 -d 2>/dev/null | jq -r '.organization_id')
docker compose exec redis redis-cli DEL "rate:tier:calls:$ORG_ID"
docker compose exec redis redis-cli DEL "rate:tier:tokens:$ORG_ID"
```
---
**`ab` not found**
Symptom: `ab: command not found`
Fix:
```bash
sudo apt-get update && sudo apt-get install -y apache2-utils
# or on macOS:
brew install httpd
```
---
**AGNTCY conformance test fails**
Symptom: One or more tests in `npm run test:agntcy-conformance` fail.
Diagnosis steps:
1. Ensure the backend is running and healthy: `curl -s http://localhost:3000/health`
2. Ensure `COMPLIANCE_ENABLED=true` in `.env` (check with `grep COMPLIANCE_ENABLED .env`)
3. Ensure at least one agent has been registered (Phase B must have been completed)
4. Check the test output for the specific assertion that failed
5. Check `docker compose logs app` for errors around compliance report generation
If the issue is a Redis cache hit returning stale data:
```bash
docker compose exec redis redis-cli KEYS "compliance:*" | xargs docker compose exec redis redis-cli DEL
```
Then re-run the conformance suite.