docs: commit all Phase 6 documentation updates and OpenSpec archives
- devops docs: 8 files updated for Phase 6 state; field-trial.md added (946-line runbook) - developer docs: api-reference (50+ endpoints), quick-start, 5 existing guides updated, 5 new guides added - engineering docs: all 12 files updated (services, architecture, SDK guide, testing, overview) - OpenSpec archives: phase-7-devops-field-trial, developer-docs-phase6-update, engineering-docs-phase6-update - VALIDATOR.md + scripts/start-validator.sh: V&V Architect tooling added - .gitignore: exclude session artifacts, build artifacts, and agent workspaces Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
SentryAgent.ai Developer
@@ -1,18 +1,28 @@
|
||||
# Database
|
||||
|
||||
AgentIdP uses PostgreSQL 14+ as its primary data store. The schema consists of four tables managed by a custom migration runner.
|
||||
AgentIdP uses PostgreSQL 14+ as its primary data store. The schema consists of 26 migrations managed by a custom migration runner.
|
||||
|
||||
---
|
||||
|
||||
## Schema Overview
|
||||
|
||||
```
|
||||
agents
|
||||
└── credentials (FK: client_id → agents.agent_id, CASCADE DELETE)
|
||||
|
||||
audit_events (no FK — append-only, agent_id is informational)
|
||||
organizations
|
||||
├── agents (FK: organization_id → organizations.org_id)
|
||||
│ ├── credentials (FK: client_id → agents.agent_id, CASCADE DELETE)
|
||||
│ └── agent_did_keys (FK: agent_id → agents.agent_id)
|
||||
└── audit_events (FK: organization_id — informational, no cascade)
|
||||
|
||||
token_revocations (no FK — independent revocation store)
|
||||
oidc_keys (standalone — OIDC signing key rotation)
|
||||
federation_partners (standalone — cross-tenant identity)
|
||||
webhook_subscriptions → webhook_deliveries (FK: subscription_id)
|
||||
agent_marketplace (standalone — agent discovery catalog)
|
||||
github_oidc_trust_policies (standalone — CI/CD trust)
|
||||
billing (FK: org_id → organizations.org_id — one row per org)
|
||||
delegation_chains (standalone — A2A delegation records)
|
||||
analytics_events (FK: organization_id — append-only)
|
||||
tenant_tiers (FK: org_id → organizations.org_id — one row per org)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -134,6 +144,234 @@ Durable record of revoked JWT tokens. Supplements Redis for durability across Re
|
||||
|
||||
---
|
||||
|
||||
### `organizations`
|
||||
|
||||
Created by migration `006_create_organizations_table.sql`.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `org_id` | `UUID` | No | Primary key |
|
||||
| `name` | `VARCHAR(255)` | No | Organisation display name |
|
||||
| `slug` | `VARCHAR(64)` | No | URL-safe unique identifier |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `agent_did_keys`
|
||||
|
||||
Created by migration `012_create_agent_did_keys_table.sql`.
|
||||
|
||||
Stores the DID document key material for each agent. One agent may have multiple keys for
|
||||
rotation purposes.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `agent_id` | `UUID` | No | FK → `agents.agent_id` |
|
||||
| `key_id` | `VARCHAR(255)` | No | DID key fragment identifier |
|
||||
| `public_key_jwk` | `JSONB` | No | Public key in JWK format |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### DID columns on `agents`
|
||||
|
||||
Added by migration `013_add_did_columns_to_agents.sql`:
|
||||
|
||||
- `did` — `VARCHAR(512)` nullable — the `did:web` identifier for this agent
|
||||
- `did_document` — `JSONB` nullable — full DID document
|
||||
|
||||
---
|
||||
|
||||
### `oidc_keys`
|
||||
|
||||
Created by migration `014_create_oidc_keys_table.sql`.
|
||||
|
||||
Stores RSA key pairs used for OIDC ID token signing. Supports key rotation — active key is
|
||||
determined by the most recently created row.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `kid` | `VARCHAR(128)` | No | Key ID — referenced in JWKS |
|
||||
| `private_key_pem` | `TEXT` | No | Encrypted RSA private key (pgcrypto) |
|
||||
| `public_key_pem` | `TEXT` | No | RSA public key |
|
||||
| `algorithm` | `VARCHAR(16)` | No | Always `RS256` |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `federation_partners`
|
||||
|
||||
Created by migration `015_create_federation_partners_table.sql`.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `org_id` | `UUID` | No | Owning organisation |
|
||||
| `partner_name` | `VARCHAR(255)` | No | Display name |
|
||||
| `partner_jwks_url` | `TEXT` | No | URL to partner's JWKS endpoint |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `webhook_subscriptions`
|
||||
|
||||
Created by migration `016_create_webhook_subscriptions_table.sql`.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `org_id` | `UUID` | No | Owning organisation |
|
||||
| `event_type` | `VARCHAR(128)` | No | Event type filter (e.g. `agent.created`) |
|
||||
| `target_url` | `TEXT` | No | HTTPS delivery endpoint |
|
||||
| `secret` | `VARCHAR(255)` | Yes | HMAC signing secret for delivery verification |
|
||||
| `active` | `BOOLEAN` | No | Default: `true` |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `webhook_deliveries`
|
||||
|
||||
Created by migration `017_create_webhook_deliveries_table.sql`.
|
||||
|
||||
Records each delivery attempt for a webhook event, including the dead-letter queue entries.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `subscription_id` | `UUID` | No | FK → `webhook_subscriptions.id` |
|
||||
| `event_type` | `VARCHAR(128)` | No | Event type delivered |
|
||||
| `payload` | `JSONB` | No | Full event payload |
|
||||
| `status` | `VARCHAR(32)` | No | `pending`, `delivered`, `failed`, `dead_letter` |
|
||||
| `response_status` | `INTEGER` | Yes | HTTP status from delivery endpoint |
|
||||
| `attempt_count` | `INTEGER` | No | Default: `0` |
|
||||
| `last_attempted_at` | `TIMESTAMPTZ` | Yes | |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
**Dead-letter queue:** After 3 failed delivery attempts, the row status is set to `dead_letter`
|
||||
and the `agentidp_webhook_dead_letters_total` Prometheus counter is incremented. The Prometheus
|
||||
metric label is `event_type`.
|
||||
|
||||
---
|
||||
|
||||
### pgcrypto extension
|
||||
|
||||
Enabled by migration `018_enable_pgcrypto.sql`. Used for encrypting sensitive columns in
|
||||
`oidc_keys` and credential data.
|
||||
|
||||
---
|
||||
|
||||
### `agent_marketplace`
|
||||
|
||||
Created by migration `021_add_agent_marketplace.sql`.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `agent_id` | `UUID` | No | FK → `agents.agent_id` |
|
||||
| `listing_name` | `VARCHAR(255)` | No | Display name in marketplace |
|
||||
| `description` | `TEXT` | Yes | Markdown description |
|
||||
| `tags` | `TEXT[]` | No | Searchable tags. Default: `{}` |
|
||||
| `published` | `BOOLEAN` | No | Default: `false` |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `github_oidc_trust_policies`
|
||||
|
||||
Created by migration `022_add_github_oidc_trust_policies.sql`.
|
||||
|
||||
Maps GitHub Actions OIDC claims to agent identities for CI/CD token exchange.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `org_id` | `UUID` | No | Owning organisation |
|
||||
| `repository` | `VARCHAR(512)` | No | GitHub repository slug (`owner/repo`) |
|
||||
| `branch` | `VARCHAR(255)` | Yes | Branch filter (null = any branch) |
|
||||
| `agent_id` | `UUID` | No | Agent to issue a token for on match |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `billing`
|
||||
|
||||
Created by migration `023_add_billing.sql`.
|
||||
|
||||
One row per organisation. Tracks the org's Stripe customer and subscription state.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `org_id` | `UUID` | No | FK → `organizations.org_id` (UNIQUE) |
|
||||
| `stripe_customer_id` | `VARCHAR(255)` | Yes | Stripe Customer ID |
|
||||
| `stripe_subscription_id` | `VARCHAR(255)` | Yes | Stripe Subscription ID |
|
||||
| `status` | `VARCHAR(64)` | No | Stripe subscription status or `none` |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `delegation_chains`
|
||||
|
||||
Created by migration `024_add_delegation_chains.sql`.
|
||||
|
||||
Records A2A delegation grants created via the delegation API.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `delegator_agent_id` | `UUID` | No | Agent granting the delegation |
|
||||
| `delegate_agent_id` | `UUID` | No | Agent receiving the delegation |
|
||||
| `scopes` | `TEXT[]` | No | Scopes being delegated |
|
||||
| `expires_at` | `TIMESTAMPTZ` | Yes | Optional expiry |
|
||||
| `created_at` | `TIMESTAMPTZ` | No | Default: `NOW()` |
|
||||
|
||||
---
|
||||
|
||||
### `analytics_events`
|
||||
|
||||
Created by migration `025_add_analytics_events.sql`.
|
||||
|
||||
Append-only event store for analytics. Supports token trend, agent activity, and usage summary
|
||||
queries.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `organization_id` | `UUID` | No | Owning organisation |
|
||||
| `date` | `DATE` | No | Calendar date of the event (UTC) |
|
||||
| `metric_type` | `VARCHAR(64)` | No | e.g. `token_issued`, `agent_called` |
|
||||
| `count` | `INTEGER` | No | Event count for this date+type |
|
||||
|
||||
**Index:** `(organization_id, date DESC)` for fast time-series queries.
|
||||
|
||||
---
|
||||
|
||||
### `tenant_tiers`
|
||||
|
||||
Created by migration `026_add_tenant_tiers.sql`.
|
||||
|
||||
One row per organisation. Stores the current tier and enforces tier limits via the
|
||||
`tierEnforcement` middleware.
|
||||
|
||||
| Column | Type | Nullable | Description |
|
||||
|--------|------|----------|-------------|
|
||||
| `id` | `UUID` | No | Primary key |
|
||||
| `org_id` | `UUID` | No | FK → `organizations.org_id` (UNIQUE) |
|
||||
| `tier` | `ENUM('free','pro','enterprise')` | No | Current tier. Default: `free` |
|
||||
| `updated_at` | `TIMESTAMPTZ` | No | Last tier change. Default: `NOW()` |
|
||||
|
||||
**Tier limits** (from `src/config/tiers.ts`):
|
||||
|
||||
| Tier | Max Agents | Max API Calls/Day | Max Tokens/Day |
|
||||
|------|-----------|-------------------|----------------|
|
||||
| free | 10 | 1,000 | 1,000 |
|
||||
| pro | 100 | 50,000 | 50,000 |
|
||||
| enterprise | unlimited | unlimited | unlimited |
|
||||
|
||||
---
|
||||
|
||||
## Migration Runner
|
||||
|
||||
Migrations are managed by `scripts/migrate.ts`. It reads `.sql` files from `src/db/migrations/` in alphabetical order, tracks applied migrations in a `schema_migrations` table, and executes only unapplied migrations — each in its own transaction.
|
||||
@@ -160,10 +398,11 @@ Expected output (first run):
|
||||
Running database migrations...
|
||||
✓ Applied: 001_create_agents.sql
|
||||
✓ Applied: 002_create_credentials.sql
|
||||
✓ Applied: 003_create_audit_events.sql
|
||||
✓ Applied: 004_create_tokens.sql
|
||||
...
|
||||
✓ Applied: 025_add_analytics_events.sql
|
||||
✓ Applied: 026_add_tenant_tiers.sql
|
||||
|
||||
Migrations complete. 4 migration(s) applied.
|
||||
Migrations complete. 26 migration(s) applied.
|
||||
```
|
||||
|
||||
Expected output (already applied):
|
||||
@@ -191,9 +430,10 @@ Expected output:
|
||||
-----------------------------------+-------------------------------
|
||||
001_create_agents.sql | 2026-03-28 09:00:00.000000+00
|
||||
002_create_credentials.sql | 2026-03-28 09:00:00.000000+00
|
||||
003_create_audit_events.sql | 2026-03-28 09:00:00.000000+00
|
||||
004_create_tokens.sql | 2026-03-28 09:00:00.000000+00
|
||||
(4 rows)
|
||||
...
|
||||
025_add_analytics_events.sql | 2026-04-04 09:00:00.000000+00
|
||||
026_add_tenant_tiers.sql | 2026-04-04 09:00:00.000000+00
|
||||
(26 rows)
|
||||
```
|
||||
|
||||
### Adding a new migration
|
||||
@@ -214,6 +454,15 @@ There is no automated rollback. To undo a migration:
|
||||
|
||||
## Connection Pool
|
||||
|
||||
The application uses `pg.Pool` with default settings (max 10 connections). The pool is a singleton — one pool per process instance.
|
||||
The application uses `pg.Pool` with settings read from environment variables. The pool is a
|
||||
singleton — one pool per process instance.
|
||||
|
||||
To override pool size, modify `src/db/pool.ts`. In production, ensure `DATABASE_URL` includes connection pool parameters if using PgBouncer or a managed connection pooler.
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `DB_POOL_MAX` | `20` | Maximum connections |
|
||||
| `DB_POOL_MIN` | `2` | Minimum idle connections |
|
||||
| `DB_POOL_IDLE_TIMEOUT_MS` | `30000` | Idle eviction timeout (ms) |
|
||||
| `DB_POOL_CONNECTION_TIMEOUT_MS` | `5000` | Acquisition timeout (ms) |
|
||||
|
||||
Pool size is exposed as Prometheus metrics: `agentidp_db_pool_active_connections` and
|
||||
`agentidp_db_pool_waiting_requests`. Monitor these in production to detect pool exhaustion.
|
||||
|
||||
Reference in New Issue
Block a user