From fec1801e8c0488e6aa66305a0735eb3ff0008319 Mon Sep 17 00:00:00 2001 From: "SentryAgent.ai Developer" Date: Thu, 2 Apr 2026 15:42:05 +0000 Subject: [PATCH] chore(openspec): trim phase-5 scope to WS1+WS2+WS5 per CEO approval MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Approved: Rust SDK, A2A Authorization, Developer Experience. Deferred to Phase 6: Analytics Dashboard, API Gateway Tiers, AGNTCY Compliance. Tasks: 119 → 76. Specs: 6 → 3. Co-Authored-By: Claude Sonnet 4.6 --- .../phase-5-scale-ecosystem/proposal.md | 4 +- .../specs/agntcy-compliance/spec.md | 320 ------------------ .../specs/analytics-dashboard/spec.md | 279 --------------- .../specs/api-gateway-tiers/spec.md | 276 --------------- .../changes/phase-5-scale-ecosystem/tasks.md | 145 ++------ 5 files changed, 38 insertions(+), 986 deletions(-) delete mode 100644 openspec/changes/phase-5-scale-ecosystem/specs/agntcy-compliance/spec.md delete mode 100644 openspec/changes/phase-5-scale-ecosystem/specs/analytics-dashboard/spec.md delete mode 100644 openspec/changes/phase-5-scale-ecosystem/specs/api-gateway-tiers/spec.md diff --git a/openspec/changes/phase-5-scale-ecosystem/proposal.md b/openspec/changes/phase-5-scale-ecosystem/proposal.md index bf96601..8d02d0b 100644 --- a/openspec/changes/phase-5-scale-ecosystem/proposal.md +++ b/openspec/changes/phase-5-scale-ecosystem/proposal.md @@ -69,4 +69,6 @@ Phase 4 made SentryAgent.ai discoverable and adoptable — developers can now fi | `archiver` | `src/` (API) | ZIP archive creation for scaffold generator — battle-tested Node.js archiver | | `@stoplight/elements` | `portal/` | Swagger UI v5 / Elements theme — modern, interactive, component-based API docs | -**Delivery sequence:** WS1 → WS2 → WS3 + WS4 (parallel) → WS5 → WS6 +**Approved scope (CEO-confirmed 2026-04-02):** WS1 → WS2 → WS5 + +**Deferred to Phase 6:** WS3 (Advanced Analytics), WS4 (API Gateway Tiers), WS6 (AGNTCY Compliance Certification) diff --git a/openspec/changes/phase-5-scale-ecosystem/specs/agntcy-compliance/spec.md b/openspec/changes/phase-5-scale-ecosystem/specs/agntcy-compliance/spec.md deleted file mode 100644 index b57c68e..0000000 --- a/openspec/changes/phase-5-scale-ecosystem/specs/agntcy-compliance/spec.md +++ /dev/null @@ -1,320 +0,0 @@ -## WS6: AGNTCY Compliance Certification Package - -### Purpose - -Position SentryAgent.ai as the reference implementation for the AGNTCY standard. Deliver four artifacts: (1) an auto-generated machine-readable AGNTCY compliance report endpoint; (2) an agent card export endpoint per the AGNTCY Agent Card specification; (3) a Jest-based interoperability test suite verifying AGNTCY alignment on every CI run; (4) a human-readable certification guide documenting how SentryAgent.ai satisfies each AGNTCY requirement. - -This workstream produces no user-facing UI changes. It is infrastructure for compliance, certification, and ecosystem trust. - -### New Endpoints - -#### `GET /agntcy/compliance-report` - -**Summary:** Generate and return a real-time AGNTCY compliance report for the authenticated tenant's environment. - -**Authentication:** Bearer token (tenant-scoped). The tenant's subscription tier must be `pro` or `enterprise`. - -**Response 200** (`application/json`): -```json -{ - "reportId": "string (UUID)", - "generatedAt": "string (ISO 8601)", - "agntcySpecVersion": "1.0.0", - "tenantId": "string (UUID)", - "overallStatus": "compliant", - "sections": [ - { - "id": "agent-identity", - "name": "Agent Identity", - "status": "compliant", - "requirements": [ - { - "id": "AI-001", - "description": "Each agent MUST have a globally unique, persistent identifier", - "status": "compliant", - "evidence": "All agents are assigned a UUID v4 at registration, stored immutably in agents.id", - "verifiedAt": "string (ISO 8601)" - }, - { - "id": "AI-002", - "description": "Each agent MUST have a W3C DID document", - "status": "compliant", - "evidence": "DID documents are auto-generated as did:web identifiers at agent registration", - "verifiedAt": "string (ISO 8601)" - } - ] - }, - { - "id": "authentication", - "name": "Authentication", - "status": "compliant", - "requirements": [ - { - "id": "AUTH-001", - "description": "Agent authentication MUST use OAuth 2.0 or OIDC", - "status": "compliant", - "evidence": "OAuth 2.0 Client Credentials flow implemented at POST /oauth2/token", - "verifiedAt": "string (ISO 8601)" - } - ] - }, - { - "id": "authorization", - "name": "Authorization", - "status": "compliant", - "requirements": [] - }, - { - "id": "audit-and-governance", - "name": "Audit & Governance", - "status": "compliant", - "requirements": [] - }, - { - "id": "interoperability", - "name": "Interoperability", - "status": "compliant", - "requirements": [] - }, - { - "id": "delegation", - "name": "Agent-to-Agent Delegation", - "status": "compliant", - "requirements": [] - } - ], - "summary": { - "totalRequirements": 24, - "compliant": 24, - "nonCompliant": 0, - "notApplicable": 0 - } -} -``` - -**`overallStatus`** values: `"compliant"` | `"partial"` | `"non-compliant"` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 401 | `UNAUTHORIZED` | Missing or invalid Bearer token | -| 403 | `TIER_REQUIRED` | Compliance report requires Pro or Enterprise tier | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - -**Business Rules:** -- Report is generated on demand from live system state — no cache -- Each requirement's `status` is computed by querying current system configuration (e.g., verify DID documents exist by checking `agents` table, verify audit log is enabled by checking config) -- `agntcySpecVersion` is hardcoded to the AGNTCY spec version the system was last validated against -- An audit log entry is created with `event_type: "compliance.report_generated"` - ---- - -#### `GET /agents/:id/agent-card` - -**Summary:** Return the AGNTCY-compliant Agent Card for a specific agent. Agent Cards are publicly accessible for public agents and require authentication for private agents. - -**Authentication:** Optional. Required only if the agent's `is_public` is `false`. - -**Path Parameter:** - -| Parameter | Type | Description | -|---|---|---| -| `id` | string (UUID) | Agent ID | - -**Response 200** (`application/json`): - -Per the AGNTCY Agent Card specification: -```json -{ - "agntcyVersion": "1.0", - "type": "agent-card", - "agent": { - "id": "string (UUID)", - "name": "string", - "description": "string | null", - "did": "did:web:sentryagent.ai:agents:abc123", - "capabilities": ["string"], - "version": "string", - "publisher": { - "tenantId": "string (UUID)", - "name": "string" - }, - "endpoints": { - "tokenEndpoint": "https://api.sentryagent.ai/oauth2/token", - "delegationEndpoint": "https://api.sentryagent.ai/oauth2/token/delegate" - }, - "authentication": { - "schemes": ["oauth2_client_credentials"], - "tokenEndpoint": "https://api.sentryagent.ai/oauth2/token" - }, - "governance": { - "auditLogEnabled": true, - "credentialRotationPolicy": "manual", - "complianceStandards": ["AGNTCY-1.0", "OAuth2-RFC6749", "W3C-DID"] - }, - "metadata": {} - }, - "issuedAt": "string (ISO 8601)", - "expiresAt": "string (ISO 8601)" -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 401 | `UNAUTHORIZED` | Agent is private and no Bearer token provided | -| 403 | `FORBIDDEN` | Agent is private and authenticated tenant does not own it | -| 404 | `AGENT_NOT_FOUND` | No agent with the given ID | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - -**Business Rules:** -- Public agents (`is_public: true`) return agent card without authentication -- Private agents require the owning tenant's Bearer token -- Agent card `expiresAt` is `issuedAt + 24 hours` (cards are short-lived — consumers should re-fetch daily) -- `complianceStandards` array is sourced from system config, not per-agent configuration - ---- - -### AGNTCY Interoperability Test Suite - -**File:** `tests/agntcy/interoperability.test.ts` - -A Jest test suite that verifies AGNTCY alignment on every CI run. Tests run against a live API instance (reads `AGENTIDP_API_URL` from environment). - -**Test categories and cases:** - -```typescript -// AGNTCY-AI-001: Agent identity uniqueness -test('each registered agent receives a unique UUID', ...) -test('agent UUID is immutable after registration', ...) - -// AGNTCY-AI-002: W3C DID documents -test('registered agent has a valid did:web DID', ...) -test('DID document resolves via GET /agents/:id', ...) - -// AGNTCY-AUTH-001: OAuth 2.0 token issuance -test('POST /oauth2/token returns access_token and token_type: bearer', ...) -test('access token is a valid JWT with correct claims', ...) -test('expired token is rejected with 401', ...) - -// AGNTCY-AUTH-002: OIDC compliance -test('GET /.well-known/openid-configuration returns valid OIDC discovery document', ...) -test('JWKS endpoint returns valid JWK Set', ...) - -// AGNTCY-AUTHZ-001: Scope-based access control -test('token with agent:read scope cannot call agent:write operations', ...) -test('scopes are included in JWT payload', ...) - -// AGNTCY-DEL-001: Agent-to-Agent delegation -test('POST /oauth2/token/delegate creates a valid delegation chain', ...) -test('delegated scopes cannot exceed delegator scopes', ...) -test('POST /oauth2/token/verify-delegation returns valid: true for active chain', ...) -test('POST /oauth2/token/verify-delegation returns valid: false for expired chain', ...) - -// AGNTCY-AUDIT-001: Immutable audit logs -test('every token issuance creates an audit log entry', ...) -test('audit log entries cannot be deleted via API', ...) - -// AGNTCY-GOV-001: Agent lifecycle governance -test('credential rotation is logged in audit log', ...) -test('agent deletion logs deletion event in audit log', ...) - -// AGNTCY-INTER-001: Agent Card export -test('GET /agents/:id/agent-card returns valid AGNTCY Agent Card', ...) -test('Agent Card contains required agntcyVersion, did, capabilities fields', ...) - -// AGNTCY-COMP-001: Compliance report -test('GET /agntcy/compliance-report returns compliant status', ...) -test('compliance report covers all 6 AGNTCY sections', ...) -test('compliance report totalRequirements >= 24', ...) -``` - -**Running the suite:** -```bash -# In CI (requires live API): -AGENTIDP_API_URL=http://localhost:3000 npm run test:agntcy - -# Added to package.json: -"test:agntcy": "jest --testPathPattern=tests/agntcy --forceExit" -``` - ---- - -### AGNTCY Certification Guide - -**File:** `docs/agntcy/certification-guide.md` - -A markdown document structured as follows: -1. **Overview** — What AGNTCY certification means and how SentryAgent.ai achieves it -2. **Requirement Mapping** — Table mapping each AGNTCY requirement ID to the SentryAgent.ai implementation (endpoint, service, or config) -3. **Running the Compliance Report** — Step-by-step guide to generating and interpreting the compliance report -4. **Agent Card Usage** — How to retrieve, cache, and use Agent Cards in multi-agent workflows -5. **Self-Certification Checklist** — Checklist for operators deploying self-hosted SentryAgent.ai to verify their instance's compliance -6. **Submitting for Official AGNTCY Certification** — Links and instructions for the Linux Foundation AGNTCY certification program - ---- - -### New Source Files - -| File | Description | -|---|---| -| `src/services/ComplianceService.ts` | Business logic: query system state, evaluate each AGNTCY requirement, build report | -| `src/controllers/ComplianceController.ts` | HTTP handlers for compliance report and agent card endpoints | -| `src/routes/agntcy.ts` | Express router: `GET /agntcy/compliance-report`, `GET /agents/:id/agent-card` | -| `src/types/compliance.ts` | TypeScript interfaces: `ComplianceReport`, `ComplianceSection`, `ComplianceRequirement`, `AgentCard` | -| `src/config/agntcyRequirements.ts` | Static array of AGNTCY requirement definitions (id, description, evaluator function reference) | -| `tests/agntcy/interoperability.test.ts` | Jest interoperability test suite | -| `docs/agntcy/certification-guide.md` | Human-readable certification guide | - -### Modified Source Files - -| File | Change | -|---|---| -| `src/routes/index.ts` | Register `agntcy` router | -| `src/routes/agents.ts` | Add `GET /agents/:id/agent-card` route (or register via agntcy router — agent-card is agent-scoped) | -| `package.json` (API) | Add `"test:agntcy"` script | -| `docs/openapi.yaml` | Add `GET /agntcy/compliance-report` and `GET /agents/:id/agent-card` endpoints | - -### `ComplianceService` Interface - -```typescript -interface IComplianceService { - /** - * Generate a live AGNTCY compliance report for the given tenant. - * Evaluates all registered AGNTCY requirements against current system state. - */ - generateComplianceReport(tenantId: string): Promise; - - /** - * Generate an AGNTCY Agent Card for a specific agent. - */ - generateAgentCard(agentId: string): Promise; -} -``` - -### Prometheus Metrics - -| Metric | Type | Labels | Description | -|---|---|---|---| -| `agentidp_compliance_reports_generated_total` | Counter | `tenant_id` | Total compliance reports generated | -| `agentidp_compliance_report_duration_ms` | Histogram | — | Time to generate compliance report | -| `agentidp_agent_cards_served_total` | Counter | `visibility` (public/private) | Agent cards served by visibility | - -### Feature Flag - -`AGNTCY_ENABLED` (default: `true`). When `false`, all `/agntcy/` routes and `GET /agents/:id/agent-card` return HTTP 404. - -### Acceptance Criteria - -- `GET /agntcy/compliance-report` returns a report with `overallStatus: "compliant"` on a correctly configured instance -- Report contains all 6 sections: agent-identity, authentication, authorization, audit-and-governance, interoperability, delegation -- Report `totalRequirements >= 24` -- `GET /agents/:id/agent-card` returns a valid AGNTCY Agent Card with all required fields -- Agent Card is accessible without auth for public agents -- Agent Card requires owning tenant's auth for private agents -- All 25+ interoperability test cases pass against a live API instance -- `npm run test:agntcy` exits 0 on a correctly configured instance -- `docs/agntcy/certification-guide.md` is complete — no TODOs, no placeholders -- Unit tests cover: compliance report generation (compliant system, partially compliant), agent card generation (public agent, private agent) diff --git a/openspec/changes/phase-5-scale-ecosystem/specs/analytics-dashboard/spec.md b/openspec/changes/phase-5-scale-ecosystem/specs/analytics-dashboard/spec.md deleted file mode 100644 index e4643d2..0000000 --- a/openspec/changes/phase-5-scale-ecosystem/specs/analytics-dashboard/spec.md +++ /dev/null @@ -1,279 +0,0 @@ -## WS3: Advanced Analytics Dashboard - -### Purpose - -Give paying tenants actionable visibility into their agent usage patterns. Analytics surface four dimensions: agent activity over time (heatmap), token issuance frequency and volume (trends), credential rotation frequency (rotation frequency table), and per-endpoint API call patterns (call patterns breakdown). Data is pre-aggregated nightly from the existing `usage_events` table into a new `analytics_daily_aggregates` table. Analytics are rendered in a new Analytics tab in the existing React web dashboard. - -### New Endpoints - -#### `GET /analytics/usage-summary` - -**Summary:** Return a high-level usage summary for the authenticated tenant over a date range. - -**Authentication:** Bearer token (tenant-scoped). - -**Query Parameters:** - -| Parameter | Type | Required | Default | Constraints | -|---|---|---|---|---| -| `from` | string (YYYY-MM-DD) | no | 30 days ago | Must be <= `to` | -| `to` | string (YYYY-MM-DD) | no | today | Must be <= today | - -**Response 200** (`application/json`): -```json -{ - "tenantId": "string (UUID)", - "period": { - "from": "string (YYYY-MM-DD)", - "to": "string (YYYY-MM-DD)" - }, - "summary": { - "totalApiCalls": 84320, - "totalTokenIssuances": 12400, - "totalCredentialRotations": 48, - "activeAgentCount": 23, - "averageDailyApiCalls": 2810, - "peakDailyApiCalls": 5102, - "peakDate": "2026-03-28" - } -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 400 | `INVALID_DATE_RANGE` | `from` > `to`, or date range exceeds 365 days | -| 401 | `UNAUTHORIZED` | Missing or invalid Bearer token | -| 403 | `ANALYTICS_NOT_AVAILABLE` | Tenant is on free tier — analytics require Pro or Enterprise | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - ---- - -#### `GET /analytics/agent-activity` - -**Summary:** Return per-agent daily activity counts for heatmap rendering. - -**Authentication:** Bearer token (tenant-scoped). - -**Query Parameters:** - -| Parameter | Type | Required | Default | Constraints | -|---|---|---|---|---| -| `from` | string (YYYY-MM-DD) | no | 30 days ago | Must be <= `to` | -| `to` | string (YYYY-MM-DD) | no | today | Max range: 90 days | -| `agentId` | string (UUID) | no | (all agents) | Filter to a single agent | - -**Response 200** (`application/json`): -```json -{ - "tenantId": "string (UUID)", - "period": { - "from": "string (YYYY-MM-DD)", - "to": "string (YYYY-MM-DD)" - }, - "agents": [ - { - "agentId": "string (UUID)", - "agentName": "string", - "dailyActivity": [ - { - "date": "2026-03-01", - "apiCalls": 342, - "tokenIssuances": 12, - "credentialRotations": 0 - } - ] - } - ] -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 400 | `INVALID_DATE_RANGE` | `from` > `to`, or date range exceeds 90 days | -| 401 | `UNAUTHORIZED` | Missing or invalid Bearer token | -| 403 | `ANALYTICS_NOT_AVAILABLE` | Free tier — requires Pro or Enterprise | -| 404 | `AGENT_NOT_FOUND` | `agentId` filter specified but agent does not belong to tenant | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - ---- - -#### `GET /analytics/token-trends` - -**Summary:** Return daily token issuance counts and success/failure breakdown for trend charts. - -**Authentication:** Bearer token (tenant-scoped). - -**Query Parameters:** - -| Parameter | Type | Required | Default | Constraints | -|---|---|---|---|---| -| `from` | string (YYYY-MM-DD) | no | 30 days ago | Must be <= `to` | -| `to` | string (YYYY-MM-DD) | no | today | Max range: 365 days | -| `granularity` | string | no | `day` | Enum: `day`, `week` | - -**Response 200** (`application/json`): -```json -{ - "tenantId": "string (UUID)", - "period": { - "from": "string (YYYY-MM-DD)", - "to": "string (YYYY-MM-DD)" - }, - "granularity": "day", - "dataPoints": [ - { - "date": "2026-03-01", - "totalIssuances": 420, - "successfulIssuances": 415, - "failedIssuances": 5, - "uniqueAgents": 8 - } - ] -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 400 | `INVALID_DATE_RANGE` | `from` > `to`, or date range exceeds 365 days | -| 400 | `INVALID_GRANULARITY` | `granularity` is not `day` or `week` | -| 401 | `UNAUTHORIZED` | Missing or invalid Bearer token | -| 403 | `ANALYTICS_NOT_AVAILABLE` | Free tier — requires Pro or Enterprise | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - ---- - -### Database Schema Changes - -#### Migration: `009_add_analytics_aggregates.sql` - -```sql -CREATE TABLE analytics_daily_aggregates ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE, - agent_id UUID REFERENCES agents(id) ON DELETE SET NULL, -- NULL = tenant-wide aggregate - date DATE NOT NULL, - metric_type VARCHAR(64) NOT NULL, -- 'api_calls' | 'token_issuances' | 'credential_rotations' | 'token_failures' - count BIGINT NOT NULL DEFAULT 0, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), - - CONSTRAINT uq_daily_aggregate UNIQUE (tenant_id, agent_id, date, metric_type) -); - --- Index for analytics queries (tenant + date range) -CREATE INDEX idx_analytics_tenant_date ON analytics_daily_aggregates(tenant_id, date); -CREATE INDEX idx_analytics_agent_date ON analytics_daily_aggregates(agent_id, date) WHERE agent_id IS NOT NULL; -``` - -#### Nightly Aggregation Job - -A `node-cron` job runs at `00:05 UTC` daily inside the Express API process. It executes an upsert query aggregating the previous day's `usage_events` rows into `analytics_daily_aggregates`. The job is idempotent — running it twice for the same date produces no duplicates (upsert on the unique constraint). - -Job logic (pseudocode): -``` -1. Compute target_date = yesterday (UTC) -2. SELECT tenant_id, agent_id, metric_type, SUM(count) - FROM usage_events - WHERE date = target_date - GROUP BY tenant_id, agent_id, metric_type -3. UPSERT INTO analytics_daily_aggregates - ON CONFLICT (tenant_id, agent_id, date, metric_type) - DO UPDATE SET count = EXCLUDED.count, updated_at = NOW() -``` - -### New Source Files - -| File | Description | -|---|---| -| `src/services/AnalyticsService.ts` | Business logic: query aggregates, build response shapes, Redis caching | -| `src/controllers/AnalyticsController.ts` | HTTP handlers for analytics endpoints | -| `src/routes/analytics.ts` | Express router for `/analytics/` prefix | -| `src/jobs/analyticsAggregation.ts` | `node-cron` job that aggregates usage_events nightly | -| `src/types/analytics.ts` | TypeScript interfaces: `UsageSummary`, `AgentActivityResponse`, `TokenTrendsResponse`, `DailyAggregate` | -| `dashboard/src/pages/Analytics.tsx` | New Analytics tab in existing React dashboard | -| `dashboard/src/components/charts/AgentHeatmap.tsx` | Heatmap component using `recharts` `ResponsiveContainer` + custom cells | -| `dashboard/src/components/charts/TokenTrendsChart.tsx` | Line chart of token issuance over time using `recharts` `LineChart` | -| `dashboard/src/components/charts/RotationFrequencyTable.tsx` | Sortable table of credential rotation counts per agent | -| `dashboard/src/api/analyticsApi.ts` | Typed fetch functions for analytics endpoints | - -### Modified Source Files - -| File | Change | -|---|---| -| `src/app.ts` | Register `analytics` router; start nightly aggregation cron job | -| `src/infrastructure/migrations/` | Add `009_add_analytics_aggregates.sql` | -| `dashboard/src/App.tsx` | Add Analytics route and nav link | -| `package.json` (API) | Add `node-cron` dependency | -| `package.json` (dashboard) | Add `recharts`, `date-fns` dependencies | -| `docs/openapi.yaml` | Add analytics endpoints | - -### Redis Caching - -Analytics responses are cached in Redis with `analytics:{tenantId}:{endpoint}:{queryHash}` keys. TTL: 5 minutes for agent-activity and token-trends; 60 seconds for usage-summary. Cache is invalidated on the next request after TTL expiry (no explicit invalidation). - -### `AnalyticsService` Interface - -```typescript -interface IAnalyticsService { - /** - * Return a high-level usage summary for a tenant over a date range. - */ - getUsageSummary(tenantId: string, from: Date, to: Date): Promise; - - /** - * Return per-agent daily activity data for heatmap rendering. - */ - getAgentActivity( - tenantId: string, - from: Date, - to: Date, - agentId?: string - ): Promise; - - /** - * Return daily token issuance trends with success/failure breakdown. - */ - getTokenTrends( - tenantId: string, - from: Date, - to: Date, - granularity: 'day' | 'week' - ): Promise; -} -``` - -### Prometheus Metrics - -| Metric | Type | Labels | Description | -|---|---|---|---| -| `agentidp_analytics_query_duration_ms` | Histogram | `endpoint` | Analytics query latency (before cache) | -| `agentidp_analytics_cache_hits_total` | Counter | `endpoint` | Analytics Redis cache hits | -| `agentidp_analytics_cache_misses_total` | Counter | `endpoint` | Analytics Redis cache misses | -| `agentidp_analytics_aggregation_job_duration_ms` | Gauge | — | Nightly aggregation job runtime | -| `agentidp_analytics_aggregation_job_last_run` | Gauge | — | Unix timestamp of last successful aggregation job run | - -### Feature Flags - -| Variable | Default | Description | -|---|---|---| -| `ANALYTICS_ENABLED` | `true` | When `false`, all `/analytics/` routes return HTTP 404 | -| `ANALYTICS_FREE_TIER` | `false` | When `true`, free tier tenants can access analytics (for beta/testing) | - -### Acceptance Criteria - -- `GET /analytics/usage-summary` returns correct aggregate counts for a date range -- `GET /analytics/agent-activity` returns per-agent daily rows matching `analytics_daily_aggregates` -- `GET /analytics/token-trends` returns daily and weekly granularity correctly -- All three endpoints return HTTP 403 for free-tier tenants (when `ANALYTICS_FREE_TIER=false`) -- Date range validation rejects `from > to` with HTTP 400 -- Nightly aggregation job runs idempotently — running twice for same date produces no duplicates -- Analytics responses are cached in Redis — a second identical request does not hit the DB -- Dashboard Analytics tab renders heatmap, trend chart, and rotation table with mock data in Storybook -- Unit test coverage >= 80% on `AnalyticsService` -- Integration tests cover: summary, activity, trends (daily), trends (weekly), free-tier rejection, invalid date range diff --git a/openspec/changes/phase-5-scale-ecosystem/specs/api-gateway-tiers/spec.md b/openspec/changes/phase-5-scale-ecosystem/specs/api-gateway-tiers/spec.md deleted file mode 100644 index e0ae60b..0000000 --- a/openspec/changes/phase-5-scale-ecosystem/specs/api-gateway-tiers/spec.md +++ /dev/null @@ -1,276 +0,0 @@ -## WS4: Public API Gateway & Rate Limiting SaaS - -### Purpose - -Replace the single flat rate limit (Phase 4) with a multi-tier enforcement model where each tenant's rate limits are determined by their subscription tier (`free` | `pro` | `enterprise`). Expose the tier definitions publicly via `GET /tiers` so developers can understand limits before registering. Add `POST /billing/upgrade` so tenants can self-service upgrade their tier without contacting support. - -This workstream closes the gap between Phase 4's flat rate limiter and a proper commercial SaaS gateway model. - -### New Endpoints - -#### `GET /tiers` - -**Summary:** Return the current tier definitions including rate limits, feature flags, and pricing. - -**Authentication:** None (public endpoint). - -**Response 200** (`application/json`): -```json -{ - "tiers": [ - { - "id": "free", - "name": "Free", - "price": { - "monthly": 0, - "currency": "USD" - }, - "limits": { - "registeredAgents": 10, - "apiCallsPerDay": 1000, - "tokenIssuancesPerDay": 200, - "rateLimitPerMinute": 60, - "rateLimitBurst": 10, - "auditLogRetentionDays": 30 - }, - "features": { - "marketplace": true, - "githubActions": true, - "analytics": false, - "webhooks": false, - "sso": false, - "sla": false, - "customDomain": false, - "prioritySupport": false - } - }, - { - "id": "pro", - "name": "Pro", - "price": { - "monthly": 49, - "currency": "USD" - }, - "limits": { - "registeredAgents": 100, - "apiCallsPerDay": 50000, - "tokenIssuancesPerDay": 10000, - "rateLimitPerMinute": 600, - "rateLimitBurst": 100, - "auditLogRetentionDays": 90 - }, - "features": { - "marketplace": true, - "githubActions": true, - "analytics": true, - "webhooks": true, - "sso": false, - "sla": false, - "customDomain": false, - "prioritySupport": false - } - }, - { - "id": "enterprise", - "name": "Enterprise", - "price": { - "monthly": null, - "currency": "USD", - "note": "Contact sales" - }, - "limits": { - "registeredAgents": null, - "apiCallsPerDay": null, - "tokenIssuancesPerDay": null, - "rateLimitPerMinute": 6000, - "rateLimitBurst": 1000, - "auditLogRetentionDays": 365 - }, - "features": { - "marketplace": true, - "githubActions": true, - "analytics": true, - "webhooks": true, - "sso": true, - "sla": true, - "customDomain": true, - "prioritySupport": true - } - } - ] -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 429 | `RATE_LIMITED` | Rate limit exceeded (even unauthenticated endpoints have a global IP-based limit) | - -**Notes:** -- `null` limits mean unlimited -- Tier definitions are sourced from a static configuration object in the codebase, not a database table -- The response is cached at the HTTP layer with `Cache-Control: public, max-age=3600` - ---- - -#### `POST /billing/upgrade` - -**Summary:** Initiate a self-service tier upgrade for the authenticated tenant. Creates a Stripe Checkout session for the target tier. - -**Authentication:** Bearer token (tenant-scoped). - -**Request Body** (`application/json`): -```json -{ - "targetTier": "pro" -} -``` - -| Field | Type | Required | Constraints | -|---|---|---|---| -| `targetTier` | string | yes | Enum: `pro`, `enterprise` — cannot downgrade via this endpoint | - -**Response 200** (`application/json`): -```json -{ - "checkoutUrl": "https://checkout.stripe.com/pay/cs_...", - "sessionId": "cs_...", - "targetTier": "pro", - "expiresAt": "string (ISO 8601)" -} -``` - -**Error Responses:** - -| Status | Code | Description | -|---|---|---| -| 400 | `ALREADY_ON_TIER` | Tenant is already subscribed to `targetTier` | -| 400 | `INVALID_TARGET_TIER` | `targetTier` is not a valid upgradeable tier | -| 400 | `DOWNGRADE_NOT_SUPPORTED` | `targetTier` is lower than the tenant's current tier | -| 401 | `UNAUTHORIZED` | Missing or invalid Bearer token | -| 422 | `STRIPE_ERROR` | Stripe API returned an error creating the Checkout session | -| 429 | `RATE_LIMITED` | Rate limit exceeded | - -**Business Rules:** -- This endpoint extends the existing `BillingService` — a new `upgradeTier(tenantId, targetTier)` method creates a Stripe Checkout session with the correct Stripe Price ID for the target tier -- The Stripe Price IDs per tier are configured via environment variables: `STRIPE_PRICE_ID_PRO`, `STRIPE_PRICE_ID_ENTERPRISE` -- After payment, Stripe sends `customer.subscription.created` webhook → existing webhook handler updates `tenant_subscriptions` -- The `TierRateLimiter` reads the updated tier from `tenant_subscriptions` within 60 seconds (Redis cache TTL for tier lookup) -- Downgrade is handled via the existing Stripe customer portal — not exposed as an API endpoint - ---- - -### `TierRateLimiter` Middleware - -This replaces the single `RateLimiterRedis` middleware for all authenticated routes. It reads the tenant's current tier, looks up the tier rate limit configuration, and enforces it using per-tenant Redis keys via `rate-limiter-flexible`. - -**Middleware behavior:** -1. Extract `tenantId` from the authenticated request context -2. Look up tier from Redis cache key `tier:{tenantId}` (TTL: 60 seconds) -3. On cache miss: query `tenant_subscriptions` for `tenantId`, cache result for 60s -4. Look up rate limit configuration for the tier from the static tier config -5. Apply `rate-limiter-flexible` with key `rl:{tier}:{tenantId}` and tier-specific limits -6. On rate limit exceeded: return HTTP 429 with headers: - - `X-RateLimit-Limit: ` - - `X-RateLimit-Remaining: ` - - `X-RateLimit-Reset: ` - - `Retry-After: ` -7. Increment `agentidp_rate_limit_hits_total` counter (labels: `tier`, `tenant_id`, `endpoint`) - -**Unauthenticated routes:** Continue to use the existing flat `RateLimiterRedis` with IP-based keys (unchanged from Phase 4). - -### Tier Configuration Object - -Centralized in `src/config/tiers.ts` — this is the single source of truth for all tier limits and features. Both `GET /tiers` and `TierRateLimiter` read from this same object. - -```typescript -export const TIER_CONFIG: Record = { - free: { - id: 'free', - limits: { - registeredAgents: 10, - apiCallsPerDay: 1000, - tokenIssuancesPerDay: 200, - rateLimitPerMinute: 60, - rateLimitBurst: 10, - auditLogRetentionDays: 30, - }, - features: { analytics: false, webhooks: false, sso: false, sla: false }, - stripeProductId: null, - }, - pro: { - id: 'pro', - limits: { - registeredAgents: 100, - apiCallsPerDay: 50000, - tokenIssuancesPerDay: 10000, - rateLimitPerMinute: 600, - rateLimitBurst: 100, - auditLogRetentionDays: 90, - }, - features: { analytics: true, webhooks: true, sso: false, sla: false }, - stripeProductId: process.env.STRIPE_PRICE_ID_PRO ?? '', - }, - enterprise: { - id: 'enterprise', - limits: { - registeredAgents: null, - apiCallsPerDay: null, - tokenIssuancesPerDay: null, - rateLimitPerMinute: 6000, - rateLimitBurst: 1000, - auditLogRetentionDays: 365, - }, - features: { analytics: true, webhooks: true, sso: true, sla: true }, - stripeProductId: process.env.STRIPE_PRICE_ID_ENTERPRISE ?? '', - }, -}; -``` - -### New Source Files - -| File | Description | -|---|---| -| `src/config/tiers.ts` | Static tier configuration — single source of truth for limits and features | -| `src/middleware/tierRateLimiter.ts` | `TierRateLimiter` middleware — reads tenant tier, enforces tier-specific limits | -| `src/routes/tiers.ts` | Express router for `GET /tiers` | -| `src/types/tiers.ts` | TypeScript interfaces: `TierDefinition`, `TierName`, `TierLimits`, `TierFeatures` | - -### Modified Source Files - -| File | Change | -|---|---| -| `src/middleware/rateLimiter.ts` | Retain for unauthenticated routes; authenticated routes switch to `tierRateLimiter` | -| `src/services/BillingService.ts` | Add `upgradeTier(tenantId, targetTier)` method | -| `src/controllers/BillingController.ts` | Add handler for `POST /billing/upgrade` | -| `src/routes/billing.ts` | Register `POST /billing/upgrade` route | -| `src/routes/index.ts` | Register `tiers` router | -| `.env.example` | Add `STRIPE_PRICE_ID_PRO`, `STRIPE_PRICE_ID_ENTERPRISE`, `TIER_RATE_LIMITING_ENABLED` | -| `docs/openapi.yaml` | Add `GET /tiers` and `POST /billing/upgrade` endpoints | - -### Prometheus Metrics - -| Metric | Type | Labels | Description | -|---|---|---|---| -| `agentidp_rate_limit_hits_total` | Counter | `tier`, `tenant_id`, `endpoint` | Rate limit rejections per tier (replaces old flat counter) | -| `agentidp_tier_cache_hits_total` | Counter | — | Tier Redis cache hits | -| `agentidp_tier_cache_misses_total` | Counter | — | Tier Redis cache misses | -| `agentidp_billing_upgrades_total` | Counter | `from_tier`, `to_tier` | Self-service upgrade checkout sessions created | - -### Feature Flag - -`TIER_RATE_LIMITING_ENABLED` (default: `true`). When `false`, the system uses the old flat `RateLimiterRedis` middleware — this is the rollback mechanism. - -### Acceptance Criteria - -- `GET /tiers` returns all three tier definitions matching `TIER_CONFIG` exactly — no database query, cached `Cache-Control: max-age=3600` -- `POST /billing/upgrade` creates a Stripe Checkout session and returns `checkoutUrl` -- `POST /billing/upgrade` returns HTTP 400 `ALREADY_ON_TIER` when tenant is already on the target tier -- `POST /billing/upgrade` returns HTTP 400 `DOWNGRADE_NOT_SUPPORTED` when target tier is lower than current -- `TierRateLimiter` enforces free tier limits (60 req/min) for free tenants -- `TierRateLimiter` enforces pro tier limits (600 req/min) for pro tenants -- Tier lookup is cached in Redis — second request does not query `tenant_subscriptions` -- Rate limit response includes `X-RateLimit-*` headers and `Retry-After` -- After a Stripe webhook updates `tenant_subscriptions` to `pro`, `TierRateLimiter` applies pro limits within 60 seconds (next cache refresh) -- Unit tests cover: tier lookup (cached), tier lookup (miss), free limit enforcement, pro limit enforcement, upgrade (success), upgrade (already on tier), upgrade (downgrade rejected) diff --git a/openspec/changes/phase-5-scale-ecosystem/tasks.md b/openspec/changes/phase-5-scale-ecosystem/tasks.md index 91000cf..4373f81 100644 --- a/openspec/changes/phase-5-scale-ecosystem/tasks.md +++ b/openspec/changes/phase-5-scale-ecosystem/tasks.md @@ -56,120 +56,45 @@ - [ ] 7.7 Write unit tests for `DelegationService` — mock DB and audit service; test: create delegation (valid), create delegation (scope escalation rejected), create delegation (self-delegation rejected), create delegation (delegatee in different tenant rejected), verify delegation (valid), verify delegation (expired — returns valid: false not throw), verify delegation (revoked — returns valid: false), revoke delegation (by delegator — succeeds), revoke delegation (by non-delegator — throws ForbiddenError), revoke delegation (already revoked — throws ConflictError) - [ ] 7.8 Write integration tests for delegation endpoints — test all happy paths and all error cases defined in spec; verify audit log entries are created for each delegation operation -## 8. WS3: Analytics — Database, Aggregation Job +## 8. WS5: Developer Experience — Scaffold Service -- [ ] 8.1 Create `src/infrastructure/migrations/009_add_analytics_aggregates.sql` — create `analytics_daily_aggregates` table with columns: `id` (UUID PK), `tenant_id` (UUID FK), `agent_id` (UUID nullable FK), `date` (DATE), `metric_type` (VARCHAR 64), `count` (BIGINT), `created_at`, `updated_at`; add unique constraint on `(tenant_id, agent_id, date, metric_type)`; create indexes on `(tenant_id, date)` and `(agent_id, date) WHERE agent_id IS NOT NULL` -- [ ] 8.2 Install `node-cron` npm package — add to `package.json` -- [ ] 8.3 Create `src/jobs/analyticsAggregation.ts` — implement `runAnalyticsAggregation(targetDate: Date): Promise`: execute upsert query aggregating previous day's `usage_events` rows into `analytics_daily_aggregates`; query is idempotent (upsert on unique constraint); update `agentidp_analytics_aggregation_job_duration_ms` gauge and `agentidp_analytics_aggregation_job_last_run` gauge on completion -- [ ] 8.4 Register cron job in `src/app.ts` — schedule `runAnalyticsAggregation` at `00:05 UTC` daily using `node-cron`; log job start, completion, and any errors; do not crash the process on job failure — log error and continue +- [ ] 8.1 Install `archiver` and `@types/archiver` in API `package.json` +- [ ] 8.2 Create `src/types/scaffold.ts` — define `ScaffoldLanguage` union (`'typescript' | 'python' | 'go' | 'java' | 'rust'`), `ScaffoldOptions` interface, `ScaffoldTemplate` interface +- [ ] 8.3 Create scaffold template files for TypeScript in `src/templates/scaffold/typescript/`: `package.json.tmpl`, `tsconfig.json.tmpl`, `src/index.ts.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` — each file uses `{{AGENT_ID}}`, `{{AGENT_NAME}}`, `{{CLIENT_ID}}`, `{{API_URL}}` as template variables; `.env.example.tmpl` MUST include `AGENTIDP_CLIENT_SECRET=` placeholder (never inject real secret) +- [ ] 8.4 Create scaffold template files for Python in `src/templates/scaffold/python/`: `requirements.txt.tmpl`, `main.py.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` — same template variable convention +- [ ] 8.5 Create scaffold template files for Go in `src/templates/scaffold/go/`: `go.mod.tmpl`, `main.go.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` +- [ ] 8.6 Create scaffold template files for Java in `src/templates/scaffold/java/`: `pom.xml.tmpl`, `src/main/java/Main.java.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` +- [ ] 8.7 Create scaffold template files for Rust in `src/templates/scaffold/rust/`: `Cargo.toml.tmpl`, `src/main.rs.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` +- [ ] 8.8 Create `src/services/ScaffoldService.ts` — implement `IScaffoldService`; `generateScaffold(agentId, language, apiUrl)`: load template files for language, inject template variables (replace `{{AGENT_ID}}`, `{{AGENT_NAME}}`, `{{CLIENT_ID}}`, `{{API_URL}}`), build in-memory ZIP using `archiver`; return `{ stream: NodeJS.ReadableStream, filename: string }`; emit `agentidp_scaffold_generated_total` counter and `agentidp_scaffold_generation_duration_ms` histogram -## 9. WS3: Analytics — Service, Controller, Routes +## 9. WS5: Developer Experience — Scaffold Controller & Route -- [ ] 9.1 Create `src/types/analytics.ts` — define interfaces: `UsageSummary`, `AgentActivityResponse`, `TokenTrendsResponse`, `DailyAggregate`, `AnalyticsDateRange` -- [ ] 9.2 Create `src/services/AnalyticsService.ts` — implement `IAnalyticsService`; `getUsageSummary`: validate date range (from <= to, max 365 days), check Redis cache (`analytics:{tenantId}:summary:{hash}`, TTL 60s), on miss query `analytics_daily_aggregates`, compute totals, write to cache, return `UsageSummary` -- [ ] 9.3 Implement `AnalyticsService.getAgentActivity(tenantId, from, to, agentId?)` — validate date range (max 90 days), check Redis cache (TTL 5 min), on miss query `analytics_daily_aggregates` grouped by `agent_id` and `date`, join agent names from `agents` table, write to cache, return `AgentActivityResponse` -- [ ] 9.4 Implement `AnalyticsService.getTokenTrends(tenantId, from, to, granularity)` — support `day` and `week` granularity (weekly: `date_trunc('week', date)`), check Redis cache (TTL 5 min), return `TokenTrendsResponse` with `successfulIssuances`, `failedIssuances`, `uniqueAgents` per period -- [ ] 9.5 Create `src/controllers/AnalyticsController.ts` — handlers for `getUsageSummary`, `getAgentActivity`, `getTokenTrends`; parse and validate query parameters; return HTTP 403 for free-tier tenants (check `ANALYTICS_FREE_TIER` env and tenant subscription); emit `agentidp_analytics_query_duration_ms` histogram and cache hit/miss counters -- [ ] 9.6 Create `src/routes/analytics.ts` — Express router for `/analytics/usage-summary`, `/analytics/agent-activity`, `/analytics/token-trends`; all routes require authentication middleware -- [ ] 9.7 Register analytics router in `src/routes/index.ts` behind `ANALYTICS_ENABLED` feature flag -- [ ] 9.8 Add analytics endpoints to `docs/openapi.yaml` — all query parameters, response schemas, and error codes as defined in spec -- [ ] 9.9 Write unit tests for `AnalyticsService` — test: usage-summary (cache hit), usage-summary (cache miss → DB query), agent-activity (with agentId filter), agent-activity (no filter — all agents), token-trends (daily), token-trends (weekly), date range validation (from > to rejected), date range validation (> max days rejected), free-tier rejection -- [ ] 9.10 Write integration tests for analytics endpoints — test all three endpoints with valid date ranges, verify free-tier rejection, verify invalid date range errors +- [ ] 9.1 Create `src/controllers/ScaffoldController.ts` — implement `getScaffold` handler for `GET /sdk/scaffold/:agentId`: validate `language` query param against `ScaffoldLanguage` union (HTTP 400 on invalid); fetch agent, verify agent belongs to authenticated tenant (HTTP 403 if not); call `ScaffoldService.generateScaffold`; set `Content-Type: application/zip`, `Content-Disposition: attachment; filename="..."`, pipe stream to response; write audit log entry (`scaffold.generated`, metadata: `{ language }`) +- [ ] 9.2 Create `src/routes/scaffold.ts` — Express router for `GET /sdk/scaffold/:agentId` with authentication middleware; apply scaffold-specific rate limiter (10 req/min per tenant, separate from global rate limiter) +- [ ] 9.3 Register `scaffold` router in `src/routes/index.ts` +- [ ] 9.4 Add `GET /sdk/scaffold/:agentId` to `docs/openapi.yaml` — document binary response type, query parameters, all error responses +- [ ] 9.5 Write unit tests for `ScaffoldService` — test: generate TypeScript scaffold (verify ZIP contains all 6 files), generate Python scaffold (verify all 5 files), verify `{{CLIENT_ID}}` is replaced in `.env.example`, verify `{{AGENTIDP_CLIENT_SECRET}}` is placeholder not real secret, verify invalid language throws `ValidationError` +- [ ] 9.6 Write integration tests for scaffold endpoint — test: TypeScript scaffold returns ZIP with correct `Content-Type` and `Content-Disposition`; Python scaffold returns ZIP; HTTP 400 on invalid language; HTTP 403 when agent belongs to different tenant; HTTP 404 when agent does not exist -## 10. WS3: Analytics — Dashboard UI +## 10. WS5: Developer Experience — Portal & CLI -- [ ] 10.1 Install `recharts` and `date-fns` in `dashboard/package.json` -- [ ] 10.2 Create `dashboard/src/api/analyticsApi.ts` — typed fetch functions for all three analytics endpoints: `fetchUsageSummary(token, from, to)`, `fetchAgentActivity(token, from, to, agentId?)`, `fetchTokenTrends(token, from, to, granularity)`; all functions return typed response objects; handle 403 response with a typed `AnalyticsNotAvailableError` -- [ ] 10.3 Create `dashboard/src/components/charts/AgentHeatmap.tsx` — renders a grid heatmap (agents × dates) using `recharts` or a custom CSS grid; color intensity represents `apiCalls` count; hover tooltip shows agent name, date, apiCalls, tokenIssuances, credentialRotations; accepts `agents` prop from `AgentActivityResponse` -- [ ] 10.4 Create `dashboard/src/components/charts/TokenTrendsChart.tsx` — renders a `recharts` `ComposedChart` with a `Line` for `successfulIssuances` and a `Bar` for `failedIssuances`; X-axis is dates; tooltip shows all three metrics per period; accepts `dataPoints` prop from `TokenTrendsResponse` -- [ ] 10.5 Create `dashboard/src/components/charts/RotationFrequencyTable.tsx` — renders a sortable table of credential rotation counts per agent; columns: Agent Name, Rotations (period), Last Rotation Date; sortable by any column; accepts `agents` prop derived from `AgentActivityResponse` filtering `credentialRotations` -- [ ] 10.6 Create `dashboard/src/pages/Analytics.tsx` — analytics tab page; renders date range picker (from/to), calls all three analytics APIs, renders `AgentHeatmap`, `TokenTrendsChart`, `RotationFrequencyTable`; shows a `UpgradeRequired` component when API returns 403 -- [ ] 10.7 Add Analytics route to `dashboard/src/App.tsx` — add `/analytics` route; add "Analytics" link to dashboard navigation -- [ ] 10.8 Run `npm run build` in `dashboard/` — zero TypeScript errors, zero ESLint errors +- [ ] 10.1 Install `@stoplight/elements` in `portal/package.json` — remove `swagger-ui-react` +- [ ] 10.2 Rewrite `portal/app/api-explorer/page.tsx` — replace `SwaggerUI` component with `@stoplight/elements` `` component; set `apiDescriptionUrl`, `router="hash"`, `layout="sidebar"`, `hideSchemas={false}`, `tryItCredentialsPolicy="same-origin"`; import Elements CSS; remove all Swagger UI imports and CSS +- [ ] 10.3 Run `npm run build` in `portal/` — verify zero TypeScript errors and zero ESLint errors after Elements integration +- [ ] 10.4 Install `unzipper` and `@types/unzipper` in `cli/package.json` +- [ ] 10.5 Create `cli/src/commands/scaffold.ts` — implement `sentryagent scaffold` command with Commander options: `--agent-id ` (required), `--language ` (default: typescript), `--out ` (default: `.`); load config, issue Bearer token, call `GET /sdk/scaffold/{agentId}?language={language}`, pipe response through `unzipper.Extract({ path: outDir })`, print success message and next steps; handle errors (404, 403, 400) with human-readable messages +- [ ] 10.6 Register `scaffold` command in `cli/src/index.ts` — add `.addCommand(scaffoldCommand)` to Commander program +- [ ] 10.7 Run `npm run build` in `cli/` — zero TypeScript errors; run `node dist/index.js scaffold --help` — outputs correct usage -## 11. WS4: API Gateway Tiers — Configuration & Middleware +## 11. QA & Release -- [ ] 11.1 Create `src/types/tiers.ts` — define interfaces: `TierName` (union: `'free' | 'pro' | 'enterprise'`), `TierLimits`, `TierFeatures`, `TierDefinition` (includes `id`, `limits`, `features`, `stripeProductId`) -- [ ] 11.2 Create `src/config/tiers.ts` — define `TIER_CONFIG: Record` with complete limit and feature definitions for `free`, `pro`, and `enterprise` tiers as specified in spec; export `getTierConfig(tier: TierName): TierDefinition` helper -- [ ] 11.3 Create `src/middleware/tierRateLimiter.ts` — implement `TierRateLimiter` middleware: extract `tenantId` from authenticated request context; check Redis key `tier:{tenantId}` (TTL 60s) for cached tier; on miss query `tenant_subscriptions` for tenant's current tier, cache for 60s; look up rate limit config from `TIER_CONFIG`; apply `RateLimiterRedis` with key `rl:{tier}:{tenantId}`; on rejection return HTTP 429 with `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `Retry-After` headers; increment `agentidp_rate_limit_hits_total` counter with `tier` and `tenant_id` labels -- [ ] 11.4 Replace `RateLimiterRedis` middleware on all authenticated routes in `src/routes/index.ts` with `TierRateLimiter`; keep the flat IP-based `RateLimiterRedis` on unauthenticated routes unchanged; wrap replacement in `TIER_RATE_LIMITING_ENABLED` feature flag (fall back to old middleware when `false`) - -## 12. WS4: API Gateway Tiers — Endpoints - -- [ ] 12.1 Create `src/routes/tiers.ts` — Express router for `GET /tiers`; handler reads `TIER_CONFIG`, formats response as specified in spec, sets `Cache-Control: public, max-age=3600` header; no database query; no authentication required -- [ ] 12.2 Register `tiers` router in `src/routes/index.ts` -- [ ] 12.3 Implement `BillingService.upgradeTier(tenantId: string, targetTier: 'pro' | 'enterprise'): Promise<{ checkoutUrl: string; sessionId: string; expiresAt: string }>` — fetch current tier from `tenant_subscriptions`, validate no self-upgrade or downgrade, create Stripe Checkout session with `STRIPE_PRICE_ID_PRO` or `STRIPE_PRICE_ID_ENTERPRISE`, return checkout URL -- [ ] 12.4 Add `upgradeTier` handler to `src/controllers/BillingController.ts` — validate `targetTier` enum, call `BillingService.upgradeTier`, return HTTP 200 with `checkoutUrl`, `sessionId`, `targetTier`, `expiresAt` -- [ ] 12.5 Register `POST /billing/upgrade` route in `src/routes/billing.ts` with authentication middleware -- [ ] 12.6 Add `STRIPE_PRICE_ID_PRO`, `STRIPE_PRICE_ID_ENTERPRISE`, `TIER_RATE_LIMITING_ENABLED` to `.env.example` with documentation comments -- [ ] 12.7 Add `GET /tiers` and `POST /billing/upgrade` to `docs/openapi.yaml` -- [ ] 12.8 Write unit tests for `TierRateLimiter` — test: free tier limit enforced (60 req/min), pro tier limit enforced (600 req/min), tier looked up from Redis cache (DB not called), tier fetched from DB on cache miss, rollback path (`TIER_RATE_LIMITING_ENABLED=false` uses old flat limiter) -- [ ] 12.9 Write unit tests for `BillingService.upgradeTier` — test: upgrade free → pro (creates Stripe session), upgrade free → enterprise (creates Stripe session), already on pro (returns ALREADY_ON_TIER error), downgrade attempt (returns DOWNGRADE_NOT_SUPPORTED error) -- [ ] 12.10 Write integration tests for `GET /tiers` — verify response structure, verify `Cache-Control` header, verify no auth required; write integration tests for `POST /billing/upgrade` — mock Stripe, verify checkout URL returned - -## 13. WS5: Developer Experience — Scaffold Service - -- [ ] 13.1 Install `archiver` and `@types/archiver` in API `package.json` -- [ ] 13.2 Create `src/types/scaffold.ts` — define `ScaffoldLanguage` union (`'typescript' | 'python' | 'go' | 'java' | 'rust'`), `ScaffoldOptions` interface, `ScaffoldTemplate` interface -- [ ] 13.3 Create scaffold template files for TypeScript in `src/templates/scaffold/typescript/`: `package.json.tmpl`, `tsconfig.json.tmpl`, `src/index.ts.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` — each file uses `{{AGENT_ID}}`, `{{AGENT_NAME}}`, `{{CLIENT_ID}}`, `{{API_URL}}` as template variables; `.env.example.tmpl` MUST include `AGENTIDP_CLIENT_SECRET=` placeholder (never inject real secret) -- [ ] 13.4 Create scaffold template files for Python in `src/templates/scaffold/python/`: `requirements.txt.tmpl`, `main.py.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` — same template variable convention -- [ ] 13.5 Create scaffold template files for Go in `src/templates/scaffold/go/`: `go.mod.tmpl`, `main.go.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` -- [ ] 13.6 Create scaffold template files for Java in `src/templates/scaffold/java/`: `pom.xml.tmpl`, `src/main/java/Main.java.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` -- [ ] 13.7 Create scaffold template files for Rust in `src/templates/scaffold/rust/`: `Cargo.toml.tmpl`, `src/main.rs.tmpl`, `.env.example.tmpl`, `.gitignore.tmpl`, `README.md.tmpl` -- [ ] 13.8 Create `src/services/ScaffoldService.ts` — implement `IScaffoldService`; `generateScaffold(agentId, language, apiUrl)`: load template files for language, inject template variables (replace `{{AGENT_ID}}`, `{{AGENT_NAME}}`, `{{CLIENT_ID}}`, `{{API_URL}}`), build in-memory ZIP using `archiver`; return `{ stream: NodeJS.ReadableStream, filename: string }`; emit `agentidp_scaffold_generated_total` counter and `agentidp_scaffold_generation_duration_ms` histogram - -## 14. WS5: Developer Experience — Scaffold Controller & Route - -- [ ] 14.1 Create `src/controllers/ScaffoldController.ts` — implement `getScaffold` handler for `GET /sdk/scaffold/:agentId`: validate `language` query param against `ScaffoldLanguage` union (HTTP 400 on invalid); fetch agent, verify agent belongs to authenticated tenant (HTTP 403 if not); call `ScaffoldService.generateScaffold`; set `Content-Type: application/zip`, `Content-Disposition: attachment; filename="..."`, pipe stream to response; write audit log entry (`scaffold.generated`, metadata: `{ language }`) -- [ ] 14.2 Create `src/routes/scaffold.ts` — Express router for `GET /sdk/scaffold/:agentId` with authentication middleware; apply scaffold-specific rate limiter (10 req/min per tenant, separate from `TierRateLimiter`) -- [ ] 14.3 Register `scaffold` router in `src/routes/index.ts` -- [ ] 14.4 Add `GET /sdk/scaffold/:agentId` to `docs/openapi.yaml` — document binary response type, query parameters, all error responses -- [ ] 14.5 Write unit tests for `ScaffoldService` — test: generate TypeScript scaffold (verify ZIP contains all 6 files), generate Python scaffold (verify all 5 files), verify `{{CLIENT_ID}}` is replaced in `.env.example`, verify `{{AGENTIDP_CLIENT_SECRET}}` is placeholder not real secret, verify invalid language throws `ValidationError` -- [ ] 14.6 Write integration tests for scaffold endpoint — test: TypeScript scaffold returns ZIP with correct `Content-Type` and `Content-Disposition`; Python scaffold returns ZIP; HTTP 400 on invalid language; HTTP 403 when agent belongs to different tenant; HTTP 404 when agent does not exist - -## 15. WS5: Developer Experience — Portal & CLI - -- [ ] 15.1 Install `@stoplight/elements` in `portal/package.json` — remove `swagger-ui-react` -- [ ] 15.2 Rewrite `portal/app/api-explorer/page.tsx` — replace `SwaggerUI` component with `@stoplight/elements` `` component; set `apiDescriptionUrl`, `router="hash"`, `layout="sidebar"`, `hideSchemas={false}`, `tryItCredentialsPolicy="same-origin"`; import Elements CSS; remove all Swagger UI imports and CSS -- [ ] 15.3 Run `npm run build` in `portal/` — verify zero TypeScript errors and zero ESLint errors after Elements integration -- [ ] 15.4 Install `unzipper` and `@types/unzipper` in `cli/package.json` -- [ ] 15.5 Create `cli/src/commands/scaffold.ts` — implement `sentryagent scaffold` command with Commander options: `--agent-id ` (required), `--language ` (default: typescript), `--out ` (default: `.`); load config, issue Bearer token, call `GET /sdk/scaffold/{agentId}?language={language}`, pipe response through `unzipper.Extract({ path: outDir })`, print success message and next steps; handle errors (404, 403, 400) with human-readable messages -- [ ] 15.6 Register `scaffold` command in `cli/src/index.ts` — add `.addCommand(scaffoldCommand)` to Commander program -- [ ] 15.7 Run `npm run build` in `cli/` — zero TypeScript errors; run `node dist/index.js scaffold --help` — outputs correct usage - -## 16. WS6: AGNTCY Compliance — Compliance Service - -- [ ] 16.1 Create `src/types/compliance.ts` — define interfaces: `ComplianceRequirement` (id, description, status, evidence, verifiedAt), `ComplianceSection` (id, name, status, requirements), `ComplianceReport` (reportId, generatedAt, agntcySpecVersion, tenantId, overallStatus, sections, summary), `AgentCard` (agntcyVersion, type, agent, issuedAt, expiresAt) -- [ ] 16.2 Create `src/config/agntcyRequirements.ts` — define the complete array of AGNTCY requirement objects (minimum 24 requirements), each with: `id` (e.g., `AI-001`), `description` (from AGNTCY spec), `section` (e.g., `agent-identity`), and `evaluate(tenantId: string, db: Pool): Promise` function — each evaluator queries the live system and returns `{ status, evidence }` -- [ ] 16.3 Create `src/services/ComplianceService.ts` — implement `IComplianceService`; `generateComplianceReport(tenantId)`: run all requirement evaluators from `agntcyRequirements.ts` in parallel, group results by section, compute overall status (`compliant` if all pass, `partial` if any non-compliant, `non-compliant` if >20% fail), build `ComplianceReport`, write audit log entry (`compliance.report_generated`), emit `agentidp_compliance_reports_generated_total` counter and `agentidp_compliance_report_duration_ms` histogram -- [ ] 16.4 Implement `ComplianceService.generateAgentCard(agentId)` — fetch agent from DB, build `AgentCard` per AGNTCY spec format, set `expiresAt = issuedAt + 24 hours`, set `complianceStandards` from system config, emit `agentidp_agent_cards_served_total` counter with `visibility` label - -## 17. WS6: AGNTCY Compliance — Controller, Routes - -- [ ] 17.1 Create `src/controllers/ComplianceController.ts` — implement `getComplianceReport` handler: check tenant tier is pro or enterprise (HTTP 403 `TIER_REQUIRED` for free tier), call `ComplianceService.generateComplianceReport`, return HTTP 200; implement `getAgentCard` handler: check agent visibility (HTTP 401 if private and unauthenticated, HTTP 403 if private and wrong tenant), call `ComplianceService.generateAgentCard`, return HTTP 200 -- [ ] 17.2 Create `src/routes/agntcy.ts` — Express router for `GET /agntcy/compliance-report` (requires auth) and `GET /agents/:id/agent-card` (auth optional); register behind `AGNTCY_ENABLED` feature flag -- [ ] 17.3 Register `agntcy` router in `src/routes/index.ts` -- [ ] 17.4 Add `GET /agntcy/compliance-report` and `GET /agents/:id/agent-card` to `docs/openapi.yaml` -- [ ] 17.5 Write unit tests for `ComplianceService` — test: `generateComplianceReport` (all 24 requirements pass → `compliant`), `generateComplianceReport` (one evaluator fails → `partial`), `generateAgentCard` (public agent), `generateAgentCard` (private agent — verify agent data is included), `generateAgentCard` (non-existent agent → throws NotFoundError) -- [ ] 17.6 Write integration tests for compliance endpoints — test: compliance report for pro tenant (HTTP 200, overallStatus), compliance report for free tenant (HTTP 403), agent card for public agent (no auth required), agent card for private agent (auth required, correct tenant succeeds, wrong tenant HTTP 403) - -## 18. WS6: AGNTCY Compliance — Interoperability Tests & Docs - -- [ ] 18.1 Create `tests/agntcy/interoperability.test.ts` — implement all 25+ AGNTCY interoperability test cases as defined in spec: AI-001 (agent UUID uniqueness), AI-002 (W3C DID document), AUTH-001 (OAuth 2.0 token issuance), AUTH-002 (OIDC discovery), AUTHZ-001 (scope enforcement), DEL-001 through DEL-004 (delegation chain), AUDIT-001 through AUDIT-002 (immutable audit log), GOV-001 through GOV-002 (lifecycle governance), INTER-001 (agent card), COMP-001 (compliance report) -- [ ] 18.2 Add `"test:agntcy": "jest --testPathPattern=tests/agntcy --forceExit"` script to `package.json` -- [ ] 18.3 Write `docs/agntcy/certification-guide.md` — complete document with all 6 sections: Overview, Requirement Mapping table, Running the Compliance Report (step-by-step), Agent Card Usage, Self-Certification Checklist, Submitting for Official AGNTCY Certification; no placeholders, no TODOs - -## 19. QA & Release - -- [ ] 19.1 Run `cargo build` and `cargo clippy -- -D warnings` in `sdk-rust/` — zero warnings; run `cargo test` — all unit tests pass -- [ ] 19.2 Run `tsc --noEmit` across API, dashboard, portal, and CLI — zero TypeScript errors -- [ ] 19.3 Run full Jest suite (`npm test`) — all unit tests pass, coverage >= 80% across all new services: `DelegationService`, `AnalyticsService`, `ScaffoldService`, `ComplianceService`, `TierRateLimiter` -- [ ] 19.4 Run `npm run build` in `portal/` with Elements integration — zero errors; verify `/api-explorer` page renders Elements `` component -- [ ] 19.5 Run `npm run build` in `cli/` — zero errors; run `node dist/index.js scaffold --help` — shows correct options; run `node dist/index.js --help` — shows `scaffold` command listed -- [ ] 19.6 Apply database migrations `008_add_delegation_chains.sql` and `009_add_analytics_aggregates.sql` against a test database — verify migrations run without errors and tables are created with correct schemas -- [ ] 19.7 Run integration tests for all Phase 5 endpoints — delegation (create, verify, revoke), analytics (usage-summary, agent-activity, token-trends), tiers (GET /tiers, POST /billing/upgrade), scaffold (all 5 languages), AGNTCY (compliance-report, agent-card) -- [ ] 19.8 Run `npm run test:agntcy` — all 25+ interoperability test cases pass -- [ ] 19.9 Verify feature flags: `A2A_ENABLED=false` → delegation routes return 404; `ANALYTICS_ENABLED=false` → analytics routes return 404; `TIER_RATE_LIMITING_ENABLED=false` → flat rate limiter used; `AGNTCY_ENABLED=false` → AGNTCY routes return 404 -- [ ] 19.10 Verify tier rate limiting: free tenant receives 429 at 61st request/minute; pro tenant allows 600 requests/minute; tier cache refresh within 60s after Stripe webhook updates subscription -- [ ] 19.11 Verify scaffold security: `GET /sdk/scaffold/:agentId` response ZIP never contains a real `client_secret` value — `.env.example` placeholder only -- [ ] 19.12 Commit all Phase 5 work on `main` — one conventional commit per workstream (e.g., `feat(phase-5): WS1 — Rust SDK`, `feat(phase-5): WS2 — A2A Authorization`, etc.) +- [ ] 11.1 Run `cargo build` and `cargo clippy -- -D warnings` in `sdk-rust/` — zero warnings; run `cargo test` — all unit tests pass +- [ ] 11.2 Run `tsc --noEmit` across API, portal, and CLI — zero TypeScript errors +- [ ] 11.3 Run full Jest suite (`npm test`) — all unit tests pass, coverage >= 80% across all new services: `DelegationService`, `ScaffoldService` +- [ ] 11.4 Run `npm run build` in `portal/` with Elements integration — zero errors; verify `/api-explorer` page renders Elements `` component +- [ ] 11.5 Run `npm run build` in `cli/` — zero errors; run `node dist/index.js scaffold --help` — shows correct options; run `node dist/index.js --help` — shows `scaffold` command listed +- [ ] 11.6 Apply database migration `008_add_delegation_chains.sql` against a test database — verify migration runs without errors and table is created with correct schema +- [ ] 11.7 Run integration tests for all Phase 5 endpoints — delegation (create, verify, revoke), scaffold (all 5 languages) +- [ ] 11.8 Verify feature flag: `A2A_ENABLED=false` → delegation routes return 404 +- [ ] 11.9 Verify scaffold security: `GET /sdk/scaffold/:agentId` response ZIP never contains a real `client_secret` value — `.env.example` placeholder only +- [ ] 11.10 Commit all Phase 5 work on `main` — one conventional commit per workstream: `feat(phase-5): WS1 — Rust SDK`, `feat(phase-5): WS2 — A2A Authorization`, `feat(phase-5): WS5 — Developer Experience`