sentryagent-idp/docs/engineering/03-tech-stack.md

Technology Stack and Architecture Decision Records

Every technology choice in AgentIdP was made deliberately. This document records the decision, rationale, and alternatives considered for each major technology. New engineers should read this before making any technology additions or changes — the pattern here is the template for future ADRs.

---

ADR-1: Node.js 18 LTS

Status: Adopted Component: AgentIdP server runtime and Node.js SDK runtime

Decision: Use Node.js 18 LTS as the server runtime.

Rationale: Node.js 18 LTS provides native fetch, native ESM support, and a stable V8 engine with long-term security updates. The ecosystem for Express, PostgreSQL (pg), Redis (redis), JWT (jsonwebtoken), and bcrypt (bcryptjs) is mature and well-maintained on this version. The non-blocking I/O model is well-suited for an IdP that handles many concurrent short-lived authentication requests. The engines.node field in package.json enforces >=18.0.0.

Alternatives considered:

• Deno — rejected because the npm ecosystem compatibility layer introduced friction with key dependencies (pg, bcryptjs), and the production deployment story on ECS and Cloud Run was less mature at the time of the decision.
• Bun — rejected because it lacked LTS stability guarantees at the time of the decision, which is not acceptable for a security-critical authentication service.

Consequences: All Dockerfiles and Terraform ECS/Cloud Run task definitions must target Node.js 18 or a compatible LTS release. Upgrading the Node.js version requires CTO approval and a QA sign-off on the full test suite.

---

ADR-2: TypeScript 5.3 Strict Mode

Status: Adopted Component: All source files — server, all SDKs, dashboard

Decision: TypeScript 5.3 with strict: true and every additional strictness flag enabled in tsconfig.json.

Rationale: AgentIdP handles authentication tokens and cryptographic secrets. Type errors in this domain can cause security vulnerabilities — a value that should be string | null treated as string can produce silent authentication bypasses. Strict TypeScript with noImplicitAny, strictNullChecks, noUnusedLocals, noUnusedParameters, and noImplicitReturns makes these classes of bug a compile-time error rather than a runtime failure in production.

Alternatives considered:

• Plain JavaScript — rejected because a security-critical IdP with no type safety is not a system this team is willing to ship. Every public method, every error boundary, and every data transformation must be typed.

Consequences: All new code must compile cleanly under tsc --strict. Zero any types — ever. No exceptions granted without CTO approval. The tsconfig.json enables noImplicitAny, strictNullChecks, strictFunctionTypes, strictBindCallApply, strictPropertyInitialization, noImplicitThis, alwaysStrict, noUnusedLocals, noUnusedParameters, noImplicitReturns, and noFallthroughCasesInSwitch.

---

ADR-3: Express 4.18

Status: Adopted Component: HTTP server framework

Decision: Use Express 4.18 as the HTTP framework.

Rationale: Express is the most widely understood Node.js HTTP framework. Its middleware model ((req, res, next)) maps directly to the IdP's layered architecture: helmet → cors → metricsMiddleware → authMiddleware → opaMiddleware → controller → service → repository → errorHandler. The ecosystem for Express middleware (helmet, cors, morgan) is mature. For a spec-first project, Express's lack of convention about code structure is a feature — the architecture is explicit and fully visible in src/app.ts.

Alternatives considered:

• Fastify — rejected because the team's familiarity was lower and the performance gains would be negligible for a token service whose latency is dominated by PostgreSQL queries and bcrypt comparisons.
• NestJS — rejected because its decorator-heavy convention-over-configuration style adds complexity not appropriate for the current team size and project scope.
• Koa — rejected because its ecosystem is smaller and fewer engineers are familiar with it.

Consequences: All HTTP concerns (routing, middleware, error handling) use the Express 4 API. The errorHandler middleware must remain the last app.use() call in src/app.ts.

---

ADR-4: PostgreSQL 14

Status: Adopted Component: Primary data store for agents, credentials, and audit events

Decision: Use PostgreSQL 14 as the primary relational database.

Rationale: The audit log requires ACID guarantees — partial writes or uncommitted reads are not acceptable for a compliance-grade append-only event store. PostgreSQL's JSONB column type is used for the metadata field in audit_events, allowing structured context data without schema changes for each new event type. PostgreSQL's row-level security is available for multi-tenancy if that becomes a Phase 3 requirement.

Alternatives considered:

• MySQL — rejected because its JSON support is weaker than PostgreSQL's JSONB with GIN indexing, and its default transaction isolation level has historically produced surprises.
• MongoDB — rejected because the audit log must be append-only and ACID-safe. MongoDB's document model requires explicit multi-document transactions for ACID behaviour, and the schema flexibility is not needed here.

Consequences: All schema changes go through numbered SQL migration files in src/db/migrations/. Migration files are append-only — never modify an existing migration. New tables require a new numbered file (e.g. 005_create_agent_groups.sql).

---

ADR-5: Redis 7

Status: Adopted Component: Token revocation list, monthly usage counters, rate-limit sliding window

Decision: Use Redis 7 as the in-memory data store.

Rationale: Token revocation requires O(1) key lookup with TTL-based automatic expiry. SET revoked:{jti} 1 EX {seconds_until_expiry} stores a revocation entry that expires precisely when the token itself would have expired — zero manual cleanup required. The monthly token counter uses Redis INCR, which is atomic and O(1). The rate-limiter uses a Redis sorted set for the sliding-window algorithm.

Alternatives considered:

• Memcached — rejected because Memcached does not support per-key TTL on sorted-set structures, which is required for the sliding-window rate-limiter.
• PostgreSQL for revocation — rejected because the token verification path is the hot path in every authenticated request. A PostgreSQL round-trip adds 5–15 ms compared to a Redis GET at sub-millisecond latency.

Consequences: Redis is a required infrastructure dependency. A Redis instance must be running and reachable via REDIS_URL before the server starts. compose.yaml provides a Redis 7.2 Alpine container for local development on port 6379.

---

ADR-6: HashiCorp Vault

Status: Adopted (opt-in) Component: Credential secret storage — alternative to bcrypt in PostgreSQL

Decision: Integrate HashiCorp Vault KV v2 as an opt-in secret storage backend for agent credentials.

Rationale: The Phase 1 bcrypt approach stores hashes in PostgreSQL. While bcrypt hashes cannot be reversed, some enterprises require that secrets never touch a relational database — even in hashed form. Vault provides a dedicated secrets management plane with HSM backing and an independent audit trail at the secrets level. The verifySecret method in VaultClient uses crypto.timingSafeEqual to prevent timing-based side-channel attacks when comparing stored and candidate secrets.

Alternatives considered:

• AWS Secrets Manager — rejected because it introduces cloud-vendor lock-in. AgentIdP must run identically on AWS, GCP, and on-premises; a Vault-based approach works in all environments.
• Plain bcrypt only — retained as the fallback path. When VAULT_ADDR is not set, createVaultClientFromEnv() returns null and the server operates identically to Phase 1.

Consequences: Vault is controlled by VAULT_ADDR (required), VAULT_TOKEN (required), and VAULT_MOUNT (optional, defaults to secret). When these are not set, bcrypt is used unchanged. Credential rows carry a nullable vault_path column: null means bcrypt; a non-null path means Vault verification is used.

---

ADR-7: OPA (Open Policy Agent)

Status: Adopted Component: Request authorisation — scope enforcement on all protected endpoints

Decision: Use Open Policy Agent with a Rego policy compiled to a Wasm bundle for runtime authorisation.

Rationale: Hard-coded scope checks in middleware would require a code deployment for every policy change. OPA decouples the policy (policies/authz.rego) from the server code. The policy can be updated, re-compiled to Wasm, and hot-reloaded via SIGHUP without restarting the server. The @open-policy-agent/opa-wasm package evaluates the compiled Wasm bundle in-process with microsecond latency. When no Wasm bundle is present (development, CI), the middleware falls back to a TypeScript implementation that reads policies/data/scopes.json.

Alternatives considered:

• Custom middleware with hard-coded scope checks — rejected because policy changes require code changes and a full deployment cycle. As the endpoint surface grows this becomes unmanageable.
• Casbin — rejected because its RBAC/ABAC model is less expressive than Rego for the compound method + path + scope-intersection pattern AgentIdP requires.

Consequences: All authorisation rules live in policies/authz.rego and policies/data/scopes.json. Adding a new endpoint requires adding its scope requirement to scopes.json. A policy change is deployed by updating scopes.json (or authz.wasm) and sending SIGHUP to the running process — no redeployment needed.

---

ADR-8: React 18 + Vite 5

Status: Adopted Component: Web dashboard SPA (dashboard/)

Decision: Use React 18 with Vite 5 as the web dashboard framework and build tool.

Rationale: React 18's concurrent rendering model handles the dashboard's async data fetching patterns cleanly. The @sentryagent/idp-sdk Node.js package is reused directly in the dashboard via TokenManager for authentication, avoiding duplicated API client code. Vite 5 provides sub-second HMR in development and a fast production build with tree-shaking. The dashboard is built to dashboard/dist/ and served as static files from Express at /dashboard, keeping the deployment footprint to a single container.

Alternatives considered:

• Next.js — rejected because server-side rendering is not needed for an internal operator dashboard, and the added complexity of a Next.js server is not justified.
• Vue — rejected because the broader SentryAgent.ai ecosystem is React-first; consistency reduces context-switching overhead.

Consequences: The dashboard must be built (cd dashboard && npm run build) before Express can serve it. In local development, run cd dashboard && npm run dev to use Vite's dev server with HMR; the Vite proxy forwards /api/ calls to Express at localhost:3000.

---

ADR-9: Prometheus + Grafana

Status: Adopted Component: Operational metrics collection and visualisation

Decision: Use Prometheus for metrics collection and Grafana for dashboards.

Rationale: Prometheus is the industry standard for metrics in container environments. The prom-client npm package integrates natively with Express and provides Counter and Histogram metric types that cover all observability needs for AgentIdP. Grafana's YAML provisioning in monitoring/grafana/provisioning/ makes dashboards reproducible and version-controlled. The monitoring stack runs as a Docker Compose overlay (compose.monitoring.yaml) without interfering with the base dev environment.

Alternatives considered:

• Datadog — rejected because SaaS cost and vendor lock-in are not acceptable for a free, open-source product. Operators who self-host AgentIdP should not be required to pay for monitoring.
• StatsD — rejected because StatsD's flat metric model lacks label/dimension support, which is essential for distinguishing metrics by method, route, and status_code.

Consequences: All metric definitions live exclusively in src/metrics/registry.ts. No other file may instantiate a Counter or Histogram — all other files import specific metrics from that registry. Grafana is available at port 3001 when the monitoring overlay is running.

---

ADR-10: Terraform

Status: Adopted Component: Infrastructure as code — multi-region AWS + GCP deployment

Decision: Use Terraform with HCL for all infrastructure provisioning across AWS and GCP.

Rationale: Terraform's HCL syntax is readable and its provider ecosystem covers both AWS and GCP with the same toolchain. Reusable modules in terraform/modules/ (agentidp, lb, rds, redis) are composed in environment-specific configurations under terraform/environments/aws/ and terraform/environments/gcp/. All infrastructure changes go through terraform plan review before terraform apply, providing a diff-based approval workflow.

Alternatives considered:

• Pulumi — rejected because the Pulumi provider ecosystem for AWS and GCP was less mature than Terraform's at the time of the Phase 2 decision, and HCL is more readable for non-engineers reviewing infrastructure changes.
• AWS CDK — rejected because it is AWS-only. AgentIdP must deploy identically to both AWS and GCP.

Consequences: All infrastructure changes must go through Terraform. No manual edits via the AWS console or GCP console are permitted — they will be overwritten on the next terraform apply. Terraform state is stored in a remote backend and must not be edited manually.

---

ADR-11: Stripe

Status: Adopted Component: Billing — subscription management and payment processing

Decision: Use Stripe as the payment processing and subscription management platform. The stripe npm package (v21+) handles Checkout Session creation, webhook event verification, and subscription lifecycle events.

Rationale: Stripe's hosted Checkout flow eliminates the need to handle PCI-DSS scope for card data. The stripe.webhooks.constructEvent() method uses HMAC-SHA256 to verify incoming webhook payloads, preventing replay attacks. The checkout.session.completed event carries metadata: { orgId, targetTier }, allowing BillingService to delegate tier upgrades to TierService.applyUpgrade() without coupling billing logic to tier logic.

Alternatives considered:

• Paddle — rejected because its global merchant-of-record model introduced complexities with the open-source free tier.
• Braintree — rejected because Stripe's webhook reliability and developer experience are superior.

Consequences: Stripe requires STRIPE_SECRET_KEY (for API calls) and STRIPE_WEBHOOK_SECRET (whsec_..., for webhook verification). Per-tier Stripe price IDs are configured via STRIPE_PRICE_ID_PRO and STRIPE_PRICE_ID_ENTERPRISE. All billing webhook handlers must pass the raw Buffer body (not parsed JSON) to stripe.webhooks.constructEvent() — use express.raw() middleware on the webhook route.

---

ADR-12: oidc-provider (A2A Delegation)

Status: Adopted Component: A2A delegation — OIDC provider for agent-to-agent trust tokens

Decision: Use the oidc-provider npm package (v9.7.x) as the OIDC provider for issuing A2A delegation tokens. The provider is mounted as a sub-application at /oidc within the Express app.

Rationale: oidc-provider is a certified OpenID Connect implementation that handles the full OIDC protocol, including JWKS serving, token endpoint, and discovery document. Rather than implementing a custom delegation token format, using a standards-compliant OIDC provider means delegation tokens can be verified by any OIDC-aware party using the published JWKS at /oidc/jwks.

Alternatives considered:

• Custom JWT signing — rejected because hand-rolled token formats cannot benefit from OIDC tooling and interoperability.

Consequences: A2A_ENABLED env var gates the OIDC provider — when set to 'false', delegation endpoints return 404. The OIDC_ISSUER env var must be set to the full base URL of the OIDC provider (e.g. https://api.sentryagent.ai).

---

ADR-13: Next.js 14 (Developer Portal)

Status: Adopted Component: Developer Portal (portal/) — public-facing documentation and onboarding

Decision: Use Next.js 14 (App Router) with Tailwind CSS for the developer portal. The portal is a separate process served on its own port (independent of the Express API server).

Rationale: The developer portal has different performance and SEO requirements than the internal operator dashboard (dashboard/). Next.js 14's App Router supports React Server Components, which allows the marketing and documentation pages to be statically generated while the analytics dashboard and API Explorer are client-rendered. Tailwind CSS enables rapid UI development consistent with the design system.

Alternatives considered:

• Extending the Vite dashboard — rejected because the developer portal requires server-side rendering for SEO on marketing pages, which Vite does not provide.
• Docusaurus — rejected because the portal includes interactive components (Swagger Explorer, analytics charts) that are not well-suited to a documentation-only tool.

Consequences: The portal (portal/) has its own package.json, tsconfig.json, tailwind.config.ts, and next.config.js. It is built and run independently: cd portal && npm install && npm run dev. The portal calls the AgentIdP REST API using the same @sentryagent/idp-sdk as the dashboard.

---

ADR-14: bull (Job Queue) + kafkajs (Event Streaming)

Status: Adopted (opt-in) Component: Async job processing and event streaming

Decision: Use bull (Redis-backed job queue) for async webhook delivery retries and kafkajs for event streaming to external consumers. Both are opt-in — the system operates correctly without Kafka configured.

Rationale: Webhook delivery requires retry logic with exponential backoff and dead-letter handling. bull provides this out of the box using the existing Redis dependency. kafkajs enables high-throughput event streaming for analytics and audit events to external data pipelines without blocking the primary request path.

Alternatives considered:

• BullMQ — considered as a more modern alternative to bull but rejected to avoid adding a new package family during Phase 6. Migration is a future backlog item.

Consequences: Kafka is entirely optional. When KAFKA_BROKERS is not set, kafkajs is not initialised and no events are published. The bull queue for webhook delivery requires only the existing Redis instance.

---

ADR-15: did-resolver + web-did-resolver (W3C DIDs)

Status: Adopted Component: W3C DID Core 1.0 document resolution

Decision: Use did-resolver (v4.1.x) as the DID resolution framework and web-did-resolver (v2.0.x) for the did:web method implementation.

Rationale: did-resolver provides a pluggable resolver interface used by both the server (for internal resolution) and by third parties who want to verify AgentIdP-issued DIDs. The did:web method maps DID identifiers to HTTPS URLs hosting the DID document JSON, requiring no blockchain. DIDService generates documents that conform to the W3C DID Core 1.0 specification and include AGNTCY-specific extension fields.

Consequences: DID_WEB_DOMAIN env var is required for DID generation. DID documents are cached in Redis (did:doc:<agentId>, TTL from DID_DOCUMENT_CACHE_TTL_SECONDS, default 300s). Private keys are stored in HashiCorp Vault KV v2 when Vault is configured; in dev mode, a dev:no-vault marker is stored and keys are ephemeral.