docs: engineering knowledge base for new hires

Complete docs/engineering/ suite — 12 documents covering company overview, system architecture, tech stack ADRs, codebase structure, service deep dives, annotated code walkthroughs, dev setup, engineering workflow, testing strategy, deployment/ops, SDK guide, and README index. All content verified against source files. All 82 tasks in openspec/changes/engineering-docs/tasks.md marked complete. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 12:38:42 +00:00
parent 1f95cfe89d
commit eced5f8699
13 changed files with 3820 additions and 0 deletions
--- a/docs/engineering/08-workflow.md
+++ b/docs/engineering/08-workflow.md
@@ -0,0 +1,372 @@
+# 08 — Engineering Workflow
+
+---
+
+## 9.1 OpenSpec Spec-First Workflow
+
+Every feature in this codebase was designed before it was implemented.
+The OpenSpec workflow enforces this order without exception.
+
+### The Full Sequence
+
+```
+1. CEO identifies a feature or change
+   └── Documents it in the backlog
+
+2. CEO approves the feature for the current sprint
+   └── Creates an OpenSpec change document
+
+3. Virtual Architect designs the API
+   └── Writes the OpenAPI 3.0 spec BEFORE any implementation
+   └── Produces spec file in docs/openapi/ or openspec/changes/<name>/specs/
+   └── Every endpoint in the spec must have:
+       - Summary and description
+       - Request body schema (with all validation rules)
+       - All response schemas (every status code)
+       - Error response schemas
+       - Authentication requirements
+       - Example requests and responses
+
+4. Virtual CTO reviews the spec
+   └── Checks DRY, SOLID, AGNTCY compliance, completeness
+   └── Either approves or returns with corrections
+
+5. CEO approves the spec
+   └── Only after CTO approval
+   └── Scope changes require re-running from step 1
+
+6. Virtual Principal Developer implements the spec
+   └── Implementation must match the spec exactly
+   └── TypeScript strict mode, DRY, SOLID, JSDoc on all public methods
+   └── Zero any types
+   └── All errors typed and handled
+
+7. Virtual QA Engineer writes and runs tests
+   └── Unit tests: >80% coverage on all services
+   └── Integration tests: every endpoint in the spec tested
+   └── Edge cases: null, empty, invalid inputs
+   └── Performance: token endpoints <100ms, all others <200ms
+   └── Verifies spec matches implementation exactly
+
+8. Virtual CTO reviews the implementation and QA report
+   └── If quality gates not met: returns to step 6 or 7
+   └── If approved: notifies CEO
+
+9. CEO approves → code is merged to develop
+   └── No code ever goes to main without CEO awareness
+```
+
+### Rule: No Code Without a Spec
+
+If you find yourself implementing something without an approved OpenAPI spec,
+stop. Write the spec first, get it reviewed, then implement. This is not
+bureaucracy — it is how you avoid building the wrong thing.
+
+---
+
+## 9.2 OpenSpec CLI Commands Reference
+
+OpenSpec is the change management workflow built into this project. Changes
+are tracked in `openspec/changes/`.
+
+```bash
+# Create a new change (starts the design process)
+openspec new change <name>
+# Creates: openspec/changes/<name>/proposal.md, design.md, specs/, tasks.md
+
+# Check the status of all active changes
+openspec status
+# Shows: change name, phase (design/implementation/review), completion %
+
+# List all active changes
+openspec list
+
+# Get implementation instructions for a change
+openspec instructions <name>
+# Outputs the tasks.md formatted for implementation
+
+# Archive a completed change
+openspec archive <name>
+# Moves: openspec/changes/<name>/ → openspec/archive/<name>/
+```
+
+### Change Lifecycle
+
+```
+openspec/changes/<name>/
+├── proposal.md     — Business case and feature description (CEO-authored)
+├── design.md       — Technical design decisions (Architect-authored)
+├── specs/          — OpenAPI specs and interface contracts
+│   └── *.md or *.yaml
+└── tasks.md        — Implementation tasks checklist (checked off as work completes)
+```
+
+---
+
+## 9.3 Branching Strategy
+
+### Branch naming
+
+```
+feature/<short-description>   — New features (from develop)
+fix/<short-description>        — Bug fixes (from develop)
+docs/<short-description>       — Documentation changes only (from develop)
+```
+
+### Workflow
+
+```
+main         — Production. Only CTO-approved, CEO-aware merges.
+  └── develop    — Integration branch. All feature branches merge here.
+        └── feature/my-feature  — Your work branch
+```
+
+1. Create your branch from `develop`:
+   ```bash
+   git checkout develop
+   git pull origin develop
+   git checkout -b feature/my-feature
+   ```
+
+2. Work on your branch, committing as you go.
+
+3. When ready, push and open a PR targeting `develop`:
+   ```bash
+   git push -u origin feature/my-feature
+   gh pr create --base develop --title "feat: my feature" --body "..."
+   ```
+
+4. Virtual QA reviews the PR (all quality gates must pass).
+
+5. Virtual CTO approves the PR.
+
+6. Merge to `develop`.
+
+7. `develop` → `main` requires an explicit CEO decision.
+
+**Rule:** Never push directly to `main` or `develop`. Always work through a PR.
+
+---
+
+## 9.4 TypeScript and Code Standards
+
+### Strict Mode
+
+All compiler strictness flags are enabled in `tsconfig.json`. These are
+non-negotiable:
+
+```json
+{
+  "strict": true,
+  "noImplicitAny": true,
+  "strictNullChecks": true,
+  "noUnusedLocals": true,
+  "noUnusedParameters": true,
+  "noImplicitReturns": true
+}
+```
+
+**Consequence of violation:** The TypeScript compiler (`npm run build`) will fail.
+PRs that cause build failures are rejected automatically.
+
+### No `any` Types
+
+Never use `any`. If a third-party library returns `unknown` or `any`, cast it to
+a specific interface you define:
+
+```typescript
+// BAD
+const result: any = await vault.read(path);
+const secret = result.data.data.clientSecret;
+
+// GOOD
+interface KvV2ReadResponse {
+  data: { data: Record<string, string>; metadata: { version: number; } };
+}
+const result = (await vault.read(path)) as KvV2ReadResponse;
+const secret = result.data.data.clientSecret;
+```
+
+### DRY (Don't Repeat Yourself)
+
+Zero code duplication. See `04-codebase-structure.md` section 5.5 for the complete
+mapping of what lives where. Before writing a utility function, check whether
+it already exists in `src/utils/`.
+
+### SOLID Principles
+
+Each service has a single, clear responsibility. If you find yourself adding a
+method to `AgentService` that queries the `audit_events` table, stop — that
+belongs in `AuditService`. If you find yourself adding SQL to a controller, stop —
+that belongs in a repository.
+
+### JSDoc on All Public Methods
+
+Every public class, method, and interface must have a JSDoc comment that includes:
+- `@param` for every parameter
+- `@returns` describing the return value
+- `@throws` for every error that can be thrown
+
+```typescript
+/**
+ * Registers a new AI agent identity.
+ *
+ * @param data - Agent registration request data.
+ * @param ipAddress - Client IP for audit logging.
+ * @param userAgent - Client User-Agent for audit logging.
+ * @returns The newly created agent record.
+ * @throws FreeTierLimitError if the 100-agent limit is reached.
+ * @throws AgentAlreadyExistsError if the email is already registered.
+ */
+async registerAgent(data: ICreateAgentRequest, ipAddress: string, userAgent: string): Promise<IAgent>
+```
+
+### Error Handling
+
+Always throw a typed error from the `SentryAgentError` hierarchy. Never throw
+raw `Error` objects with string messages in service or controller code:
+
+```typescript
+// BAD
+throw new Error('Agent not found');
+
+// GOOD
+throw new AgentNotFoundError(agentId);
+```
+
+The `errorHandler` middleware maps `SentryAgentError` subclasses to HTTP status
+codes automatically. Adding a new error type only requires adding a class in
+`src/utils/errors.ts`.
+
+---
+
+## 9.5 PR Checklist
+
+Every pull request must pass all items before it can be merged.
+
+```
+Code quality:
+- [ ] TypeScript builds without errors: npm run build
+- [ ] No any types introduced
+- [ ] All new public methods have JSDoc
+- [ ] ESLint passes: npm run lint
+- [ ] No code duplication — logic extracted to utils/services
+
+Testing:
+- [ ] Unit tests added for all new service methods
+- [ ] Integration tests added for all new API endpoints
+- [ ] Coverage threshold maintained: npm run test:unit -- --coverage
+      (>80% statements, branches, functions, lines)
+- [ ] Integration tests pass against real PostgreSQL and Redis
+
+Spec compliance:
+- [ ] Implementation matches the approved OpenAPI spec exactly
+- [ ] If the spec needs updating, the spec was updated BEFORE the code was changed
+
+Documentation:
+- [ ] docs/engineering/ updated if the change affects service interfaces or workflows
+- [ ] CHANGELOG.md updated with a summary of the change
+- [ ] Any new environment variables documented in docs/engineering/07-dev-setup.md
+
+Database:
+- [ ] If a new table or column is added, a migration file exists in src/db/migrations/
+- [ ] Migration file is numbered correctly and tested locally
+
+Review:
+- [ ] Virtual CTO reviewed the implementation
+- [ ] Virtual QA signed off on tests
+```
+
+---
+
+## 9.6 Virtual Engineering Team Roles for Contributors
+
+External contributors operate within the same team structure as the internal
+Virtual Engineering Team. Here is how to interact with each role:
+
+**Virtual CTO (architecture gate)**
+Opens architectural discussions by filing a GitHub issue labeled `architecture`.
+The Virtual CTO must approve any change to:
+- The layered architecture (adding a direct DB call in a controller)
+- The error hierarchy
+- The authentication or authorisation flow
+- Any new dependency (package)
+- Multi-region deployment topology
+
+**Virtual Architect (spec owner)**
+All API changes require an updated OpenAPI spec. If you are adding an endpoint,
+file an issue labeled `spec-change` with your proposed additions before writing
+any code. The Architect will review and approve the spec.
+
+**Virtual Principal Developer (code reviewer)**
+All implementation PRs are reviewed for TypeScript compliance, DRY violations,
+SOLID violations, JSDoc completeness, and correctness against the spec.
+
+**Virtual QA Engineer (quality gate)**
+All PRs require >80% coverage and passing integration tests. The QA Engineer
+will review test completeness and flag edge cases that need coverage.
+
+---
+
+## 9.7 Commit Message Conventions
+
+This project uses **Conventional Commits** (https://www.conventionalcommits.org).
+
+### Format
+
+```
+<type>(<optional scope>): <short description>
+
+<optional body — why this change was made>
+
+<optional footer — breaking changes, issue references>
+```
+
+### Types
+
+| Type | When to use | Example |
+|------|-------------|---------|
+| `feat` | A new feature | `feat(agents): add agent status filter to list endpoint` |
+| `fix` | A bug fix | `fix(auth): handle TokenExpiredError separately from JsonWebTokenError` |
+| `docs` | Documentation only | `docs(engineering): add credential rotation walkthrough` |
+| `test` | Adding or updating tests | `test(oauth2): add monthly token limit integration test` |
+| `chore` | Build, tooling, dependencies | `chore: update jest to 29.7.0` |
+| `refactor` | Code change with no behaviour change | `refactor(credential): extract secret storage to VaultClient` |
+| `perf` | Performance improvement | `perf(token): make audit log write fire-and-forget` |
+
+### Rules
+
+- Keep the description under 72 characters
+- Use the imperative mood: "add" not "added" or "adds"
+- Include the scope in parentheses when the change is limited to one area
+- Reference issues in the footer: `Closes #123`
+
+### Examples
+
+```
+feat(vault): add optional HashiCorp Vault credential backend
+
+Adds VaultClient wrapping node-vault for KV v2 operations.
+When VAULT_ADDR and VAULT_TOKEN are set, new credentials are
+stored in Vault instead of as bcrypt hashes in PostgreSQL.
+
+Backwards compatible: existing bcrypt credentials continue to work.
+
+Closes #45
+```
+
+```
+fix(opa): normalise /token/revoke path before OPA lookup
+
+The path /api/v1/token/revoke was not in the normalisation
+switch, causing OPA to deny all revocation requests even with
+the correct scope.
+```
+
+```
+docs: add engineering knowledge base for new hires
+
+Adds docs/engineering/ with 11 documents covering architecture,
+service deep-dives, code walkthroughs, dev setup, workflow,
+testing, deployment, and SDK guide.
+```