SDLC.md - Specs Your Agents Can Execute

# SDLC.md - Software Development Lifecycle Context
<!-- Context for AI coding assistants - drop this into AGENTS.md, CLAUDE.md, or both -->
<!-- Covers the spec, architecture decisions, stack, workflows, deployment, and verification gates -->
<!-- Last updated: 2026-07-27 -->

## How to Use This File

This file is the durable context an AI coding assistant reads before it touches your repository. It is plain Markdown - no frontmatter, no schema, no build step - so every tool can consume it and every human can review it in a pull request.

### Where to put it

Put this content in `AGENTS.md` at the repository root. `AGENTS.md` is the cross-tool convention for agent-readable project context: it originated at OpenAI, was transferred to the Linux Foundation's Agentic AI Foundation, and is read natively by Claude Code, Codex CLI, Cursor, GitHub Copilot, Gemini CLI, Aider, Windsurf, Amazon Q, and others. One file, every tool.

If your team already keeps a `CLAUDE.md`, keep both. Claude Code reads and merges `AGENTS.md` and `CLAUDE.md` together with any files it finds in subdirectories, so this is not an either/or choice. The split that works well in practice: shared project facts in `AGENTS.md`, tool-specific operating instructions in `CLAUDE.md`.

### Precedence

1. An explicit instruction in the current prompt beats every file.
2. The nearest file to the code being edited beats files further up the tree. A `packages/api/AGENTS.md` overrides the root file for work inside `packages/api/`.
3. Root-level context applies everywhere else.

Add a small `AGENTS.md` to any subdirectory whose conventions differ from the root - a generated client, a legacy module, an infrastructure directory.

### What belongs here

| Belongs in this file | Belongs somewhere else |
|----------------------|------------------------|
| Decisions and the reasoning behind them | Auto-generated API reference |
| Commands that must be run, exactly as typed | Long prose that duplicates the code |
| Scope boundaries and things never to touch | Secrets, tokens, connection credentials |
| Verification gates and who signs off | Ticket-by-ticket sprint status |
| Known issues, with the workaround | Anything that changes weekly |

Keep it under a few hundred lines. A context file that nobody maintains is worse than no context file, because agents follow it confidently after it has gone wrong.

## Spec-Driven Development

The organizing idea behind this file: **the spec is the primary artifact and the code is a regenerable output produced from it** - by humans, by agents, or by both working together.

That is a real inversion of the old model, where the code was the truth and documentation was a lagging description of it. When the spec is primary:

- Requirements changes are edits to the spec, followed by regenerating the affected code. You do not patch code and then backfill the documentation.
- Review shifts earlier. Arguing about a paragraph in the spec is enormously cheaper than arguing about a thousand-line diff.
- An agent handed a complete spec has the context it otherwise has to guess at, and guessing is where most bad agent output comes from. Teams working this way report a substantially higher first-pass success rate on non-trivial tasks; treat the size of that effect as reported experience rather than a benchmark.

### The six things a good spec defines

1. **Outcomes** - what is true after this ships, stated so it can be checked.
2. **Scope boundaries** - what is explicitly out, and which files or modules must not change.
3. **Constraints** - platform, performance, security, compliance, timeline, and existing patterns to follow.
4. **Prior decisions** - the ADRs that already constrain the solution space, so nobody relitigates them.
5. **Task breakdown** - the work split into units small enough that each one has an obvious done state.
6. **Verification criteria** - the exact commands and gates that prove the change is correct.

If you cannot fill in section 6, the spec is not finished. See the Verification Criteria section below for this project's gates.

### Keeping the spec alive

- The spec changes in the same pull request as the code it governs. A pull request that changes behavior without changing the spec is incomplete.
- When implementation diverges from the spec, the divergence is written down and the spec is corrected. Silent divergence is how a spec becomes a lie.
- Superseded specs are marked `Superseded by <id>` and kept. History is the point.

## Project Overview

**Project**: Meridian - Customer Analytics Platform
**Version**: 3.2.1
**Status**: Production
**Repository**: https://github.com/acme-corp/meridian
**Team**: Platform Engineering (6 developers, 2 QA)

### Elevator Pitch

Meridian is a real-time customer analytics platform that ingests event data from web and mobile clients, processes it through a streaming pipeline, and serves interactive dashboards to business users. It replaces our legacy batch-processing system with sub-second query performance across 2TB+ of event data.

### Key Business Context

- Serves 340 internal business users across marketing, product, and sales
- Processes 12M events/day from 3 client applications
- SLA: 99.9% uptime, dashboard queries under 2 seconds
- Revenue impact: directly supports USD 4.2M ARR through customer insights

### Scope Boundaries

- **Never modify** `packages/legacy-etl/` - it is frozen pending decommission and has no test coverage.
- **Never modify** generated files: `packages/api/src/graphql/generated/`, `prisma/client/`.
- Schema changes to `events` in ClickHouse require a data platform review before implementation.
- Do not add a new runtime dependency without an entry in Critical Dependencies below.

## Architecture Decisions (ADRs)

Every ADR records **provenance**: who or what authored it, who reviewed it, and when. This is not bureaucracy. Once agents draft decisions as well as implement them, "a human decided this" and "a human accepted a drafted decision" are different facts, and six months later the difference is exactly what you need to know.

**Provenance values**:

| Value | Meaning |
|-------|---------|
| `human` | Written by a named person from their own analysis |
| `agent` | Drafted and merged by an agent with no human author in the loop |
| `agent-with-human-review` | Drafted by an agent, then read, edited, and accepted by a named human |

An ADR whose provenance is `agent` and whose reviewer is empty is a finding, not a decision. Fix it or downgrade its status to Proposed.

### ADR-001: ClickHouse for Analytics Storage

**Status**: Accepted
**Date**: 2026-07-27
**Provenance**: human
**Authored by**: Priya Raman (Staff Engineer)
**Reviewed by**: Marcus Webb (Data Platform Lead), Dana Okonjo (Principal Architect)
**Context**: Our PostgreSQL analytics tables hit 800M rows. Aggregate queries were taking 30+ seconds even with materialized views and proper indexing. Business users complained about dashboard load times daily.
**Decision**: Adopt ClickHouse as the primary analytics data store. PostgreSQL remains for transactional data (users, configs, permissions). Events flow from Kafka into ClickHouse via a custom consumer service.
**Consequences**: Query performance improved from 30s to under 500ms at the 95th percentile. Trade-off is that ClickHouse does not support UPDATE/DELETE well, so we use an append-only model with deduplication views. Team needed 3 weeks of ClickHouse training.

### ADR-002: Migrate from REST to GraphQL for Dashboard API

**Status**: Accepted
**Date**: 2026-07-27
**Provenance**: agent-with-human-review
**Authored by**: Coding agent, from the dashboard latency investigation trace in `docs/traces/`
**Reviewed by**: Priya Raman (Staff Engineer) - rewrote the Consequences section and rejected the agent's proposal to retire the REST surface
**Context**: The dashboard frontend was making 8-12 REST calls per page load to assemble widget data. This created waterfall latency and tight coupling between frontend components and backend endpoints.
**Decision**: Implement a GraphQL gateway (Apollo Server) in front of the existing services. REST endpoints remain for mobile and third-party integrations.
**Consequences**: Frontend page loads dropped from 12 requests to 1-2 and average dashboard load time fell by 60%. Added complexity on the backend: the resolver layer needs its own tests and monitoring. Schema changes require coordination between the frontend and backend teams.

### ADR-003: Event-Driven Architecture with Kafka

**Status**: Accepted
**Date**: 2026-07-27
**Provenance**: human
**Authored by**: Marcus Webb (Data Platform Lead)
**Reviewed by**: Dana Okonjo (Principal Architect)
**Context**: The old system used synchronous API calls between services. When the analytics service was slow or down, it cascaded failures into the event ingestion API and caused data loss.
**Decision**: Introduce Apache Kafka as the central event bus. All services publish and consume events asynchronously. Events are retained in Kafka for 7 days, allowing replay if a consumer falls behind or fails.
**Consequences**: Services are fully decoupled and new consumers can be added without touching producers. Operational complexity increased - the Kafka cluster needs dedicated monitoring and occasional partition rebalancing. Event processing gained roughly 200ms of latency, which is acceptable for an analytics workload.

### ADR Template

```markdown
### ADR-NNN: [Short decision title]

**Status**: [Proposed | Accepted | Superseded by ADR-NNN | Rejected]
**Date**: [YYYY-MM-DD]
**Provenance**: [human | agent | agent-with-human-review]
**Authored by**: [Name, or agent plus a link to the run record]
**Reviewed by**: [Names - required for Accepted status]
**Context**: [What forced a decision? Include the numbers that made it urgent.]
**Decision**: [What we are doing, stated actively.]
**Consequences**: [What got better, what got worse, what we now have to live with.]
```

## Tech Stack

### Frontend

- **Framework**: React with TypeScript
- **State Management**: TanStack Query (server state) + Zustand (UI state)
- **Charting**: Recharts for dashboards, D3.js for custom visualizations
- **Styling**: Tailwind CSS with an internal component library
- **Build**: Vite, deployed as static assets to CloudFront

### Backend

- **Runtime**: Node.js, current LTS line. The exact version is pinned in `.nvmrc` - that file is the source of truth, not this document.
- **API Layer**: Apollo Server (GraphQL) + Express (REST)
- **Event Processing**: Custom Kafka consumers in Node.js
- **Task Queue**: BullMQ with Redis for scheduled reports and exports
- **Authentication**: Auth0 with SAML SSO for enterprise customers

### Data Layer

- **Transactional DB**: PostgreSQL (AWS RDS) - users, configs, permissions
- **Analytics DB**: ClickHouse cluster (3 nodes) - event data, aggregations
- **Cache**: Redis (ElastiCache) - session data, query cache, rate limiting
- **Message Bus**: Apache Kafka (MSK) - event streaming between services

### Infrastructure

- **Cloud**: AWS (us-east-1 primary, us-west-2 disaster recovery)
- **Container Orchestration**: EKS
- **CI/CD**: GitHub Actions with ArgoCD for GitOps deployments
- **Monitoring**: Datadog (APM, logs, metrics), PagerDuty (alerting)
- **IaC**: Terraform for AWS resources, Helm charts for Kubernetes

### Version pinning policy

Do not hardcode a version number in prose - prose does not get validated and rots silently. Pin it in a lockfile or manifest and reference the file:

```bash
node --version          # expect the version pinned in .nvmrc
cat .nvmrc              # source of truth for the Node line
npm ls --depth=0        # resolved versions from package-lock.json
terraform version       # must satisfy required_version in versions.tf
```

Major-version upgrades get an ADR. Minor and patch upgrades go through the normal dependency pull request.

## Development Workflows

### Local Setup

```bash
# Clone and install
git clone [email protected]:acme-corp/meridian.git
cd meridian
nvm use                 # reads .nvmrc, no version to keep in sync here
npm ci                  # lockfile-exact install, not npm install

# Start dependencies (Postgres, Redis, ClickHouse, Kafka)
docker compose up -d

# Environment setup
cp .env.example .env
# Edit .env - get secrets from the 1Password vault "Meridian Dev"

# Run database migrations
npm run db:migrate
npm run db:seed

# Start development (all services)
npm run dev
# Frontend: http://localhost:5173
# GraphQL Playground: http://localhost:4000/graphql
# REST API: http://localhost:4000/api/v1
```

### Branch Strategy

- `main` - Production-ready code, deploys automatically
- `staging` - Pre-production integration, deploys to the staging environment
- `feature/*` - New features, branch from `main`
- `hotfix/*` - Production fixes, branch from `main`, merge to both `main` and `staging`

### Code Review Process

1. Create a feature branch from `main`
2. Implement changes with tests (minimum 80% coverage on new code)
3. Open a pull request with the description template filled out (what, why, how to test)
4. Automated checks must pass: lint, type-check, unit tests, integration tests
5. Require 1 approval from the code owner for the affected area
6. Squash and merge - the pull request title becomes the commit message

For any change an agent produced, the pull request description must also link the agent run record: goal, plan, actions taken, verification output, and what a human changed afterwards. A reviewer who cannot see how the change was produced is reviewing half of it.

### Commit Convention

```
feat: add retention cohort chart to dashboard
fix: resolve timezone offset in event timestamps
perf: add ClickHouse projection for top-events query
refactor: extract shared auth middleware to @meridian/auth
docs: document the retention cohort query in schema.md
test: add integration tests for GraphQL mutations
chore: bump TanStack Query to the version pinned in package.json
```

## Verification Criteria

How anyone - human or agent - knows a change is done and correct. These are the gates, in order. Do not declare work complete until every applicable gate has actually been run and its output read.

### Automated gates

```bash
npm run lint            # ESLint, zero errors and zero warnings
npm run typecheck       # tsc --noEmit, zero errors
npm test                # unit tests, all green, 80% minimum on changed files
npm run test:integration  # requires docker compose up -d
npm run build           # must succeed with no new warnings
npm run db:migrate:status # zero pending migrations after a schema change
```

### Gate rules

- A gate that was skipped is a failed gate. Say which ones you skipped and why.
- Never edit a test to make it pass unless the test itself is the bug, and say so explicitly when you do.
- New behavior needs a new test. A change with no test change is either untested or unnecessary.
- Performance-sensitive changes report a measured before and after, not an estimate.

### Definition of done

- [ ] Every automated gate above passes locally and in CI
- [ ] The spec was updated in the same pull request, or the change needed no spec update and the description says so
- [ ] Any behavior change is covered by a test that fails without the change
- [ ] Any decision worth remembering is captured as an ADR with provenance filled in
- [ ] Environment variables, if added, are in `.env.example` and in the table below
- [ ] Rollback path stated in the pull request description

### What a human must sign off on

An agent may not self-approve any of these. They require a named human:

- Database migrations, including anything generated into `prisma/migrations/`
- Changes to authentication, authorization, or session handling
- Anything touching payment, billing, or invoice calculation
- Deletion of data, tables, columns, or retention windows
- New third-party runtime dependencies, and any change to their scope
- Public API contract changes, including GraphQL schema removals
- Infrastructure changes that alter network exposure or IAM policy

## Deployment Procedures

### Staging Deployment

```bash
# Automated on merge to the staging branch
git checkout staging
git merge feature/my-feature
git push origin staging
# GitHub Actions: lint -> test -> build -> deploy to EKS staging
# URL: https://staging.meridian.internal.acme.com
# Slack notification sent to #meridian-deploys
```

### Production Deployment

```bash
# Merge staging to main (after QA sign-off)
git checkout main
git merge staging
git push origin main
# GitHub Actions: full test suite -> build -> push to ECR -> ArgoCD sync
# ArgoCD performs a rolling update (zero-downtime)
# Canary: 10% traffic for 15 minutes, then full rollout
# URL: https://meridian.acme.com
```

### Rollback Procedure

```bash
# Option 1: ArgoCD rollback (fastest, under 2 minutes)
argocd app rollback meridian-prod --revision [previous-revision]

# Option 2: Git revert (creates an audit trail)
git revert [commit-hash]
git push origin main
# ArgoCD auto-syncs the revert

# Option 3: Emergency - direct image rollback
kubectl set image deployment/meridian-api api=meridian-api:[previous-tag] -n production
```

### Database Migrations

```bash
# Migrations run automatically during deployment
# For manual execution:
npm run db:migrate          # Apply pending migrations
npm run db:migrate:status   # Check migration status
npm run db:migrate:rollback # Roll back the last migration

# ClickHouse migrations are separate:
npm run ch:migrate          # Apply ClickHouse DDL changes
```

## Critical Dependencies

- **Auth0** (authentication) - If Auth0 is down, no users can log in. Mitigation: JWT tokens are validated locally against cached JWKS, so existing sessions continue to work for up to 1 hour.
- **Kafka (MSK)** (event pipeline) - If Kafka is down, events queue in the ingestion API (up to 10K events in an in-memory buffer, then rejected with 503). Events are not lost if Kafka recovers within the buffer window.
- **ClickHouse** (analytics queries) - If ClickHouse is down, dashboards show cached data (up to 5 minutes stale) behind a degraded-mode banner. Transactional features are unaffected.
- **Redis** (caching/sessions) - If Redis is down, query performance degrades because every query hits ClickHouse directly, and rate limiting is disabled. Sessions fall back to JWT-only validation.

## Known Issues and Technical Debt

- [ ] **ClickHouse dedup lag** - Deduplication views can lag up to 30 seconds during high-throughput periods, so real-time dashboards occasionally show duplicate events. Priority: Medium. Workaround: client-side dedup in the dashboard query layer.
- [ ] **GraphQL N+1 in nested resolvers** - The `project.members.recentActivity` resolver generates N+1 queries when loading team dashboards with 20+ members. Priority: High. Fix: DataLoader batching, ticket MER-1841.
- [ ] **Legacy REST endpoints** - 14 REST endpoints are still used by the previous major version of the mobile app. They duplicate logic that now lives in GraphQL resolvers. Priority: Low. Removal is blocked until mobile analytics show the old major version below 1% of sessions for two consecutive weeks; track it in ticket MER-1102, not on a calendar date.
- [ ] **Test coverage gap in Kafka consumers** - Consumer error-handling paths sit near 40% coverage. Priority: Medium. Integration tests are hard to write because they need a running Kafka instance.
- [ ] **Unreviewed agent-authored ADRs** - Two ADRs in `docs/adr/` have provenance `agent` with no reviewer. Priority: Medium. Either review them or downgrade them to Proposed.

Do not record debt with a target quarter. Record the condition that clears it, and the ticket that tracks it. Quarters expire; conditions do not.

## Governance and Regulatory Documentation

Keep this section proportionate. It is here because a versioned decision record in the repository is the cheapest possible answer to a documentation request, and because the questions are now being asked.

The EU AI Act's main body of obligations applies from 2 August 2026. Two parts matter to most engineering teams:

- **Article 50 transparency obligations** apply broadly, including to systems that are not high-risk. If your product puts an AI system in front of a person, or generates content a person may take for human-made, disclosure is in scope.
- **High-risk requirements (Articles 6-15)** bring conformity assessment, technical documentation, CE marking, and EU database registration - but only for the specific use cases enumerated in Annex III, such as employment and worker management, access to essential services, or safety components of regulated products.

Being honest about the boundary: **using an AI assistant to write code, and shipping AI-generated code, does not by itself make your system high-risk.** Annex III regulates what the system is used for, not how the source was produced. Vendors selling compliance products blur this; do not repeat the blur in your own documentation.

What the regime actually asks of engineering is documentation, traceability, and human oversight. This file plus a versioned ADR history plus agent run records answer all three:

| Question | Where the answer lives |
|----------|------------------------|
| Who decided this, and on what basis? | ADR with provenance and reviewers |
| How was this change produced? | Agent run record linked from the pull request |
| What human oversight applied? | The human sign-off list in Verification Criteria |
| Can you reconstruct the state at a past date? | Git history of this file and `docs/adr/` |
| Where is the disclosure to end users? | Product transparency copy, tracked in the spec |

If your product does fall under Annex III, this file is a starting point and not a compliance program. Get counsel.

## Environment Configuration

| Variable | Required | Description |
|----------|----------|-------------|
| `DATABASE_URL` | Yes | PostgreSQL connection string |
| `CLICKHOUSE_URL` | Yes | ClickHouse HTTP endpoint |
| `REDIS_URL` | Yes | Redis connection string |
| `KAFKA_BROKERS` | Yes | Comma-separated Kafka broker addresses |
| `AUTH0_DOMAIN` | Yes | Auth0 tenant domain |
| `AUTH0_CLIENT_ID` | Yes | Auth0 application client ID |
| `AUTH0_CLIENT_SECRET` | Yes | Auth0 application client secret |
| `NODE_ENV` | No | Environment mode (development/staging/production) |
| `LOG_LEVEL` | No | Logging verbosity (debug/info/warn/error) |
| `GRAPHQL_INTROSPECTION` | No | Enable GraphQL introspection (disabled in production) |

Never paste a secret value into this file. Agents read it, and anything an agent reads can end up in a log, a transcript, or a pull request comment.

## Contact and Resources

- **Tech Lead**: [Name] - [Email/Slack handle]
- **Product Manager**: [Name] - [Email/Slack handle]
- **On-Call Rotation**: See the #meridian-oncall channel topic in Slack
- **Runbook**: [Link to the operational runbook - see Steps.md for the format]
- **Specs**: `docs/specs/` - see Spider.md for the specification format
- **ADRs**: `docs/adr/`
- **Agent run records**: `docs/traces/`
- **Architecture Diagram**: [Link]
- **Monitoring Dashboard**: [Link to the Datadog dashboard]
- **Incident Response**: Page on-call via PagerDuty, escalation in #meridian-incidents