How to Make an Agent Know It Screwed Up

me@justsomebody.dev (Rafał Opiłowski) — Mon, 06 Jul 2026 00:00:00 GMT

AI agents write code fast. They also leak null through every layer, create 2,000-line router files, leave bugs in production, and generate dead code once the happy path compiles.

This isn’t malice; LLMs optimize for plausible output, not production durability. You won’t stop them from making mistakes, but you can build a system where the agent finds out it screwed up before a human has to tell it.

I’ve implemented features and maintained code quality through an AI agent on a production TypeScript monorepo (10+ packages, 3+ apps). These are the layers that made it work.

Layer 0: Documentation

Write down what, how, and why; otherwise the agent guesses.

Agent context windows can’t hold the full codebase, so layered documentation delivers just-in-time context: project overview, package-specific rules, task specs. Write it once; the agent stops asking “what’s the structure?” on every task.

Layered Agent Instructions

A root-level AGENTS.md covers project overview, tech stack, filesystem structure, build system, and scripts. Each app and package has its own file with domain conventions: the API package documents router patterns, the database package documents schema, the frontend package documents components.

Files concatenate from root downward (OpenAI’s discovery pattern ¹), deeper files override shallower ones. An agent gets project context plus localized rules without reading every file.

Layer 1: Test-Driven Development

Tests first, then code. An agent can’t fake coverage when CI enforces thresholds.

Agents make tests pass, but they don’t know which tests to write. The configurations below remove that choice: passing tests mean working code, not well-mocked code.

Base Test Configuration

A shared config centralizes test settings: global utilities, dependency optimization, and coverage thresholds. In CI, the floor is 80% across branches, functions, lines, and statements.

Tests live next to the code they test (__tests__/ directories co-located with source), so the agent finds them without guessing where test files are stored.

In-Memory Database Tests

Integration tests use PGLite², an in-memory WebAssembly Postgres that runs in-process. No Docker, no network setup. A setup helper creates a fresh database, applies the production schema, and returns a Drizzle ORM client identical to production.

Benefits over traditional mocking:

Same schema as production, so no drift between test and real queries
Real PostgreSQL constraints, triggers, transactions, and locking all work as expected
No mocked query builders, so the agent can’t write a query that passes tests but fails in prod

Network-Layer API Tests

For client-side code, tests use Mock Service Worker (MSW³) to intercept HTTP requests at the network layer. The test server uses production router definitions. The agent tests against real request/response cycles, not mocked function returns.

This catches mistakes module-level mocks miss:

Request serialization errors (wrong content-type, malformed body)
Response parsing failures (wrong shape, missing fields)
Error handling paths (HTTP 400, 401, 403, 500)
Caching and retry behavior

The full client stack (transport, serialization, error handling, caching) runs unmodified. Only the network hop is replaced.

Layer 2: Spec-Driven Development ⁴

The best time to catch a mistake is before the agent writes any code.

Give an agent a vague prompt and it guesses scope, usually wrong. A spec forces thinking before typing:

Scope boundaries prevent feature creep. The agent reads “out of scope: bulk operations” and doesn’t spend cycles building pagination when all that’s needed is a text input.
User scenarios encode edge cases the agent doesn’t know to ask about. What happens when the search returns zero results? What’s the error message for a network timeout?
Acceptance criteria are the test the agent can verify against. No “looks done.” The spec says exactly what done means.

The Workflow

The screenshots below span different projects within a similar tech stack. The workflow uses NeoLab’s Context Engineering Kit.

Human writes → Initial prompt (captured verbatim in draft/)
Agent expands → Structured spec with scope, scenarios, ACs
Human reviews → Validates direction, adjusts scope
Agent implements → Follows spec, verifies against ACs
Spec graduates → Moved to done/ as documentation

The spec is the contract. It stops the agent from building the wrong thing.

Layer 3: Architectural Isolation

Some bug classes shouldn’t be possible. Architecture, not code review, enforces that.

When business logic tangles with infrastructure, agents inject database calls into validators, API calls into pure functions, and framework imports into domain models. Physical separation prevents this because you can’t import what isn’t there.

Monorepo Structure

Everything in one repo. The agent gathers cross-domain context in a single workspace instead of hopping repos and losing state.

.
├── apps/
│   ├── backend/               # API server
│   ├── admin/                 # Web admin
│   └── frontend/              # Frontend app
└── packages/
    ├── api/                   # API router definitions
    ├── auth/                  # Authentication primitives
    ├── common/                # Pure business logic & utilities
    └── db/                    # Database schema, migrations, client

Module Boundaries

A monorepo without enforcement is just a folder convention. Explicit module boundaries prevent packages from importing each other’s internals:

packages/common can’t import from packages/db. Pure logic stays database-agnostic.
apps/frontend can’t import from apps/backend. They share typed packages, not each other’s code.
Each package exports a controlled surface area via its index.ts; everything else is private.

Lint rules enforce this (ESLint import/no-restricted-paths, Nx module boundary tags). The agent can’t wire a frontend import to a server-only module, and the build fails.

Pure Logic Separation

Business logic lives in common with zero infrastructure dependencies. No database, HTTP, or framework imports:

packages/common/src/
├── challenges/
│   ├── validators.ts          # Pure validation, no mocks needed
│   ├── dateUtils.ts           # Date math, no timezone mocking
│   └── votingTimes.ts         # Computation, no DB to seed
├── rewards/
│   ├── validators.ts          # Reward rule validation
│   └── calculateProgress.ts   # Progress math
├── users/
│   └── banUtils.ts            # Status parsing
├── coins/
│   └── statistics.ts          # Statistics
└── __tests__/                 # Co-located unit tests, instant feedback

Each function takes inputs, returns outputs, has no side effects. Testing is trivial with no mocks or seeding. The agent can’t accidentally call a database inside a pure function because there’s no client to import.

Layer 4: Automated Enforcement

Every mistake should surface within minutes, and the earlier it surfaces, the cheaper it is to fix.

Local Gate: Git Hooks

The first defense runs on the developer’s machine, before anything reaches the remote.

Pre-commit: Formatter auto-fixes staged files, linter runs on changed files, type checker validates the full project. If formatting or types fail, the commit stops.

Commit message: An interactive prompt guides through Conventional Commits⁵ format. This isn’t cosmetic; downstream automation parses this string. A validation hook checks conformity and appends a DCO⁶ sign-off automatically.

Pre-push: Test runner triggers before push, but only if test files changed (skipping docs-only pushes saves minutes). All branch commits re-validate against Conventional Commits, catching violations from squashed commits. Dead code detection flags every unused export and orphaned file the AI left behind.

Hooks install automatically via prepare with no setup and no excuses.

CI Pipeline

When code passes local gates, CI takes over. Each workflow covers a distinct risk:

Lint & Format: Runs the same formatter and linter as local hooks, plus module boundary enforcement.
Type Check: Full TypeScript type checking. On PRs, reviewdog⁷ annotates errors on the diff. The agent sees the exact line, not a wall of logs.
Test Suite: Tests shard across parallel runners. On main branch pushes or critical path changes, the full suite runs. Coverage artifacts upload per-shard for merged reporting.
Coverage Gate: A diff-coverage comment on every PR shows lines touched vs. covered, failing if the threshold drops.
PR Title Lint: PR titles must follow Conventional Commits, the same standard as commits. No agent opens a PR titled ‘fix stuff’.

Only when all checks pass can the PR merge. This gate is a hard firewall against sloppy code.

Layer 5: The Human + Agent Loop

None of these layers work without a human. Guardrails don’t replace review; they make review productive by catching boring mistakes so humans focus on architecture.

When Guardrails Fail

The layers catch mechanical mistakes: wrong types, missing tests, dead code, broken boundaries. They don’t catch:

Wrong feature: Spec fits the prompt but doesn’t solve the user’s actual problem
Right feature, wrong UX: Everything works, nobody can use it
Technical debt acceleration: Code passes all checks but makes the next change harder

These require human judgment. Zero mistakes is the wrong target, so aim for zero surprises. Every error should be visible and cheap to fix before production.

OpenAI guidelines to AGENTS.md ↩
PGLite - In-process WebAssembly PostgreSQL ↩
Mock Service Worker - API mocking at the network layer ↩
Spec-Driven Development - methodology that prioritizes the creation of precise, machine-readable specifications over immediate coding ↩
Conventional Commits - Structured commit message specification ↩
DCO - Developer Certificate of Origin ↩
reviewdog - Automated code review tool ↩

Rafał Opiłowski