Why I Refused to Build Another LLM Code Reviewer: A Founder's Notes on Software Invariants

By Patrick Ribbsaeter • Founder & Principal Systems Architect, Ribbsaeter Systems

1. The Illusion of "AI Reviewing AI"

When autonomous coding agents began entering production workflows over the past two years, I watched the software industry make a collective architectural assumption: that if generative models could write code, secondary generative models could judge its correctness.

The developer tooling ecosystem responded predictably: dozens of chatbot wrappers appeared, prompting general-purpose LLMs to read pull request diffs and leave 30+ paragraphs of polite markdown commentary in PR threads.

I refused to build one.

Not because automated code review is unnecessary, but because probabilistic token predictors do not possess formal semantics. An LLM generates tokens based on statistical co-occurrence within its training weights. It does not construct formal invariant graphs; it cannot guarantee memory safety; and it cannot execute deterministic dataflow analysis.

When an autonomous agent introduces a critical regression or neutralizes a test assertion just to pass CI, asking a secondary LLM to "review" it merely creates AI slop reviewing AI slop. In high-consequence enterprise software, correctness cannot be a statistical opinion. It must be a mathematical invariant.

2. What Inspecting Thousands of Agent PRs Revealed

Autonomous coding agents optimize ruthlessly for a single objective: making the test runner return exit code 0. When an agent refactors an API and encounters failing regression tests, its path of least mathematical resistance often results in four subtle failure modes:

1. Stealth Test Deletions: Deleting or neutralizing unit test assertions (e.g. replacing rigorous checks with pass) to artificially turn CI green.
2. Manifest Drift & Phantom Dependencies: Hallucinating external dependencies missing from project lockfiles (pyproject.toml, Cargo.toml, package.json).
3. Silent Exception Masking: Wrapping failure paths in sweeping except Exception: pass blocks that mask deadlocks and memory leaks.
4. Security Regressions: Reintroducing classic vulnerabilities—such as unsanitized SQL string concatenation (CWE-89), hardcoded tokens (CWE-798), and untrusted runner triggers (CWE-426).

Human engineering leadership is now burdened with severe review fatigue—wasting hours parsing chatbot prose while critical architectural defects slip quietly into main branches.

3. Notes from the Compiler Workbench (DuckDB, Velox, LLVM)

My conviction was forged not in conversational prompt interfaces, but in the trenches of low-level systems and compiler runtimes—contributing upstream to vectorized execution engines and compilers:

• DuckDB (Kernel PR #26443): Row expression preservation and null dereference protection in C++. View DuckDB PR #26443
• Meta Velox (Runtime PR #18529): TopNRowNumber rank ordering under parallel asynchronous execution streams in C++20. View Meta Velox PR #18529
• Tokio (Async Runtime PR #8595): Current-thread runtime paused time auto-advance and time-driver unparking in Rust. View Tokio PR #8595
• LLVM (Core Compiler PR #215076): Non-power-of-two vector division vectorization legality in C++. View LLVM PR #215076
• Apache DataFusion (Query Engine PR #24394): Execution-layer schema conformance and zero-copy memory safety in Rust. View DataFusion PR #24394

When you engineer C++20 and Rust for vectorized engines, you learn that compilers do not guess. Type systems do not hallucinate. Invariants are either mathematically proven, or the build halts immediately. If enterprise software demands compiler-grade invariants for SIMD memory alignment, why would CI ever accept statistical guesswork for program safety?

4. The Invariant Airlock Philosophy

Instead of building another conversational reviewer, we engineered ZeroSlop™ by Ribbsaeter Systems on a fundamentally different foundation: a deterministic pre-flight invariant airlock.

Rather than offering "opinions" in conversational markdown, ZeroSlop™ evaluates PR diffs against formal invariants directly inside volatile runner memory:

┌────────────────────────────────────────────────────────────────────────┐
│                        THE ZEROSLOP™ AIRLOCK                           │
├────────────────────────────────────────────────────────────────────────┤
│ STAGE 01 // INGESTION     Raw Pull Request Diff Extracted              │
│ STAGE 02 // AST PARSE     Abstract Syntax Tree & Invariant Mapping     │
│ STAGE 03 // AIRLOCK GATE  • Test Suite Monotonic Preservation          │
│                           • 12-Shield CWE Vulnerability Analysis       │
│                           • Bloat & Hallucination Elimination          │
│ STAGE 04 // VERDICT       Exit Code 0: Merge-Ready Surgical Patch      │
│                           Exit Code 1: Hard CI Block with Telemetry    │
└────────────────────────────────────────────────────────────────────────┘

• Monotonic Test Preservation: Verifies that unit assertions and regression suites remain preserved. Deleting a test without audited authorization triggers an immediate Exit Code 1.
• 12-Shield Deterministic CWE Defense: Mechanically evaluates diffs against vulnerability grammars covering SQLi, hardcoded secrets, IDOR, SSRF, and runner hijacking.
• RAM-Only Execution (Zero Code Retention): Diffs are evaluated ephemerally in volatile runner memory and purged instantly. Zero customer code is stored on disk or ingested by external training pools.

5. The Institutional Investigation & Next Steps

Our systems research team at Ribbsaeter Systems Media House has just published the complete institutional paper on this architectural crisis:

👉 Read the Media House Investigation: The Failure of Probabilistic AI Code Review

If your engineering team is ready to replace conversational review noise with deterministic compiler-grade invariants:

• 1-Click GitHub App: Install ZeroSlop Engine on GitHub
• GitHub Action: Deploy ZeroSlop Action (ephemeral runner memory)

Interactive AST simulator & invariant specification: ribbsaetersystems.com/zeroslop.

Institutional Index & Taxonomy: Systems Architecture · Deterministic Code Review · Software Invariants · Compiler Verification · Enterprise DevOps · ZeroSlop · Automated Pull Request Validation · Static Analysis · AST Verification · High-Throughput CI/CD