If you’re building with LLMs today, you’ve likely been sold a bill of goods about “reflection.” The narrative is seductive: just have the model check its own work, and watch quality magically improve. It’s the software equivalent of telling a student to “review your exam before turning it in.” The reality, backed by a mounting pile of peer-reviewed evidence, is far uglier. In most production scenarios, adding a self-reflection loop is the most expensive way to achieve precisely nothing—or worse, to degrade your output. The seminal paper that shattered the illusion is Huang et al.’s 2023 work, “Large Language Models Cannot Self-Correct Reasoning Yet.” Their finding was blunt: without external feedback, asking GPT-4 to review and correct its own answers on math and reasoning tasks consistently decreased accuracy. The model changed correct answers to wrong ones more often than it fixed errors. This isn’t an edge case; it’s a fundamental limitation of an autoregressive model critiquing its own autoregressive output with the same data, same biases, and zero new information.

Here's the counterintuitive premise: for any LLM application where errors have real consequences, you must build your evaluation harness before you write a single prompt. You don't prompt-engineer by vibes, tweaking until an output looks good. You start by defining what "good" means, instrumenting its measurement, and only then do you optimize. This is Eval-Driven Development. It's the only sane way to build reliable, high-stakes AI systems.

If your platform team’s North Star is getting every development squad into the “elite” performer bracket for DORA metrics, you’re aiming at the wrong target. You’re probably making things worse. I’ve watched organizations obsess over average deployment frequency or lead time, only to see platform complexity balloon and team friction increase. The real goal isn’t to build a rocket ship for your top performers; it’s to build a reliable highway for everyone else.

The corrective lens comes from a pivotal but under-appreciated source: the CNCF’s Platform Engineering Metrics whitepaper. It makes a contrarian, data-backed claim that cuts through the industry hype. The paper states bluntly that platform teams should focus on “improving the performance of the lowest-performing teams” and “reducing the spread of outcomes, not just the average.” This isn’t about settling for mediocrity. It’s about systemic stability and scaling effectively. When you measure platform success by how much you compress the variance in team performance, you start building for adoption and predictability—not vanity metrics.

Last verified: March 2026

Boris Cherny's team built RAG into early Claude Code. They tested it against agentic search. Agentic search won — not narrowly. A Claude engineer confirmed it in a Hacker News thread: "In our testing we found that agentic search outperformed [it] by a lot, and this was surprising."

That thread is the clearest primary source on how Claude Code actually works — and why it works that way. Most articles on the topic paraphrase it from memory. This one starts from the source.

Q: Does Claude Code index your codebase? A: No. Claude Code does not pre-index your codebase or use vector embeddings. Instead, it uses filesystem tools — Glob for file pattern matching, Grep for content search, and Read for loading specific files — to explore code on demand as it works through each task. Anthropic calls this "agentic search."

I asked an AI coding assistant to implement a page layout from a Figma design. It got the heading size wrong (28px instead of 24px), inserted a 4px gap where there should have been 8px, and hallucinated a duplicate magnifying glass icon inside the search bar. The overall structure was fine. The details were not.

This is the state of AI-assisted design-to-code in 2026. The tools get you 65-80% of the way there, then leave you in a no-man's land where the remaining pixels matter more than all the ones that came before. Every frontend engineer who has shipped production UI knows: "close enough" is not close enough.

I spent a session trying to close that gap using the toolchain everyone is talking about -- Figma MCP for design context, headless Playwright for runtime measurement, and an AI assistant for the correction loop. Here is what happened, what broke, and what produced results.

Multi-agent AI engineering has become a core discipline in production software development. The interesting question is no longer whether to build multi-agent systems. It is how — and specifically, which architectural pattern to reach for given the nature of the work. The clearest demonstration is that multiple fundamentally different paradigms live inside the same codebase.

There's a difference between an AI that can edit code and an AI that can repair code. Editing is mechanical — find a string, replace it. Repair requires understanding what's broken, why it's broken, and what the minimal fix looks like within the constraints of an existing codebase.

The Code Improver is the fourth agent in our six-agent autonomous self-improvement pipeline for nomadically.work. It's the only agent that writes application code. The Trajectory Miner finds patterns, the Codebase Auditor diagnoses issues, and the Skill Evolver improves instructions — but the Code Improver is the one that actually opens files and changes them.

Five research papers informed its design, curated from the VoltAgent/awesome-ai-agent-papers collection. The central insight across all of them: structured repair workflows outperform ad-hoc fixing.

Static analysis tools find pattern violations. Linters catch style issues. But neither traces an N+1 query from a GraphQL resolver through a DataLoader absence to a frontend performance degradation. That requires understanding execution paths — and that's what the Codebase Auditor does.

The Codebase Auditor is the second agent in our six-agent autonomous self-improvement pipeline for nomadically.work. It receives pattern IDs from the Trajectory Miner, investigates the actual code exhaustively, and produces findings with exact file:line references. It never modifies code — it only reads and reports.

Four research papers shaped its design, curated from the VoltAgent/awesome-ai-agent-papers collection. Here is how each one translated into practice.

Five agents in our pipeline know how to mine patterns, audit code, evolve skills, fix bugs, and verify changes. None of them knows when to do any of those things. That is the Meta-Optimizer's job.

The Meta-Optimizer is the sixth and final agent in our autonomous self-improvement pipeline for nomadically.work. It is the strategic brain: it reads all reports from other agents, determines the current phase of the system, creates prioritized action plans, and enforces safety constraints. It never edits code or skills directly. It only decides what should happen next.

Six research papers shaped its design. Together, they address the hardest problem in autonomous improvement: knowing when to improve, when to stop, and when to call for help.

Most AI systems have a hard boundary between the instructions they follow and the work they do. Developers write prompts; the AI executes them. If the prompts are wrong, a human fixes them. We built an agent that fixes its own prompts.

The Skill Evolver is the third agent in our six-agent autonomous self-improvement pipeline for nomadically.work. Its scope is precisely defined: it can edit skill files, commands, hooks, CLAUDE.md, and memory files. It cannot touch application source code — that's the Code Improver's job. This agent improves the instructions that all other agents follow.

Five research papers informed its design, curated from the VoltAgent/awesome-ai-agent-papers collection. Each one solved a different aspect of the self-modification problem.