A Framework for AI Dev Tooling: Models, Harnesses, and Loops
A practical framework for AI development tooling: models, agent harnesses, repo harnesses, and inner, outer, and campaign loops for scaling AI-assisted engineering.
I feel that there’s a pattern forming in how cutting-edge engineering orgs use AI tooling to speed up the dev process.
Two years ago the debate was about models. While models are still important, the LLM harness (this is what Claude Code and Codex CLIs are — a harness around LLMs) is becoming an important differentiator. Most cutting edge companies have settled on Claude Code or Codex CLI as their primary LLM harness.
Some quick definitions before we move on.
Coding agent harness — the deterministic logic around the LLM itself (Claude Code, Codex CLI). The planning loop, the tool calls, the memory architecture.
Repo harness — CLAUDE.md / AGENTS.md (with other repo documentation), custom slash commands & skills, linters, automated tests, MCP servers.
Looking at the AI tooling market today, newer products are clearly converging on the repo harness + outer-loop orchestration pattern. Older products — and in the age of AI, “older” means over a year — are more piecemeal, built as standalone vertical features. Walking through the segments:
Code review segment - Code Rabbit, Qodo, Baz, etc. First-generation products, each with their own LLM, prompting, and harness. My read is that this category is gradually being absorbed into the outer-loop implementation tools below.
Implementation segment - Devin, GitHub Copilot Coding Agent, OpenHands, and others. This is where the convergence is most visible — these tools have grown meaningful repo harness support (AGENTS.md and similar) and outer-loop orchestration primitives, though each still ships its own agent harness. A few tools let you delegate to Claude Code or Codex (Conductor, OpenRig), but they’re aimed at individuals running the tool on their laptop. OpenHands recently added the option to run frontier coding agents via ACP.
Fix code review findings segment - many CI tools, Linear, etc are all jumping into “press a button and fix your findings”. This is the one segment where I think the bet is wrong — piecemeal features running in proprietary agents, disconnected from the dev’s primary loop. Not what developers actually need.
The bigger story, though, is what the companies that have figured this out are building themselves. Stripe, Shopify, Uber, Airbnb have all talked publicly about their internal AI dev platforms. These orgs are not bolting on piecemeal vendor features — they’re building full outer-loop orchestration that integrates one agent harness across local dev, cloud agents, and review. Smaller cutting-edge teams I’ve talked to are doing the same (but they don’t post this online). The reason is always the same: integrate the agent into your process, don’t import someone else’s.
A couple more definitions for the AI development process. The loops are best defined by how often the developer has to check in with the AI. Note that where the loop runs (on your laptop, in the cloud) is not important - it’s the interaction model between the builder and AI that is the defining factor.
Inner loop — conversational and real-time. Check-ins happen in seconds to minutes. Even when you ask the agent to do research or a small implementation and wait a few minutes for it to come back, that’s still part of the conversation. Typically this is your laptop (but not necessarily): ticket/task definition & architecture, exploration, smaller implementations.
Outer loop — not conversational. Check-ins happen in 30-minute to multi-hour increments. The agent runs, hits a gate, asks for input, and runs again. Longer implementation tasks, PR reviews & fixing findings, investigating production incidents & error spikes, recurring tasks (security/architecture drifts, analyzing new CVEs, etc).
Campaign loop — also HITL-based, but the goals span days to weeks. Think: “migrate this service off the old framework,” “harden auth across the platform.” This is a much more advanced concept and the industry is barely starting to figure it out. I won’t focus on it here, but it’s worth naming because it’s coming.
The core principle: same agent harness, same repo harness, and same context across both loops. And every handoff between inner and outer should be warm-context.
There’s one more axis worth getting right: the interaction model for outer-loop agents. Platforms like OpenHands offer both — workflow-style entry points (GitHub labels, Slack mentions, CLI) and a conversational dashboard you can sit in front of. The principle I lean toward: conversation is what the inner loop is for. If I’m going to iterate with an agent in natural language, I’d rather do it on my laptop, where the full repo harness is loaded and the feedback loop is tight.
The outer loop calls for something different: deterministic workflows that incorporate AI as a step, with explicit HITL gates at the surfaces where work already happens — GitHub PRs, Linear tickets, Slack threads. The market is moving this way — Devin Playbooks, GitHub Copilot’s .github/agents/ profiles, OpenHands hooks and skills are all takes on this.
An outer-loop code review workflow could look like this:
On PR open, AI runs the initial review and posts findings as inline GitHub comments.
AI responds to developer questions and “skip this” instructions through the GitHub UI — same surface, no new tool.
AI confirms “this is fixed” comments and does incremental reviews on new commits.
AI auto-approves the PR when conditional gates pass: all findings addressed, risk below threshold, no sensitive modules touched.
One honest caveat on the outer loop: the framework is sound, but the devil is in two related problems. First, HITL design — you can’t just let these agents run unattended, but where exactly do you build in the gates? When should the agent stop and ask vs. push through? The market is actively exploring this — confidence-driven (Devin’s self-grading), single-gate (Copilot’s PR-only review), hook-driven (OpenHands milestones) — each with different trade-offs. Second, context — how does context carry across HITL moments and across the agent’s run so it doesn’t lose the thread between gates? Both topics are meaty enough that I’ll dig into them in a follow-up article.
To summarize - there are three concepts to take away:
LLMs ≠ agent harness ≠ repo harness. The model is the engine. The agent harness (Claude Code, Codex CLI) is the loop around it. The repo harness (CLAUDE.md, skills, MCP servers, tests, linters, etc) is the project context that makes either useful.
Three loops, defined by check-in frequency. Inner loop is conversational, minute-scale — your laptop. Outer loop is HITL-based, 30-min to multi-hour — cloud agents on bounded tasks. Campaign loop is HITL-based, days to weeks — advanced territory, not the focus here. The agent harness, repo harness, and MCPs should be identical across inner and outer.
Conversation in the inner loop. Workflows in the outer loop. Chatting with an agent on your laptop is the right inner-loop UX. Cloud agents should run deterministic workflows with AI incorporated & HITL gates at the surfaces where work already happens.
The market is still organizing around these distinctions. If you’re building or evaluating AI dev tooling, these three concepts are useful axes — both for thinking about your own stack and for the conversations you have with your vendors.
See my next post — Mapping the AI coding orchestrator market — where I apply this framework to the current market landscape for coding orchestrators.