Mapping the AI coding orchestrator (software factory) market

Jun 29, 2026

After this post went live I’ve learned that there’s a new term emerging - Software Factories. I like this term - it’s exactly what I meant by coding orchestrator.

At some point in every org’s AI adoption curve, gaining further effectiveness requires a cloud-based coding-agent orchestrator. The work that benefits is the kind that should run without a developer driving — “fix this bug”, “investigate this log spike”, or “ship this feature I’ve already specced”. Solo developers benefit too. But the inherent standardization that an orchestrator brings is more beneficial for larger orgs.

In this post I look at how Stripe, Coinbase, Uber, Meta, and Shopify built their internal AI coding tools. I also look at how they compare to on-the-market AI coding tools. I was surprised to find out that there’s a clear division between internal vs. on-the-market tools.

The technical language used in this article — models vs agent harnesses vs repo harnesses, inner vs outer loops — is something I laid out in a previous post. This post is a survey of sixteen orchestrators (all that I could find) analyzed along these axis.

What counts as an orchestrator

There are several categories that got swept up in my initial broad search. These are all excluded from my analysis:

Coding agents (Claude Code, Codex CLI) — also called the agent harness. They aren’t orchestrators themselves.
Point-solution tools offer a single specialized capability — code-review tools (CodeRabbit, Greptile), bug detectors (Cursor Bugbot). One scope, one job.

What’s left is the layer that takes an intent — “fix this bug”, “investigate this log spike”, or “ship this feature I’ve already specced” — and runs it through one or more coding agents to completion. Sixteen of them have public details good enough to analyze.

On-the-market (8): Conductor, Sculptor, Nimbalyst, OpenHands, Warp Oz, Augment Code, Factory Droid, Devin Desktop.
Internal (8): Stripe Minions, Block Builderbot, Coinbase Mux + NodeSmith, Shopify Roast, Uber LangFX, Meta Confucius, Google’s migration pipeline, Airbnb’s batch migration pipeline.

Findings, by axis

Major findings

Two patterns dominate the dataset - the workflow model and the agent harness support.

Two workflow models

All orchestrators break into two workflow models:

Handoff orchestrators. Non-deterministic. The orchestrator hands work off to the LLM via a single prompt or skill; the LLM controls 100% of the flow from there on.
Pipeline orchestrators. Deterministic. The devs design the pipeline as explicit steps, interleaving calls to a coding agent and HITL (Human-In-The-Loop) gates between deterministic execution. The orchestrator runs the pipeline as a state machine.

Here’s the interesting part. The handoff / pipeline split is almost perfectly the build/buy split.

Internal platforms cluster on the pipeline side. Stripe Minions, Shopify Roast, Uber LangFX, Meta Confucius, Coinbase NodeSmith, Google’s migration pipeline, Airbnb’s migration pipeline — all of them. Stripe’s Blueprints alternate deterministic and agentic nodes; Shopify’s Roast is a Ruby DSL with named cogs; Meta’s Confucius is a DAG with HITL gates between stages. The pipeline languages vary — Ruby (Roast), LangGraph/Python (LangFX), bespoke DSL (Minions), state-machine-in-code (Airbnb) — but the shape is uniform: deterministic steps with LLM and HITL nodes interleaved. The rest fit the same shape — see appendix for the per-tool teardown. Block’s Builderbot and Coinbase’s Mux look like handoff at first glance but sit on top of internal pipeline infrastructure; even where internal orgs ship handoff dispatch, what’s underneath is pipeline.

On-the-market orchestrators cluster on the handoff side. Conductor, Sculptor, Nimbalyst, Warp Oz, OpenHands, Devin Desktop. Each dispatches an agent and lets it figure out the steps.

Two on-the-market products break the handoff pattern. Factory Droid ships Custom Droids you author; Augment Code ships a coordinator + specialist + verifier architecture with explicit scope-approval HITL. Both target enterprise structured workloads. Both expose real pipeline design.

Agent harnesses

How does the orchestrator relate to the coding agent it dispatches? Three patterns:

Proprietary agent harness. The orchestrator ships its own. Block built Goose, Meta built Confucius, Stripe forked Goose into Minions. On-the-market, Factory Droid runs Droid and Augment Code runs Auggie underneath. The org owns and maintains its own agent harness and uses raw LLM models from LLM vendors.
BYO direct. The orchestrator dispatches a third-party agent harness untouched — native execution, no interception. Internally, Airbnb runs Claude Code as-is and Coinbase mixes harnesses through Mux. On-the-market, Conductor, Sculptor, Nimbalyst, OpenHands, and Devin Desktop all pass the chosen harness through; ACP (the Agent Coordination Protocol) is the standard underpinning the latter two.
BYO wrapped. The orchestrator runs the chosen agent harness inside its own context/memory layer. Warp Oz adds “Agent Memory” and “Skills” on top of execution; Augment Code layers a structured context including system prompts and workspace metadata. The wrapper can degrade the inner harness’s context — Augment Code’s own docs explicitly disclose the added context. Verify before depending on it.

Internally, six of the eight platforms in the dataset have consolidated on one agent harness across the org: Goose at Stripe and Block, Claude Code at Anthropic and Airbnb, Confucius at Meta, LangFX at Uber. Polyglot agent harnesses are rare inside.

On-the-market, the 8 products split into 5 BYO direct, 2 BYO wrapped, and 1 Proprietary (Factory Droid).

Supporting patterns

Smaller findings that reinforce or color the major split.

HITL surfaces

How do humans interact with the orchestrator? That’s mostly settled - Slack, Linear, etc. The chat-in-dashboard anti pattern is essentially dead.

When do humans interact with the orchestrator? That’s a deeper question with three approaches in the wild:

Confidence-driven. The agent grades itself, a verifier sub-agent grades (Meta KernelEvolve, Augment Code’s verifier role, Devin’s auto-review), or the user sets sensitivity dials (Cursor Bugbot effort levels).
Single-gate. One HITL event, otherwise autonomous. Two variants worth distinguishing: upfront plan-approval (Cursor’s long-running agents, Augment Code scope approval, Devin’s preflight checklist) and exit-only PR review (Copilot Coding Agent, Stripe Minions, OpenHands headless). Upfront catches direction errors before compute spend; exit reuses existing review process.
Hook-driven. The adopter writes scripts or config that fires at named execution milestones — pre-plan, pre-tool-call, post-tool-call, pre-commit. The script can halt the agent (exit 1), modify state, or run guardrails. OpenHands’ hooks.json is the canonical declarative-registration version; Claude Code ships its own hook system. Hooks are the lowest-level, most flexible HITL surface — you write code and it runs.

The three approaches aren’t equivalent in coverage. Confidence-driven is opaque to the adopter — you tune indirectly by enriching context, not by setting thresholds. Single-gate is binary — one ask, then nothing until the PR. Hook-driven is fully programmable but pushes the work onto the user and offers no built-in human-prompt UI.

Loop types (and the new campaign loop)

All sixteen orchestrators target outer-loop work — the developer hands off and checks back. Inner-vs-outer isn’t an interesting axis in this dataset; orchestrators exist for outer-loop work by definition. What IS interesting is a third loop the binary missed: the campaign loop.

Campaign-loop work is autonomous multi-day-to-multi-week execution against a goal. The developer doesn’t check back in any meaningful cadence; the orchestrator owns long-running state.

Uber Shepherd runs multi-day migration campaigns across services.
Meta REA has a “hibernate-and-wake” pattern that bridges multi-day training waits inside one logical run.
Factory Missions have a median around two hours, with 14% running over 24 hours and the longest recorded at 16 days.
Google’s migration pipeline has run multi-month migrations across 149 teams.

These aren’t outer-loop with a slow review cadence. The orchestrator is driving goal-with-stopping-criteria rather than a task-with-handoff. What makes the loop distinct is the durable state — these orchestrators persist work across host / service boundaries because no single machine outlives the campaign.

Most orgs aren’t close to needing campaign-loop tooling, so I won’t focus on it for the rest of this post.

What else stood out

Repo harness. Every orchestrator in the dataset respects the standard repo-harness files — AGENTS.md, CLAUDE.md, .cursorrules, and friends all coexist. No tool earns the “imposes” verdict.
Audience. Individual-developer tools (Conductor, Sculptor, Nimbalyst, Warp Oz, Devin Desktop) all sit on the handoff side. Org-wide tools (Factory Droid, Augment Code, all internal platforms) all sit on the pipeline side.
BYO harness orchestrators are a real 2026 category. Most on-the-market products now let the user swap agent harnesses. ACP and broad MCP adoption made the swap technically straightforward by mid-2026 — with the caveat that wrapping can silently degrade the inner harness.

The synthesis

The market is bifurcated along the build/buy boundary. On-the-market: handoff dispatch with BYO harnesses (7 of 8 products). Internal: pipelines with Proprietary harnesses (6 of 8 platforms).
Internal engineering teams are using this framework’s vocabulary. Coinbase explicitly segments inner loop vs. outer loop in its own engineering docs. Anthropic and Shopify use the same distinction internally. Three independent organizations adopted this mental model before my framework post was published.
Factory Droid & Augment Code are the only on-the-market pipeline orchestrators. But neither supports Claude Code or Codex CLI. Factory Droid runs Droid — its own agent harness (you can use any LLM model). Augment Code runs Auggie underneath, also Augment’s own agent harness with its own model routing.

This leaves us with some questions…

The insights from this research left me with some new questions:

Why does the handoff/pipeline bifurcation track the build/buy boundary so closely? Is it just headcount — bigger orgs build internal platforms — or is something else pulling the architectures apart?
Similarly is Proprietary agent harness vs. BYO here to stay? Are Proprietary harnesses beneficial? Will they survive?
What would be an ideal orchestrator that will scale for the next 3-5 years? I don’t think any of the on-the-market orchestrators are it. So what would it take?

Addendum

The matrix and vendor blurbs below is the raw data behind the synthesis. And all tools are linked to their primary source.

How to read the matrix

Build/buy — On-the-market (commercial or OSS) product vs. internal platform. The build/buy split is the central organizing distinction in the post.
Workflow model — Handoff = one prompt, agent controls the flow. Pipeline = explicit deterministic + LLM steps wired by the adopter.
Agent harness — Proprietary = built and owned by the org/vendor. BYO direct = third-party harness run as-is. BYO wrapped = third-party harness inside a vendor context/memory layer.
Model — Open = any LLM, BYOK. Fixed = vendor picks.
HITL — Three approaches from the post. Confidence = agent grades itself. Single-gate = one HITL event, otherwise autonomous; variants noted as (upfront) plan-approval or (exit) PR review. Hook = adopter-authored scripts at named milestones.
Audience — Individual = solo dev installs. Org = deployed as a platform. Both = packaged for either.

The matrix

On-the-market tools

Conductor (commercial)

Free macOS desktop app (Apple Silicon only) from a YC S24 company that raised a $22M Series A. Spawns parallel Claude Code / Codex CLI / Cursor sessions in isolated git worktrees, each with its own branch, terminal, and Diff Viewer. Adds no agent logic of its own — thin orchestration shell over the underlying harnesses; users bring their own subscriptions. The canonical “outer loop defined by attention, not execution” example.

Sculptor (OSS)

Open-source Mac/Linux desktop app (Electron) from Imbue. Wraps Claude Code and Codex in Docker containers (vs Conductor’s worktrees) for tighter isolation, with a Pairing Mode that bidirectionally syncs container files into the developer’s IDE for live co-editing. Explicitly labeled “experimental research preview”; MCP and Sculptor-layer CLAUDE.md handling still roadmap.

Nimbalyst (OSS)

Stravu’s MIT-licensed successor to Crystal (deprecated Feb 2026). Cross-platform desktop (Mac/Win/Linux + iOS companion) for parallel Claude Code / Codex / OpenCode / Aider sessions in isolated worktrees. Adds a session kanban, inline red/green diff review, and an extension SDK with ACP alpha for Copilot. Very young (open-sourced April 2026); team tier not yet GA.

OpenHands (OSS)

All Hands AI’s MIT-licensed agent platform, 78K+ GitHub stars. Docker-sandboxed CodeAct agent with LiteLLM model abstraction (100+ providers). Five surfaces: Agent Canvas GUI, headless CLI, GitHub Action triggered by fix-me labels, Slack, and ACP for IDE embedding. The cleanest external zero-chat outer-loop path; .openhands/hooks.json is the canonical declarative hook-driven HITL primitive in the dataset.

Warp Oz (commercial)

The first multi-harness cloud-agent control plane: runs Warp Agent, Claude Code, and Codex side-by-side under unified governance and credit billing. Triggers via Slack @Oz, Linear issue assignment, GitHub Actions, cron, or webhooks; HITL surfaces are native to the triggering tool. Internal stat: 60% of Warp’s own PRs by Oz agents. Warp terminal open-sourced May 2026.

Augment Code (commercial)

Three-surface platform — Cosmos (IDE plugin), Auggie CLI, Intent (macOS desktop orchestrator) — anchored on a proprietary Context Engine indexing 400K+-file monorepos. Intent’s BYOA wraps Claude Code / Codex / OpenCode in a coordinator + specialist + verifier architecture with explicit scope-approval HITL. Warning: BYOA agents get degraded Context Engine access vs first-party Auggie, Augment’s own docs disclose it.

Factory Droid (commercial)

Multi-specialist outer-loop platform on a proprietary harness (Droid). A Delegator decomposes work and dispatches to Code / Review / Knowledge / Reliability / Product / Tutorial Droid sub-agents. Flagship “Missions” feature is the only external product with campaign-loop duration: median ~2hr, 14% over 24hr, longest recorded 16 days. $1.5B valuation April 2026; reads AGENTS.md only (Factory co-authored the standard with OpenAI).

Devin Desktop (commercial)

Cognition’s June 2026 rebrand of the Windsurf acquisition. Agent Command Center kanban manages local + cloud + third-party (ACP-hosted Codex / Claude / Gemini / OpenCode / Junie) agents from one surface. The fleet-management piece of Cognition’s product line, separate from Devin Cloud (excluded as a coding agent, not an orchestrator). Devin 2.1’s confidence scoring (🟢🟡🔴) is the canonical confidence-driven HITL.

Internal

Stripe Minions

The prime empirical example of the post’s thesis. Engineer tags a Slack bot → pre-warmed AWS devbox containing Stripe’s 30M-line monorepo spins up in under 10 seconds → Toolshed (Stripe’s ~500-tool MCP server) provides context → a Blueprint alternates deterministic steps (git, lint, CI) with agentic LLM nodes → PR opens, CI green, no human in the loop. 1,300+ PRs/week merged. Built on a fork of Block’s Goose with interruptibility and confirmation prompts deliberately removed.

Block Builderbot

Block’s internal orchestration layer on Goose (which Block built and donated to the Linux Foundation Dec 2025). Vertically integrated stack: Anthropic Claude → Block Goose → Anthropic MCP → Block Builderbot. Slack-native @builderbot dispatch + Linear/Jira pickup; live thread updates while the agent works across the monorepo (Cash App, Square, all subsidiaries). ~1,500 PRs/week merged, ~15% of all production code changes.

Coinbase Mux + NodeSmith

Coinbase explicitly segments inner vs outer loop in its own engineering docs — direct framework-vocabulary match. Forge is the Slack/Linear/GitHub-native task agent (1,000+ engineers, 5% of merged PRs, 150h→15h cycle time). Mux layers parallel-agent orchestration on Forge (600+ users, 5,068 merged PRs across 461 repos in one month). NodeSmith is a two-phase pipeline for blockchain node upgrades — LLM Triage Agent + Upgrade Orchestrator with 5 specialized sub-agents blending LLM reasoning with deterministic Python. OpenAI-compatible model router used daily by 1,500+ engineers; explicitly multi-vendor.

Shopify Roast

Two-layer stack. Genie is an internal LLM proxy that sits under every AI tool Shopify uses (Claude Code, Copilot, Cursor, Codex) — PII masking, prompt-injection detection, cost analytics, model switching, plus 24+ internal MCP servers exposing Vault, GSD, data warehouse, Salesforce, Slack, Google Workspace, GitHub, Figma. Roast is the Ruby DSL workflow framework on top: named cogs (chat, agent, ruby, cmd, map, repeat) interleave deterministic code with LLM calls. Open-sourced June 2025. Tobi Lütke’s April 2025 memo: “reflexive AI usage is now a baseline expectation.”

Uber LangFX

Full-stack internal AI dev suite for ~5,000 engineers on a multi-hundred-million-line monorepo. Three layers: end-user tools (uReview, AutoCover, Minion, Shepherd, Validator), agent framework (LangFX — opinionated LangGraph wrapper integrated with Michelangelo), infra (AIFX CLI, MCP Gateway exposing 10,000+ internal services as MCP tools, Agent Builder). 60K agent tasks/week. 92% of devs use agents monthly; 11% of PRs by agents. Multi-vendor model strategy benchmarked per task (Claude-4-Sonnet + o4-mini-high for uReview was optimal).

Meta Confucius

Specialist-per-domain pattern on a shared internal harness (Confucius — ~2 years old, 60+ applications, DAG workflows + HITL gates). REA owns the ML experimentation loop for ads ranking (hibernate-and-wake across multi-day training waits). KernelEvolve owns GPU/MTIA/AMD kernel optimization (60%+ inference throughput on Andromeda; ISCA 2026 paper). Tribal Knowledge Agents produce ~59 concise context files per codebase (40% fewer tool calls per task downstream). Capacity Efficiency owns perf regression detection + fix generation.

Google migration pipeline

Fully vertically integrated stack: Cider V (VS Code fork on Borg, indexes all of Google3 monorepo) + Gemini fine-tuned via DIDACT + Kythe code index + Critique review tool as the HITL surface. The migration pipeline uses Kythe BFS for deterministic site discovery, fine-tuned Gemini for the diff, deterministic validation cascade (whitespace → AST → build → tests), then per-CL human review in Critique with OWNERS-routing (effectively protected-code HITL via Google’s existing ownership graph). Example campaigns: int32→int64 proto migration (595 CLs, 149 teams, 12 months); JUnit3→JUnit4 (5,359 files, ~149K LOC, 3 months). Pichai April 2026: 75% of new Google code AI-generated.

Airbnb migration pipeline

The “rent the agent harness, build the repo-harness layer” pattern. Off-the-shelf Claude Code as the harness, custom internal MCP servers as the context layer — distinct from Stripe (fork) and Block (own). Two surfaces: engineers running 3–5 parallel Claude Code CLI sessions on the monorepo, plus a standalone batch migration pipeline (per-file state machine: refactor → fix jest → fix lint → fix tsc → complete; LLM invoked only on gate failure) that migrated ~3,500 Enzyme test files to React Testing Library in 6 weeks (97% automated, 3% manual long-tail). Chesky Q1 2026: 60% of new code AI-written.

Jack Kora

Discussion about this post

Ready for more?