Orchestrating AI Code Review at scale

Title

Orchestrating AI Code Review at scale

Content

Cloudflare deployed a CI-native AI code review system built on OpenCode that reduced the median wait time for first code review from hours to near-instantaneous across tens of thousands of merge requests, using up to seven specialised AI agents per review coordinated by a single orchestrator. "We've been running this system internally across tens of thousands of merge requests. It approves clean code, flags real bugs with impressive accuracy, and actively blocks merges when it finds genuine, serious problems or security vulnerabilities," said Ryan Skidmore, Software Engineer at Cloudflare. The system is part of Code Orange: Fail Small, Cloudflare's broader engineering resilience initiative. Code review is essential for catching bugs and sharing knowledge but remains a primary bottleneck. Cloudflare's median wait for a first review was measured in hours. Third-party AI code review tools were insufficiently flexible, and a naive LLM summarisation approach produced "a flood of vague suggestions, hallucinated syntax errors, and helpful advice to 'consider adding error handling' on functions that already had it." After rejecting off-the-shelf tools and naive prompts, the team built around OpenCode, an open-source coding agent. Instead of a monolithic agent, they created a composable plugin architecture: a coordinator process running via Bun.spawn delegates reviews to up to seven specialised agents covering security, performance, code quality, documentation, release management, and compliance with the internal Engineering Codex. The coordinator deduplicates findings, judges severity, and posts a single structured review. The architecture uses a ReviewPlugin interface with three lifecycle phases: non-fatal bootstrap hooks (concurrent), fatal configure hooks (if the VCS provider cannot connect, the job stops), and post-configure hooks for asynchronous work. Plugins contribute through a controlled context API. The roster includes GitLab VCS provider, Cloudflare AI Gateway for model tiers and failback, Codex compliance, and Braintrust distributed tracing. OpenCode runs server-first with sessions created via SDK using JSONL events on stdout, with a 2.5 GB heap cap and Bun as the runtime. The system runs across thousands of Cloudflare repositories in CI/CD, launching multiple concurrent reviewer sessions per merge request. The coordinator posts structured, deduplicated feedback, blocking merges only for genuine severity issues. Cloudflare engineers have contributed over 45 pull requests upstream to OpenCode, and the plugin architecture allows new agents to be added without modifying core code.