How GitHub built an internal data analytics agent with Copilot

Title

How GitHub built an internal data analytics agent with Copilot

Content

GitHub deployed Qubot, an internal Copilot-powered analytics agent accessible via Slack, VS Code, and the Copilot CLI, enabling any employee to query the company's data warehouse in plain language and receive answers within seconds, with hundreds of active users executing thousands of queries. Matteo Vasirani, staff manager of software engineering leading product analytics and data science at GitHub, and Cynthia Joseph, senior product manager for the Data team, wrote: "Qubot allows any Hubber to ask questions about any data model in GitHub's data warehouse in plain language and get an answer within seconds." Large data and analytics organizations have struggled for decades to make data access truly self-serve. At GitHub's scale, providing dedicated analytics support to dozens of product teams is challenging. Qubot was designed not as a reporting tool or dashboard replacement, but for exploratory questions such as "Which cohort of users has the highest retention on this feature?" or "What product contributed to move this metric the most last week?" Qubot was built as a Copilot Cloud Agent running on github.com. It has been widely adopted, dramatically reducing the number of questions in data and analytics Slack channels. The team discovered that structured and well-curated context made Qubot three times faster at returning the right answer, making context artifacts a first-class citizen in data modeling. The architecture has three components. The user interface spans Slack, VS Code, and the Copilot CLI, with Slack as the preferred collaboration tool where Qubot instances are spawned as Copilot Cloud Agents and results delivered as markdown reports in pull requests. The context layer uses a federated bronze-silver-gold model: raw events (bronze), conformed facts and dimensions (silver), and curated business-use-case datasets (gold). Context is contributed through a standardized template and ingested by a context agent, then evaluated via an offline framework before shipping. The query engine connects to both Kusto (fast, for exploratory questions over recent data) and Trino (for complex joins and deep historical analysis) via MCP servers, defaulting to Kusto and switching automatically. Qubot removed strain from GitHub's data and analytics team by making product teams own telemetry for their surfaces and business teams own definitions of gold data. The evaluation framework runs each test case multiple times against the agent, measuring completion rate, accuracy, and duration, catching regressions before reaching users. The engineering team included Weijie Tan, Tobias Tschuemperlin, and Vamsi Anamaneni.