Disrupting the industry: GitHub’s Copilot supercharged by insights from Microsoft Azure Data Explorer

In 2021, GitHub—an online software development platform used by hundreds of millions of developers—completely disrupted the tech industry. It launched Copilot, an AI tool for autocompleting code, already used by over a million customers. “We use a specialization of the GPT-3 model which learns by looking at the world’s open-source code,” explains Trevor Carnahan, Senior Director at GitHub. “We enable our users to save 50 percent of their time when writing code.” 

Like any other product, GitHub needs to rely on data to evolve Copilot in the right direction. “We’re trying to measure everything from kinds of prompts and details about code completions to engagement and business metrics,” shares Carnahan. “We’re always running several A/B tests to make the right business and product decisions.” 

GitHub chose Microsoft Azure Data Explorer (ADX), also known as Kusto, as Copilot’s real-time analytics service that would enable the team to obtain insights independently and easily manage access to sensitive data. “With ADX, our developers could become self-sufficient and faster, and quality data and time savings are both critical for AI and data businesses right now,” explains Carnahan. The team also started using Power BI for additional insights and integrated services like Apache Kafka for ingesting data, Airflow for workflow management, Azure Functions for faster coding, and Log Analytics for simpler analytics. 

Embracing novelty to innovate faster

In order to store, manipulate, or retrieve data from databases in ADX, users have the choice of using Structured Query Language (SQL) or Kusto Query Language (KQL). For GitHub, having KQL as an option made all the difference as it was more efficient and easier to use.  

“We held sessions to encourage the team to start using KQL efficiently,” notes Thomas Mahlberg, Senior Software Engineer at GitHub. “Initially, only 30 members of our team were using ADX. Half a year later, we’re up to 500. Once they’ve tried KQL, no one wants to go back to SQL because it’s intuitive.” Carnahan agrees, “Usually, people are resistant to learning yet another language. But as soon as they spend 30 minutes using KQL, they turn into superfans.” 

With near real-time data and an empowered, self-serving team, Copilot can now iterate faster. “ADX has made a huge difference in terms of the time it takes to make decisions, gain insights, experiment, and ship,” says Carnahan. Running queries itself now takes less time, which has even enhanced Copilot security. “Our security analysts used to take several hours to run queries, which can now be completed within seconds,” says Mahlberg. Copilot has also gained the agility it needs to scale further. “ADX is a strong, malleable platform that lets us change the definitions of our key metrics as we evolve,” adds Carnahan. “Without KQL, without insights and constant experimentation and iteration, we have nothing. Copilot has taken off like a rocket ship and it’s had ADX behind it from the beginning.”

As Copilot grows, it plans to bring more workloads into ADX, making it its single source of truth. “Our team is excited to reap the benefits of the transition to a dedicated ADX environment,” says Carnahan. “I think we have yet to observe the full impact in terms of scale, control, privacy, and compliance.”