How GitHub Copilot Boosted Developer Productivity

Learn how GitHub Copilot is revolutionizing developer productivity.

By Shannon Prior

The IT Services Application Development & Integration Services Team shared an exciting project exploring how AI could enhance developer productivity. They experimented with GitHub Copilot, an AI coding assistant, aiming to validate its benefits with solid data. Prachi Raheja, the Continuous Improvement Lead from IT Services, introduced the initiative, emphasizing the goal of understanding how AI tools like GitHub Copilot can improve developers' efficiency and satisfaction.

"AI helps developers complete tasks faster, reduces context switching, and boosts satisfaction. But we wanted to go a bit deeper and prove this with data," said Prachi. With this objective, the team defined their problem, set up metrics, and launched a thorough assessment using a Lean Six Sigma approach.

The study spanned eight weeks:

• Weeks 1-4: Development tasks were performed without AI assistance.
• Weeks 5-8: Development tasks were performed with AI assistance.

Data was collected from culture surveys, time tracking logs, project tracking logs, and source code activities from Bitbucket.

Alex Wu, the Director of Application Development, shared the key findings:

• 45% reduction in hours spent per user contribution (code commits).
• 59% decrease in web searches for coding solutions.
• The most pronounced impact was on tasks of moderate and high complexity.
• For an individual developer, 48% less time was spent on high-complexity tasks and 63% less time on low-complexity tasks with AI.

April (blue) is without AI and May (orange) is with AI.

"These results align with industry findings. For instance, Microsoft's 2023 study showed that AI accelerates task completion by 55.8%," noted Alex.

Other findings included a noticeable decline in the frequency of web searches for solutions after introducing AI, and users made more code commits with AI assistance. Higher AI usage was observed for tasks of medium and high complexity, and contributions increased during the AI-assisted period.

Alex demonstrated GitHub Copilot in action, showing how it assists from the initial commenting stage to generating robust code and creating test cases. This live example illustrated how Copilot enhanced the developer experience. "Copilot continuously improves behind the scenes. It is not just generating code. It can actually explain, document, create test cases, and do many different things as well," Alex explained.

(Video has no sound.)

Participants highlighted various benefits of GitHub Copilot, from code generation to documentation and infrastructure scripting. "It's not just about generating code. Copilot helps with configuration, scripting, documentation, and understanding the code you're working with," emphasized Alex. "The integration of AI into development workflows is not just a trend but a transformative approach that will continue to evolve and benefit the industry."

The significant improvements in developer productivity and satisfaction observed in the study also translate into a strong return on investment for IT Services. By reducing the time spent on high-complexity and low-complexity tasks, GitHub Copilot allows developers to focus more on creative and strategic work, potentially leading to faster project completions and more innovative solutions. The reduction in hours spent per user contribution directly correlates with cost savings and increased capacity for additional projects.

Overall, integrating GitHub Copilot represents not only an improvement in workflow efficiency but also a strategic investment that yields substantial economic benefits for IT Services.