A Dev's Initial Experience Using GitHub Copilot

Intro

Here at NewOrbit we're constantly looking for ways to learn new skills, improve our existing knowledge, and generally embrace change in a way that benefits our clients.

A current hot topic driving such a change is AI, and more specifically generative AI and large language models, with many big players including OpenAI, Google, Microsoft, Amazon, and Meta being heavily invested in this area.

The current most popular example of these is likely ChatGPT, a free-to-use chat bot, giving the general public the ability to receive nuanced answers to detailed questions, or generate rich content such as scripts, novels and images based on descriptive prompts.

Another practical application of these advancements is GitHub Copilot, which is an AI pair programmer designed to assist developers write better code faster.

Whereas such functionality was previously restricted to large corporations due to the sheer investment needed in terms of time, domain knowledge, and capital due to the infrastructure required to train and ultimately host such a system, now anyone from small teams down to an individual developer can harness this power in any way they choose.

GitHub Copilot

GitHub Copilot can be accessed directly within your chosen IDE, with current support for Visual Studio, VS Code, JetBrains and Vim. In addition, GitHub Copilot Chat is an interface allowing you to interact with Copilot and provide extra context or ask specific questions.

As a Microsoft Solution Partner, we gained early access to this tool and were encouraged to gain hands-on experience to evaluate its effectiveness: how it can be integrated into our daily workflows, where it excels (and sometimes falls short), and ultimately whether it can aid us in delivering innovative systems and products.

The Good

In a recent project, I found Copilot to be an invaluable tool in expediting the often tedious and error-prone task of data transformation.

I was assigned the task of migrating a large volume of data from a legacy MS Access DB to Cosmos DB. When done entirely manually, this would have required identifying each property in the existing data that was needed in the new system and then performing the necessary conversions to make it compatible with Cosmos DB, e.g. changing data types and re-structuring hierarchical data. When dealing with 100's or 1000's of properties, it's not uncommon to overlook something on your first pass.

However, with Copilot, the majority of the work could be accomplished with just a few basic prompts and examples. I was able to simply provide a snippet of the source data, along with a sample of the desired model shape, and essentially ask it to transform from A to B. Within minutes I had a script that could read a source CSV, map the legacy properties to their new naming conventions, types and structure, and output JSON files ready for direct import into Cosmos DB.

Another area in which I've found Copilot to excel is in the generation of unit tests. This is another task that can sometimes feel slow and tedious, as you work through all the branches in your newly written method, making sure you take into account all possible variations and exceptions. With Copilot you can simply highlight your code and, instantly, these are generated for you.

I've also seen huge time savings when I've gone to use an API or 3rd-party library for the first time. Sometimes, these are really well documented and you can find what you need within minutes, but that's definitely not always the case. In turn, that can lead to time spent scouring the internet for anecdotal experiences and troubleshooting where you really hope someone else wants to achieve the exact same as yourself or has solved that specific problem only you seem to be encountering. Copilot can drastically reduce the time needed to get you moving again, through essentially ingesting all the documentation and previous experiences across the web, you can arrive at an answer sometimes immediately.

The Bad

As with all productivity tools, GitHub Copilot isn't a one-size-fits-all experience, with some developers simply not wanting to trust it to help write their code, and others having a bad experience that dissuades them from fully integrating it into their daily workflow - I've had my share of both scenarios.

One such negative experience is the lack of accuracy with some of Copilot's suggestions. There have been instances where the suggestions were not just slightly off, but completely incorrect. For example, it would suggest code that was syntactically incorrect or did not adhere to the best practises of the language I was using at the time. There are other instances where crucial parts of my prompt were overlooked, which resulted in a flawed suggestion that didn't meet my requirements. These experiences can be quite jarring and can lead to a loss of trust in the tool's capabilities.

In addition to not always being accurate, Copilot can at times be slow with autocomplete suggestions. This has the potential to interrupt your coding flow, thereby actually hindering your performance, rather than enhancing it.

Summary

The past few months of using GitHub Copilot has generally been a positive experience for myself, with previously cumbersome and error-prone tasks becoming quick and painless, confusing legacy code being explained in plain and simple terms, and long sprawling documentation being summarised to just the parts you need.

Whilst acknowledging these benefits, it's always paramount to remember that, despite how accurate and helpful the responses can sometimes be, they are AI-generated and as such should always be checked and validated. As the developer you know your code, the context, and the overall goal best. While it could be tempting to blindly paste that 100-line function that looks like it does exactly what you want and move onto the next task, you should always be asking yourself questions such as "do I understand this code?" and "could I have written it myself?" If the answer to these is ever "no" then that should be your warning sign to pause and turn those into a "yes" before continuing.

Ultimately, I've found GitHub Copilot to be an extremely useful tool when used in the right situations and with the understanding that it can be wrong. By knowing when, and how to use it, I believe it has the potential to enhance a developer's skills and productivity.

As with most technology, AI is developing at a breakneck speed and as such there will be many updates and improvements to such tools in the months and years to come. I encourage people to try these tools, see if they work for you and if not, revisit later down the line once such updates have been released. It's exciting to see where this leads!