This blog post focuses on Microsoft’s Copilot product however some of the concepts I discuss here can be applied to any Copilot product or offering. As such, in an attempt to keep things clear where I am discussing something purely in the context of Microsoft’s Copilot I shall refer to it as “MS Copilot” while more for generic discussions I’ll simply refer to “Copilots”.
The explosion of Copilots
AI is undoubtedly proving itself to be a massive disruptive force into businesses (and society in general) right now. One of the ways this is most visible is through the introduction of Copilots, they are popping up everywhere.
For those living under the metaphorical rock for the past few months let’s just take a moment to explore what I mean when I say Copilot. Needless to say, I am not talking about the second/relief pilot of an aircraft, I’m not an expert but those have been around for decades and are not major disruptive forces in businesses.
No, the Copilot definition I’m talking about hasn’t even made it into the dictionary yet and is the term that is being applied to Generative artificial intelligence chatbots that use large language models to reason over data to produce text and other data outputs in response to the users input.
One of the biggest copilots in the context of Businesses is the development and introduction of Microsoft Copilot, previously known as Microsoft 365 Copilot. At NewOrbit we’re starting to make great usage of MS Copilot to find and reason with the vast amount of data we hold as a business within the Microsoft 365 systems we use such as Teams messages, Outlook emails and Office documents in Sharepoint.
Microsoft Copilot extensibility
But why should we stop at being able to search through information stored in our Microsoft 365 services and heck why can’t we make MS Copilot do stuff for us; Microsoft proudly documents how MS Copilot can be extended with additional capabilities, so how hard can it be?
Well for anyone who has read the above documentation you’ll be aware that the options for extending MS Copilot are numerous and it’s easy to get a bit lost.
In this first blog in a series of blog posts join me as I outline how we’ve extended our MS Copilot to include data from the project management application we use, TargetProcess, similar to Jira that many readers will be familiar with. I’ll go over the anatomy of the tooling we’ve used to achieve this and demonstrate and explain why, in our view, they are a great first step to introducing your business and it’s data to the world of Copilots as well as relatively cheaply allowing you to evaluate if further investment might be worth it.
Graph connectors
MS Copilot can be extended with additional data via something Microsoft call a “Connector”, for this blog post we’ll be focusing on Graph Connectors. Graph Connectors allow you to inject unstructured documents and data into Microsoft 365. MS Copilot can then query and reason over this data and provide reference to this data in it’s responses; and the goodness does not stop there. As I explained Graph connectors inject data into Microsoft 365 but MS Copilot is not the only consumer of the data and the data will surface itself more widely within Microsoft 365, most notably in Microsoft Search which can be highly useful in business where some employees are not given a MS Copilot license.
API or Graph connector agent
One thing you might find when you start experimenting with creating your own custom graph connectors is that there are two main ways to build these, either via a Graph connector agent or via the MS Graph API.
The Graph connector agent is an application you install on a Windows PC or Server alongside a windows service you write yourself and the agent then orchestrates with your windows service and arranges when data is synchronised with Microsoft 365 and can simplify the set up and configuration of the connectors.
An alternative to using a graph connector agent is to utilise the APIs within the Microsoft Graph. With this approach, just like you did previously in the windows service you need to write code to manage the data that will flow into MS 365 however unlike previously you are not restricted to windows so you can deploy in a wider variety of places plus you take on the burden of orchestrating when data does and does not get synchronised into MS365.
At NewOrbit we make extensive use of Azure Cloud offerings and as such we don’t have servers where we can install the connector agent and while you can create VMs in the cloud we avoid this, instead preferring the cheaper SaSS and PaSS offerings within Azure. Given that we are much more interested in following the Graph API approach described above and is the approach that this blog will set out and describe although some of the things I’ll discuss do apply to the connector agent so do stick around.
Anatomy of a Connector
Microsoft provide a great demonstration tutorial for building a console app based C# application which can create and manage the connector/connection entities with the Graph API so I shall not repeat those instructions here. We used this tutorial to create a simple console app which when runs, creates the connection in MS 365, sets up the schema for the data and then pulls user stories from TargetProcess and pushes (or updates) these into the graph api as an item. What we can then do is hook this console app up to a schedule of some kind and periodically run it to start getting a feel for how it works once the data in in place but before all that let’s explain what a “connection”, a “schema” and and “item” is in the context of all this.
Connection
A connection (aka externalConnection) is simply the entity you need to create in the graph to define the connector.
Schema
The schema belongs to the connection and defines the structured elements on your documents such as the name, id, created and modified dates and original url to the document. In addition it describes how MS 365 should index and query the data.
Item
Finally a connection has a number of externalItems. These are, as the name would suggest, the items that are being imported from the external system, in our case from TargetProcess. When you create an item you set it up with all the properties you defined previously in the schema plus some free text content in HTML or Markdown, this is the main unstructured content of the document. You can also attach control information to ensure that only authorised users will be able to query this data at a later point.
Loading and viewing the data
With all that context we followed the tutorial to create a console app which could pull data from the TargetProcess API and push it into Graph. We hosted this console app as a scheduled CRON job in Azure Container apps as a simple way to repeatedly keep the data up to date.
Next we exposed this newly loaded data in MS 365, this, it turns out can be exposed in a handful of ways via what Microsoft call experiences. Obviously we wanted to expose the data in MS Copilot but graph connectors but Microsoft Search was also an interesting extensibility point. Each experience has certain requirements to ensure that the data is visible and optimally structured, for us MS Copilot has the most in depth requirements around which fields should be included in the schema as well as providing a urlToItemResolver which allows MS copilot to convert a url pasted into it’s chat into the external item it links to.
How does it perform
With the data in the system and rigged up, what do we make of Graph Connectors has a way to extend MS Copilot
It extends more than MS Copilot
So one of the biggest wins I see from this, perhaps unintuitively for a Blog post about MS Copilot extensibility is it’s ability to extends other Microsoft 365 services, in particular Microsoft Search. At NewOrbit, while we make extensive use of MS Copilot, not all employees have access to it because we find they generally have no particular usage for it. Developers are the prime example of this, generally speaking they will get much better benefit from other Copilots such as Github Copilot and as such they have no access to MS Copilot. But with Graph Connectors they can still have access to this additional data and can gain from it’s offerings. Often at NewOrbit we’ll put together initial requirement documents or be sent documents relating to upcoming requirements from Clients which get stored in Sharepoint and being able to search for key words and have these documents come back alongside more developed requirements from TargetProcess is a big plus for all our employees, especially when we don’t realise the information we’re looking for might be in both places.
MS Copilot power
For employees who do have access to MS Copilot the data can be even more powerful with MS Copilot and its LLM adding additional context to the data we’ve imported into the Graph allowing it to be easier to consume and digest. In addition being able to ask MS Copilot to summarise & comment on items in TargetProcess directly as well as alongside items within Microsoft 365 more generally can provide additional insights that might not otherwise exist.
Context unawareness
While having this data added into the Graph is proving a powerful tool it’s worth understanding that it’s not perfect and it has its limitations. The biggest limitation is that MS Copilot (and MS Graph in general) cannot fully understand the full context of the documents you upload. You can give the connection a description when you create the external connection via the Graph and this will give everything some context but it’s limited to a small 512 characters and cannot easily be used to express the full context of the data you’re importing and how it links to other data. This ultimately means that MS Copilot cannot always give you the best results and struggles to link data together from various sources.
Lack of index control
To compound this when we load data into the Graph API we are relying on the API to index the data for later retrieval by MS Copilot and its LLM. This means we’re relying on Microsoft indexing the data in the most efficient way - an impossible task for data that could be anything we would like it to be.
Our CTO Frans has recently posted about how the way you index data for LLMs can dramatically change the way data is found and reasoned within Copilots and the lack of this flexibility is a massive limitation here.
Ease of setup
Having said all this setting up a connector was very simple and we have been able to load our TargetProcess data into MS 365 relatively quickly in order to produce this evaluation and while not perfect it has provided significant gain to our business with little investment and paves the way for our next steps.
Next steps
So where do we go next with this Graph Connector? For starters it’s given us something useful and exciting so we’re planning to keep it but there are some things we’ll be considering to make it more useful.
Adjusting the index content
For the initial demonstration of adding TargetProcess data to the Graph we’ve selected a number of different pieces of data from TargetProcess that we think will be useful. Much of this data we’ve formatted into the main text content of the externalItem which is used by MS Copilot. We think there is likely scope to experiment with this data to improve the responses MS Copilot (or another copilot) can produce but this will take more time to perform these experiments.
In addition there are extensive recommendations and suggestions regarding how to set up the schema and items to “make sure that Microsoft 365 Copilot uses your content effectively”.
Addition of externalActivities
One of the biggest recommendations we’ve not yet experimented with at all is the concept of externalActivities. These are an extension of the externalItems and allow you to inform the graph when the item was interacted (created, modified, viewed & commented) with which gives MS Copilot a hint as to how important something is because it’s been interacted with recently. Unfortunately externalActivities can only be created via the API and cannot be read. Therefore in order to properly manage these additional entities we’ll add an additional state store to our code so that it can track when updates have been updated and when they have not.
Simplified admin experience
A minor improvement but going forward we’ll also be working to improve the administration of the Graph Connector using the simplified admin experience whereby you can package the graph connector as a Teams plugin and allow administrators to enable and disable the connector at will as well as offering the potential to package it alongside other related teams and MS Copilot extensibility components. To make use of this simplified admin experience the current basic console app will need to expose a webhook endpoint that will allow teams to enable and disable the connector.
Custom Agent
While extending MS Copilot has been a quick and easy way to give our data to the MS Copilot bot and it has allowed us to evaluate and test the potential value in extending MS Copilot with our businesses data as I’ve described the default MS Copilot is constrained by it’s lack of understanding of the data it’s dealing with plus how it might utilise and reason over that data.
With all this in mind one potential option is to write a custom agent whereby we can write our own specialised version of MS Copilot which we can design such that it can specialise to the context of TargetProcess or the NewOrbit business in general. Writing a custom agent will be a whole other blog post(s) so watch this space.
Closing remarks
As you can hopefully see, Graph Connectors can, despite being constrained in some places, be a cost effective way to introduce your business and it’s data into the world of LLMs and Copilot agents as well as evaluate if there is value in investing further resources into improving the quality of the data you are loading in or investing in a more advanced copilot agent.
We’d love to hear what data your business might have which you’d like to explore the potential of adding to MS Copilot and talk about how NewOrbit might be able to help. Please get in touch.