Unlocking Integration Testing With a Fake OIDC Identity Service

At NewOrbit we develop a variety of different software applications and solutions for a wide variety of clients and we're always looking to implement automated integrations into these solutions to make developing these solutions quicker, easier and safer but authentication has often proved to be a big barrier to implementing automated tests fully. That is, until now.

In this blog post I'll dive into how we've recently made big strides into removing this barrier and unlocking the huge potential of automated tests for systems that rely on authentication. Although the examples are specific to our Microsoft-based technology stack, the principles and code samples aim to simplify the core concepts, providing useful information that you can adapt to your own stack. So, even if you're from a different industry, I encourage you to continue reading.

(The core of) The problem

Let's start by outlining the problem; what exactly is the "barrier" I am talking about here and why does authentication cause such a massive headache.

Modern software Authentication and Authorisation

The first thing to outline is how most applications these days will delegate authentication and authorisation within a system out to a third party service. This is done for two main reasons. Firstly, it can often make things slicker for the user who may be able to use an existing set of credentials. And secondly, the application development team can rely on the developers of these third party systems, who are likely to have more expertise in keeping up to date on the latest security. At NewOrbit our third party service of choice is Microsoft Entra or Microsoft B2C. I will make some specific mention to these two products throughout this post but as I will also explain it's not important that we've chosen to use these services specifically, we could equally use any of their competitors such as Okta or Auth0 and the problem and solutions that I am to describe here all still apply, I'll cover why it does not matter later on.

A Demo

To help explain the concepts in this blog post I've put together an example repo. Let's start by taking a look at commit 22745e62a70471592d83d8c6cb83efab4927b3d6. This commit sets up a very simplistic project with the following components:

• A "MockOidcApp.Api" directory which is a small C# API project with an endpoint that returns the weather forecast for the next 5 days all secured with Entra using JWT Bearer tokens
• A "MockOidcApp.Vite" directory which contains a small Vite + React single page app which is set up to authenticate with Entra via OpenIdConnect and then use the resulting access token to call the weather forecast api
• A "MockOidcApp.SpaIntegrationTests" directory which contains a playwright project with a couple^* of tests to test the UI, both when the user has not logged in and when the user has logged in
• The final main component is the usage of Microsoft Aspire in the "MockOidcApp.AppHost" & "MockOidcApp.ServiceDefaults" directories which provides a convenient way to run both the api and spa projects together with the appropriate environment variables passed between the two projects.

Yes, there is technically only one test - keep reading, I'll come back to that

The usage of C#, Entra, Vite, React and Playwright is not important here, you could replace any of these tools with alternatives. The important bit here is the general industry wide "standard" of having some form of web app using a third party authentication service via the OpenIdConnect protocol. It's the usage here of the standard OpenIdConnect protocol that makes the use of Entra un-important here.

Equally the usage of Aspire here is not important here, I've added it as a convenient way to run all the projects together with one click, especially for those not familiar with C# who may want to run the sample app.

Running the Demo

Commit 22745e62a70471592d83d8c6cb83efab4927b3d6 contains a README detailing the steps you need to run the demo system, feel free to clone the repo and see it in action with an Entra tenant as it would be in a production (or even test) deployment.

The problem

The keen eyed will have noticed that this initial commit actually only has one working test, the second test for when a user is logged in is incomplete and will always fail. The reason for this is two fold, firstly we will need a working Entra set up for running integration tests, this is not impossible to set up and is certainly workable however it does provide a minor barrier to things and will need to be maintained to ensure that the tests can continue to work and the user credentials in the tests match the credentials setup in the Entra instance itself. Plus if we wanted to set up multiple independent Entra instances for independent runs of the integration tests or if we wanted tests to create users dynamically on the fly in Entra we'd need to write custom code to make all this happen, it quickly gets fiddly and cumbersome.

None of that is a particular problem for our use case though so let's gloss over that for the moment. Let's assume we set up an Entra instance purely for use by the Integration tests and create test user(s) in it that the tests can use. Great, so now we could implement the test to navigate the Entra UI and login to allow the test to be implemented and pass. 🎉

Then, let's imagine that Microsoft change the UI in Entra. Or worse they start rolling out a new UI via blue-green deployment meaning that we might have to code for multiple UIs because we cannot be sure exactly which one we'll get. The Microsoft Entra team might even start rolling out additional security features (as you'd expect a leading auth service to do) such as enforcing MFA or adding Captcha checks to detect and block automated tools (such as the one we're using to run our automated tests).

Hopefully you can see that quickly our tests will start breaking frequently and lots of development time will be wasted keeping the tests going, and that's potentially the positive outlook on things - additional security features such as Captcha and enforced MFA might break the tests entirely and be unresolvable leading to the tests becoming useless. 😢

A solution

Here we come to the solution, and we take a common solution we use in Software development when we want to remove a dependency on a third party library in our automated tests. That is, to use a mock. With a mock Entra we'll be able to shield our tests from unexpected changes to the UI and hopefully configure our mock such that we can reduce/remove security features that are not important when running the system in a testing context.

So, I hear you ask, how can we replace Entra? Well, remember how I listed out Entra as one of the components of my Demo where the chosen product was not important. Instead the important part of the demo being that it's a system using a third party authentication service via the OpenIdConnect protocol. Given that, let's reword the question: How can we replace our third party authentication service? This is now a much easier prospect whereby we can replace Entra with any other OpenIdConnect compliant Identity service, something that is very doable and someone has already though of this and has published a docker image that can be used as a Mock OpenIdConnect service.

A Demo solution

Let's dive in then and see what we need to do in order to get the demo working with a mock OpenIdConnect service instead.

Continuing with my example repo, let's move on to commit f2eaeaab63a5295395afab96affdf2f47ba2ea04. This commit introduces several key changes. The main change is the introduction of the docker image along with configuration that allows it to mimic Entra and has a set of hardcoded credentials for testing purposes. This in turn allows the test for user login to be fully implemented. By using the Docker image instead of an external service, this test becomes less brittle and easier to maintain, as we have direct control over the identity service.

Give it a go if you want, clone the repo at commit f2eaeaab63a5295395afab96affdf2f47ba2ea04 and follow the updated readme to see it all in action.

So, we've got it all working, at least for my specific set up, your mileage will vary a lot so let me discuss the process that lead up to the code in this commit and to go through the code in a little more detail to help understand where additional changes will be needed for certain set ups and equally where parts of the code might not be needed.

A bit more detail of the demo project

Before I dive into the changes in commit f2eaeaab63a5295395afab96affdf2f47ba2ea04 let's first switch back to 22745e62a70471592d83d8c6cb83efab4927b3d6 and just touch on some of the specific technology choices involved for context as these do influence some of the solution and thus changes the solution compared to other set ups.

MSAL.js

In my demo there is a single page application running in a users browser, which is handling the UI to authenticate the user in conjunction with the specified authentication service. To help with this I have employed msal.js which is a library that allows javascript applications to consume Entra (also referred to by it's old AzureAD name in the msal documentation) via OpenIDConnect. This library is designed to work specifically with Entra but does support more generic OpenIdConnect servers.

The choice of using msal.js is not super important here, I've chosen to use it as a convenience more than anything else. I could have written all the code to interact with OpenIdConnect without using msal.js and for the most part much of the solution would have been the same.

Microsoft.Identity.Web

Next, once the browser code has obtained a bearer token using msal.js it will pass it to the backend api which in my demo is a ASP.NET application written in C# and uses the Microsoft.Identity.Web to implement middleware that enforces bearer token authentication. Microsoft.Identity.Web performs a similar function on the server side to the browser side however it differs slightly in that while msal can support other OpenIdConnect services the Microsoft.Identity.Web package instead builds on top of the built-in OpenIdConnect/JWT bearer token auth support that ASP.NET has and therefore it assumes that you're using it with an Entra instance and if you want to use it with a non-Entra identity service the expectation is that you will drop down to the more basic built-in OpenIdConnect support offered by ASP.NET.

This will become important later but for now try to overlook it and instead view the api project as a web project, that could have been written in any language, with middleware in place to enforce the requirement of bearer tokens from any configured OpenIdConnect service being passed.

Solution Basics

So let's start breaking down the solution, starting with the basics.

Docker container

I'm using the ghcr.io/soluto/oidc-server-mock docker image to spin up a local OpenIdConnect compliant service to substitute for Entra. Feel free to take a look at the image to see how it's built but you don't need to, all you need to know is that it's an OpenIdConnect service that is built to be configurable exactly for the purposes of testing like we're doing here. I'm leveraging Aspire to orchestrate the running of this image alongside the app with builder.AddContainer("mock-entra", "ghcr.io/soluto/oidc-server-mock")

Client

Next we need some basic configuration of the container as described in the readme for ghcr.io/soluto/oidc-server-mock. We'll start with a client and it's custom scopes, which substitutes for the app registration we'd have had in Entra. With the docker image this is as simple as injecting an environment variables which in my Aspire code is orchestrated via:

1 var clientId = Guid.NewGuid().ToString();
2 .WithEnvironment(
3     "CLIENTS_CONFIGURATION_INLINE",
4     () => System.Text.Json.JsonSerializer.Serialize(new[]
5     {
6         new
7         {
8             ClientId = clientId,
9             AllowedGrantTypes = new [] { "authorization_code" },
10             RedirectUris = new [] { vite.GetEndpoint("http").Url },
11             PostLogoutRedirectUris = new [] { vite.GetEndpoint("http").Url },
12             AllowedScopes = new [] { "openid", "profile", $"api://{clientId}/access_as_user" },
13             AllowOfflineAccess = true,
14             RequireClientSecret = false,
15             AlwaysIncludeUserClaimsInIdToken = true
16         }
17     }))
18 .WithEnvironment(
19     "API_SCOPES_INLINE",
20     () => JsonSerializer.Serialize(new[]
21     {
22         new
23         {
24             Name = $"api://{clientId}/access_as_user",
25         }
26     }))
27 

Note: The AllowOfflineAccess & RequireClientSecret bits here are specific to my example where I have the browser performing the authentication and this means there is no client secret and the "allow_offline" scope is passed. AllowedGrantTypes would also vary depending on your system set up but essentially this config needs to mirror the config used in Entra for the live system.

User config

Next we need fake users with credentials that our tests can use. Again this is a simple case of injecting in an environment variable, in my example I pass a single hard-coded user in using this code.

1 .WithEnvironment("USERS_CONFIGURATION_INLINE", () => System.Text.Json.JsonSerializer.Serialize(new[]
2         {
3             new
4             {
5                 SubjectId = "1",
6                 Username = "admin@test.com",
7                 Password = "Password123",
8                 Claims = new []
9                 {
10                     new { Type = "name", Value = "Frank Gardner" },
11                 }
12             }
13         }))
14 

The claims here need to mirror the ones that Entra will be setup to return in live. For the basic demo here we just have a name claim but you might have a number of others.

Some wiring

With all that set up we simply need to wire in the mock identity server to the main apps which in my demo is done by passing the identity server url and mock client id into the api project as environment variables

1 api
2     .WithEnvironment("AzureAd__Instance", mockEntra.GetEndpoint("https"))
3     .WithEnvironment("AzureAd__ClientId", clientId)
4 

Note: Small note here about using WithEnvironment and GetEndpoint together. Fundamentally WithEnvironment is a method that takes two string parameters however as you can see here we're passing in the result of calling GetEndpoint which is an Aspire.Hosting.ApplicationModel.EndpointReference instance; this works because WithEnvironment has many overloads to make things simpler for us, in this case allowing us to inject the endpoint url of another service which we won't know until runtime. Aspire even supplies a neat ReferenceExpression struct that you can implicitly use when using interpolated strings such as .WithEnvironment("AzureAd__Instance", $"{mockEntra.GetEndpoint("https")}/abc/123"). But, be warned, it does not work for string concatenation so while .WithEnvironment("AzureAd__Instance", mockEntra.GetEndpoint("https") + "/abc/123") will compile it will not work at runtime; if you find you do need to do something like string concatenation then WithEnvironment provides a callback lambda overload that will work in most use cases e.g. .WithEnvironment("AzureAd__Instance", () => mockEntra.GetEndpoint("https") + "/abc/123")

HTTPS

I had hoped when I started this journey that I'd be able to run the mock identity service on http. Sadly I quickly discovered that this would not be possible, both msal.js and Microsoft.Identity.Web both require that https is used and in retrospect I imagine most OpenIdConnect libraries will enforce it so this is likely something you'll need to contend with regardless of your exact set up and technology stack.

The documentation for ghcr.io/soluto/oidc-server-mock does a pretty good job of explaining what you need to do to host it with https. In my example this is all the code related to hosting the service with a https endpoint:

1 var certPassword = Guid.NewGuid().ToString();
2 var certExportExe = builder.AddExecutable("cert-export-exe", "dotnet", ".", "dev-certs", "https", "-ep", "./dev-certificates/aspnetapp.pfx", "-p", certPassword, "--trust", "--verbose");
3 
4 mockEntra
5     .WaitForCompletion(certExportExe)
6     .WithEnvironment("ASPNETCORE_Kestrel__Certificates__Default__Password", certPassword)
7     .WithEnvironment("ASPNETCORE_Kestrel__Certificates__Default__Path", "/https/aspnetapp.pfx")
8     .WithBindMount("./dev-certificates", "/https")
9     .WithEnvironment("ASPNETCORE_URLS", "https://+:443")
10 

Reading this code, you might think you need to have a good understanding of .NET

That however would be an incorrect assumption, all you need is a password protected pfx format https certificate which can be mounted to the docker image and the password and path passed into the container as environment variables. In my demo I'm using the dotnet cli to run dotnet dev-certs https ... to generate the certificate but you could very easily use openssl or something else to generate a certificate.

Sadly in my Demo the certificate generated here is not quite trusted by the browser which playwright spins up, I am sure given more time I could have fixed this but instead I chose to cheat slightly and instead added ignoreHTTPSErrors: true to the playwright config.

But we're dealing with Entra

At this stage, believe it or not we've covered all the code you should, in theory, need to get things working but you'll notice that my solution has a bit more code involved. That is because we're now into code which comes about due to the specifics of the application set up. For this case that's because the code in this demo is geared towards Entra (mainly through it's usage of msal.js and Microsoft.Identity.Web) and the mock identity service spun up is not Entra. This all boils down to there being some friction where the code (msal.js and Microsoft.Identity.Web in this case) has some quirky bespoke configuration and validation logic that works with the identity service in use but that is not standard to OpenIdConnect.

Your options here fall into two main categories:

1. You change your application code such that it can be configured to use different OpenIdConnect services which may mean using different middlewares etc
2. You configure the mock Identity service in such a way that it can mimic the setup and behaviour of Entra.

The first of these options is arguably easier to implement but it can slowly means that the code you are testing is more and more different to the code that is running in production. The second option meanwhile should in theory require no changes to the production code (or far fewer) and thus, the code being tested and, the code in production will be far closer to each other.

So what were the main Entra related sticking points I encountered in my demo and how did I get around them?

The Tenant Id

The documentation for the Microsoft.Identity.Web which was followed here meant that from Entra we got an Instance, ClientId and TenantId config value. We've already replicated the first two with fake/mock values but what about the Tenant Id and do we even need it?

The short answer here is, yes, we do need a tenant id. At least we do while we're using the AddMicrosoftIdentityWebApi middleware from Microsoft.Identity.Web which assumes that Entra is being used, as I previously explained the Microsoft.Identity.Web is built for Entra and if you're using something else it's assumed you'll drop down to the more primitive JWT bearer token middlewares that are built into .NET. It is therefore here that we encounter the first occasion where we have a choice of changing the code to be more configurable or configuring the mock identity server to behave like Entra so the code does not need to be amended.

For this blog I opted to amend the configuration of the Identity server to better imitate Entra starting here with the inclusion of a tenant id which needs to form part of the urls via this configuration which configures the identity server to run under a specific base path which includes the tenant id, just like Entra. In addition, because I'm using browser side authentication I need to ensure that these urls are included in CORS configuration.

1 var tenantId = Guid.NewGuid().ToString();
2 mockEntra
3     .WithEnvironment(
4         "ASPNET_SERVICES_OPTIONS_INLINE",
5         System.Text.Json.JsonSerializer.Serialize(new { BasePath = $"/{tenantId}/v2.0" }))
6     .WithEnvironment(
7         "SERVER_OPTIONS_INLINE",
8         () => JsonSerializer.Serialize(new
9         {
10             Cors = new
11             {
12                 CorsPaths = new[]
13                 {
14                     $"/{tenantId}/v2.0/.well-known/openid-configuration",
15                     $"/{tenantId}/v2.0/connect/token"
16                 }
17             },
18         }))
19 
20 api
21     .WithEnvironment("AzureAd__TenantId", tenantId)
22 

aud claim

Next we come to validation built into the Microsoft.Identity.Web library which validates the aud (Audience) claim in the JWT token. This means that we need the Mock identity service to return claims that mimic Entra and allow this validation to pass. For this I've added additional claims to the Client configuration:

1 mockEntra
2     .WithEnvironment(
3         "CLIENTS_CONFIGURATION_INLINE",
4         () => System.Text.Json.JsonSerializer.Serialize(new[]
5     {
6         new
7         {
8             /* --- Existing config omitted --- */
9             AlwaysSendClientClaims = true,
10             Claims = new [] {
11                 new { Type = "aud", Value = $"api://{clientId}" },
12                 new { Type = "ver", Value = $"1.0" },
13             },
14             ClientClaimsPrefix = string.Empty,
15         }
16     }))
17 

scp claim

Another quirk of the Microsoft.Identity.Web library and it being written with Entra in mind is that it expects a scp or role claim to be returned. Again i've sorted this by adding this claim to the client config plus I've ensured that the claim is associated with the accessasuser scope requested by the app.

1 mockEntra
2     .WithEnvironment(
3         "CLIENTS_CONFIGURATION_INLINE",
4         () => System.Text.Json.JsonSerializer.Serialize(new[]
5         {
6             new
7             {
8                 /* --- Existing config omitted --- */
9                 Claims = new [] {
10                     /* --- Existing claims omitted --- */
11                     new { Type = "scp", Value = "access_as_user"},
12                 },
13             }
14         }))
15     .WithEnvironment(
16         "API_SCOPES_INLINE",
17         () => JsonSerializer.Serialize(new[]
18         {
19             new
20             {
21                 Name = $"api://{clientId}/access_as_user",
22                 UserClaims = new[]
23                 {
24                     "scp"
25                 }
26             }
27         }))
28 

A quick extra note on this change; this was one of the most time consuming parts of this whole blog post. This problem will manifest itself as a 401 response from the API and an invalid_token header but otherwise there is no useful log in the system to tell you what is the issue. Eventually I managed to solve this issue by removing the requirement to be authenticated from the api and manually added this line to the request handler var authResult = await httpContext.AuthenticateAsync() and finally I was able to inspect authResult to see what was going on. A useful hint if you find yourself in a similar position with this stuff.

knownAuthorities

The final bit is a change required because of msal.js and is the only place where I had to actually change the "Production" code to enable it to work with the fake entra service I was spinning up and that was to add the url of the service to the knownAuthorities array in the msal configuration.

1 export const getMsalConfig = async (): Promise<Configuration> => {
2     // Other code omitted
3     return ({
4         auth: {
5             // Other config omitted
6             knownAuthorities: [settings.auth.authority],
7         },
8     });
9 };
10 

Logging out

Before I wrap up a final amendment about the log out button and getting that to work like Entra. This sadly was another place I needed_* to change the "production" code.

With Entra the login page would redirect over to Entra and then once the user had finished signing out they are returned to our application. However with the mock this does not happen and the reason is detailed in the identity server (the package used by the docker image) documentation on the end session endpoint which states

If a valid idtokenhint is passed, then the client may also send a postlogoutredirect_uri parameter. This can be used to allow the user to redirect back to the client after sign-out.

This means that if we want our users to be redirected back to the post_logout_redirect_uri just like they are with Entra then we need to pass the id_token_hint parameter too which can be achieved with the following change on the msal code:

1 instance.logoutRedirect({ idTokenHint: account?.idToken });
2 

• Yes I didn't need to do this here, I could simple have accepted that the mock identity service would not perform the auto redirect like Entra does and coded the tests accordingly but I wanted to see how close the Mock could be to Entra.

Closing remarks

As you can hopefully see Authentication does not need to be the sticking point for automated UI tests. If you're using a third party authentication service like Entra, Auth0 or Okta with OpenIdConnect in your application then it is something you can replace with a Mock, in both test and development systems. Making testing and local development easier; although it's not going to be trivial either! I hope this blog will show how many of the sticking points can be worked around and offers advice for how to resolve any issues regardless of technology choices.