Tag Archives: Shields

Collecting Code Coverage Metrics for .NET Core Libraries in Azure DevOps

Last time I talked about how I added status badges to the Readme file for RogueSharp. One of the badges was for showing the Code Coverage results from my continuous Azure DevOps build.

Before you can have a badge display those numbers, you need to have a build that can actually generate and collect them. While searching I found a lot of information about how to do it for full .NET Framework projects but information on .NET Core was scarce.

Since it wasn’t very straightforward for me I’ll share what I did here. Maybe there is a better way now?

MSTest Version

The first issue I struggled with was realizing that the version of MSTest that was I was using did not support the option to collect code coverage.

https://www.nuget.org/packages/MSTest.TestFramework/

You’ll need at least version 1.4.0.

Here is the commit where I upped my version, so up yours!

https://github.com/FaronBracy/RogueSharp/commit/10775b73baaec0785155ad2be287a5c255d0212b

Azure DevOps Menu

Azure DevOps Continuous Build

The Azure DevOps continuous build should then be using the .NET Core task.

.NET Core Build Task

I added three distinct steps.

  1. Restore NuGet Packages
  2. Build
  3. Run Tests

Build Steps

It’s this third step for running tests that will get us code coverage.

Test Step Setup

  • Under Arguments check the box next to Publish test results and code coverage
  • The Arguments should be --configuration $(BuildConfiguration) --collect "Code coverage"

The --collect "Code coverage" bit will only work if you are on MSTest version 1.4.0 or higher. It also appears to be case sensitive so make sure you type it exactly as shown.

Viewing Code Coverage Metrics

Once this is setup trigger the continuous build. If you look in the summary you should see the code coverage showing up now.

Build Summary

View Build Results In Azure DevOps

Code Coverage Badge

Code coverage

Shields.IO supports code coverage badges for multiple build services.

The formatted URL for Azure DevOps looks like this:

https://img.shields.io/azure-devops/coverage/{ORGANIZATION}/{PROJECT}/{DEFINITION_ID}.svg

See my previous post on status badges/shields for more information

About Faron

Faron Bracy is a husband, father, and TechSmith employee. He maintains the open source library RogueSharp as a hobby project. Other hobbies include painting fantasy miniatures, playing board games, and playing retro video games with his family.

Creating Status Badges and Shields for your Repository

I recently added status badges to the Readme file for the open source project I maintain, RogueSharp. I’ve seen other repositories do this but I wasn’t sure how. After a lot of looking around I compiled a list of services that worked for me. I couldn’t seem to find all this information in one place, so I’m hoping that this post showcasing what worked for me will help others.

Shields.IO

Shields.IO is a service for creating concise, consistent, and legible badges in SVG format. This should be your first stop when looking for a particular badge.

Resource Links

Tutorials: Blog Documentation: Github License: MIT

I use Shields.IO to create badges that link to static resources. To create these you just need a specially formatted URL.

https://img.shields.io/badge/LABEL-MESSAGE-COLOR.svg

To create a blue badge for a BSD 3-Clause license we could use

https://img.shields.io/badge/license-BSD%203--Clause-blue.svg

Because our license has a space and a dash in it we will need to treat those specially. The space will need to be replaced with it’s equivalent URL encoding “%20” and the dash will need to be escaped with a preceding dash.

Finally we need to decorate it with the appropriate markdown to get it to display as an image which links to the license.

[![License: BSD 3-Clause](https://img.shields.io/badge/license-BSD%203--Clause-blue.svg)](https://github.com/TechSmith/hyde/blob/master/LICENSE.txt)

License: BSD 3-Clause

Test Status

Test status

It’s likely that no matter which service you are using for running continuous builds, Shields.IO also has you covered.

The formatted URL to get a test status badge for an Azure DevOps pipeline project will look like this:

https://img.shields.io/azure-devops/tests/ORGANIZATION/PROJECT/DEFINITION_ID.svg

When I did this for my own repo I wasn’t quite sure how to fill out “ORGANIZATION”, “PROJECT” and “DEFINITION_ID” placeholders.

What I found is that you can pull these directly from the query string when going to Azure DevOps and viewing your continuous build. In my case the URL is:

https://dreamersdesign.visualstudio.com/RogueSharp/_build?definitionId=1

In this example “dreamersdesign” is the organization, “RogueSharp” is the project and “1” is the definition ID.

Code Coverage

Code coverage

Shields.IO also supports code coverage badges for multiple build services.

The formatted URL for Azure DevOps looks like this:

https://img.shields.io/azure-devops/coverage/ORGANIZATION/PROJECT/DEFINITION_ID.svg

You’ll notice it looks like the test status badge, only replacing “tests” with “coverage”

BuildStats.info

Even though Shields.IO supported most of what I needed it didn’t support everything. It looks like development keeps happening on the service, so perhaps that will have changed by the time you read this. For my remaining badges I turned to BuildStats.info.

BuildStats.info creates SVG widgets to display build history charts and NuGet badges.

NuGet

Nuget

This one couldn’t be simpler. Just create a URL to

https://buildstats.info/nuget/PACKAGE_NAME

So for RogueSharp this was just

https://buildstats.info/nuget/roguesharp

It will automatically pull in the latest version number as well as the number of downloads.

Build History Chart

Build History

At the time that I setup my badges, BuildStats.info didn’t support the Build History Chart for Azure DevOps. It does look like this is now supported, but for my chart I used AppVeyor.

The formatted URL looks like this:

https://buildstats.info/BUILD_SYSTEM/chart/ACCOUNT/PROJECT

Substituting the values for my repository this yielded:

https://buildstats.info/appveyor/chart/FaronBracy/roguesharp-20n28

Build Service Specific

It’s worth noting that many build services provide their own status badge links. They probably won’t provide you with anything that you can’t get with Shields.IO, but it may be worth looking into.

Azure DevOps

Build status

When you are looking at your build pipeline you’ll see an elipsis for the build… click that.

Azure DevOps Menu

Then click on Status badge

Azure DevOps Status Badges

This will give you markdown that you can copy and paste directly into your readme.

AppVeyor

Build status

Click on AppVeyor configuration for a repo (the gear icon). Then select “Badges” from the tabs on the left.

AppVeyor Status Badges

This will give you markdown that you can copy and paste directly into your readme.

About Faron

Faron Bracy is a husband, father, and TechSmith employee. He maintains the open source library RogueSharp as a hobby project. Other hobbies include painting fantasy miniatures, playing board games, and playing retro video games with his family.