Unit testing can be very handy when it's set up correctly. The gold standard is to have regular CI builds, usually run after each commit, that executes all tests to confirm that the application still does what it's been designed to do. It can be tricky to get it all set up so all of the magic just happens and unfortunately that can mean it's never done. Get those tests running and reporting! Future You will thank you. In this post, I will walk through the process of setting up a basic build pipeline with Azure DevOps that executes unit tests and reports on code coverage.
"The best and most convenient time to set up tests is before you start the project" - William Liebenberg
Start with a project
First steps are to create a basic project. You can follow these steps, or grab the project from here. This project will be the sample angular web app.
$ mkdir BuildApp.Web
$ cd BuildApp.Web
$ dotnet new angular --no-https # Create a web application
$ cd ..
$ dotnet new sln # Create solution file
$ dotnet sln add BuildApp.Web\BuildApp.Web.csproj
$ dotnet build
$ dotnet run --project BuildApp.Web
Unit testing in VS Code
I am going to use xUnit for my tests. Why xUnit? No particular reason. There are a few options. If you don't have a favourite already, just pick one. Any unit testing framework is better than none.
$ mkdir BuildApp.Web.Tests
$ cd BuildApp.Web.Tests
$ dotnet new xunit # Create a test project
$ dotnet add reference ..\BuildApp.Web\BuildApp.Web.csproj
$ cd..
$ dotnet sln add BuildApp.Web.Tests\BuildApp.Web.Tests.csproj
$ dotnet test
Update UnitTest1.cs like so:
public class UnitTest1
{
[Fact]
public void Test1()
{
var controller = new SampleDataController();
var result = controller.WeatherForecasts();
Assert.Equal(5, result.Count());
}
}
And then execute the test:
$ dotnet test
Starting test execution, please wait...
Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.3171 Seconds
That's helpful, but we can do better.
Code coverage with Coverlet
Coverlet is a great tool that provides code coverage for your unit tests. It's super easy to install and use. Just do:
$ dotnet add package coverlet.msbuild
$ dotnet test /p:CollectCoverage=true
Starting test execution, please wait...
Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.3743 Seconds
Calculating coverage result...
Generating report 'C:\Projects\BuildApp\BuildApp.Web.Tests\coverage.json'
+--------------------+--------+--------+--------+
| Module | Line | Branch | Method |
+--------------------+--------+--------+--------+
| BuildApp.Web | 22.85% | 0% | 33.33% |
+--------------------+--------+--------+--------+
| BuildApp.Web.Views | 0% | 100% | 0% |
+--------------------+--------+--------+--------+
+---------+---------+--------+---------+
| | Line | Branch | Method |
+---------+---------+--------+---------+
| Total | 21.33% | 0% | 31.25% |
+---------+---------+--------+---------+
| Average | 10.665% | 0% | 15.625% |
+---------+---------+--------+---------+
Azure DevOps build pipeline
Unit tests on a developer machine are one thing, but what if you forget to run them? That's what continuous integration (CI) is all about. Every time code is pushed to master, it is built and the unit tests are run. If it fails to compile, or the unit tests fail, the whole build fails.
First, we will be generating code coverage results in Cobertura format. We will need to use ReportGenerator to convert this to html and before publishing the results. To do this, add the following to the BuildApp.Web.Tests.csproj:
<ItemGroup>
<DotNetCliToolReference Include="dotnet-reportgenerator-cli" Version="4.2.15" />
</ItemGroup>
Add this to the project as azure-pipelines.yml:
name: BuildApp-CI
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- task: DotNetCoreInstaller@0
displayName: 'Install .net core 3.0 (preview)'
inputs:
version: '3.0.100-preview6-012264'
- script: dotnet build --configuration $(buildConfiguration)
displayName: 'dotnet build $(buildConfiguration)'
- script: dotnet test BuildApp.Web.Tests/BuildApp.Web.Tests.csproj --logger "trx;LogFileName=testresults.trx" /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=TestResults/Coverage/
displayName: 'dotnet test'
- script: dotnet reportgenerator "-reports:$(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml" "-targetDir:$(Build.SourcesDirectory)/TestResults/Coverage/Reports" -tag:$(Build.BuildNumber) -reportTypes:htmlInline
workingDirectory: $(Build.SourcesDirectory)/BuildApp.Web.Tests
displayName: 'dotnet reportgenerator'
- task: PublishTestResults@2
inputs:
testRunner: VSTest
testResultsFiles: '**/*.trx'
failTaskOnFailedTests: true
- task: PublishCodeCoverageResults@1
inputs:
codeCoverageTool: 'cobertura'
summaryFileLocation: $(Build.SourcesDirectory)/TestResults/Coverage/**/coverage.cobertura.xml
reportDirectory: $(Build.SourcesDirectory)/TestResults/Coverage/Reports
failIfCoverageEmpty: false
Figure: Test Results for the single test
Figure: Code Coverage Results
SonarQube
SonarQube is a static analysis tool that will:
- highlight bugs and vulnerabilities
- review security hotspots
- track technical debt
- provide code quality metrics
- integrate with CI/CD
- and that's just in the community edition!
Let's hook it up.
- Sign up at sonarqube.io and create your project. Use the new Auto Scan feature.
- Drop a .sonarcloud.properties file into the root folder of your project.
- Commit and push. This will trigger the first analysis run.
Some of the results you can get from SonarQube are:
Figure: SonarQube overview
Figure: SonarQube review results
Figure: SonarQube project summary
SonarQube looks like a great addition to any project. If you're worried about giving them access to your codebase, they also have local server options that can generate reports without handing over access.
Running unit tests during your build process really helps to keep your code doing what it is designed to do. There are also many static analysis tools that can automatically scan for security vulnerabilities and best practices. By applying these tools automatically in your CI/CD pipeline, you can deploy with confidence.
References:
