Docker Windows container for Pester Tests

I recently wrote an intro to unit testing your powershell modules with Pester, and I wanted to give a walk through for our method of running these unit tests inside of a Docker for Windows container.


Before we get started, I'd like to acknowledge this post is obviously filled with trendy buzzwords (CICD, Docker, Config Management, *game of thrones, docker-compose, you get the picture). All of the components we're going to talk through today add concrete value to our business, and we didn't do any resume driven development.

Why?

Here's a quick run through of our motivation for each of the pieces I'll cover in this post.

1. Docker image for running unit tests 
1. gives engineers a consistent way to run the unit tests. One your workstation you might need different versions of SDKs and tools, but a docker container lets you pin versions of things like the AWS Powershell tools
2. Makes all pathing consistent - you can setup your laptop anyway you lock, but the paths inside of the container are consistent
2. Docker-compose
1. Provides a way to customize unit test runs to a project
2. Provides a consistent way for engineers to map drives into the container
3. Code coverage metrics
1. At my company we don't put too much stock in code coverage metrics, but they offer some context for how thorough an engineer has been with unit tests
2. We keep a loose goal of 60%
4. Unit test passing count
1. A failed unit test does not go to production. A failed unit test has a high chance of causing production outage

How!

The first step is to setup Docker Desktop for Windows. The biggest struggle I've seen people having getting docker running on Windows is getting virtualization enabled, so pay extra attention to that step.

Once you have Docker installed you'll need to create an image you can use to run your unit tests, a script to execute them, and a docker-compose file. The whole structure will look like

• /
• docker-compose.yml
• /pestertester
• Dockerfile
• Run-AllUnitTests.ps1

We call our image "pestertester" (I'm more proud of that name than I should be).

There are two files inside of the pestertester folder: a Dockerfile that defines the image, and a script called Run-AllUnitTests.ps1.

Here's a simple example of the dockerfile. For more detail on how to write a dockerfile you should explore the dockerfile reference

FROM mcr.microsoft.com/windows/servercore
RUN "powershell Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force"
RUN "powershell Install-Module -Scope CurrentUser -Name AWSPowerShell -Force;"
COPY ./Run-AllUnitTests.ps1 c:/scripts/Run-AllUnitTests.ps1

All we need for these unit tests is the AWS Powershell Tools, and we install NuGet so we can use powershell's Install-Module.

We played around with several different docker images before we picked mcr.microsoft.com/windows/servercore.

1. We moved away from any of the .NET containers because we didn't need the dependencies they added, and they were very large
2. We moved away from nano server images because some of our powershell modules call functions outside of .NET core

Next we have the script Run-AllUnitTests.ps1. The main requirement for this script to work is that your tests be stored with this file structure

• /ConfigModule.psm1
• /tests
• /ConfigModule.tests.ps1
• ConfigModule2.psm1
• /tests
• /ConfigModule2.tests.ps1

The script isn't too complicated

$results = @();
gci -recurse -include tests -directory | ? {$_.FullName -notlike "*dsc*"} | % {
    set-location $_.FullName;
    $tests = gci;
    foreach ($test in $tests) {
        $module = $test.Name.Replace("tests.ps1","psm1")
        $result = invoke-pester ".\$test" -CodeCoverage "..\$module" -passthru -quiet;
        $results += @{Module = $module;
                      Total = $result.TotalCount;
                      passed = $result.PassedCount;
                      failed = $result.FailedCount
                      codecoverage = [math]::round(($result.CodeCoverage.NumberOfCommandsExecuted / $result.CodeCoverage.NumberOfCommandsAnalyzed) * 100,2)
                    }
    }
}

foreach ($result in $results) {
    write-host -foregroundcolor Magenta "module: $($result['Module'])";
    write-host "Total tests: $($result['total'])";
    write-host -ForegroundColor Green "Passed tests: $($result['passed'])";
    if($result['failed'] -gt 0) {
        $color = "Red";
    } else {
        $color = "Green";
    }
    write-host -foregroundcolor $color "Failed tests: $($result['failed'])";
    if($result['codecoverage'] -gt 60) {
        $color = "Green";
    } elseif($result['codecoverage'] -gt 30) {
        $color = "Yellow";
    } else {
        $color = "Red";
    }
    write-host -ForegroundColor $color "CodeCoverage: $($result['codecoverage'])";
}

The script iterates through any subdirectories named "tests", and executes the unit tests it finds there, running code coverage metrics for each module.

The last piece to tie all of this together is a docker-compose file. The docker compose file handles

1. Mapping the windows drives into the container
2. Executing the script that runs the unit tests

The docker-compose file is pretty straightforward too

version: '3.7'

services:
  pestertester:
    build: ./pestertester
    volumes:
      - c:\users\bolson\documents\github\dt-infra-citrix-management\ssm:c:\ssm
    stdin_open: true
    tty: true
    command: powershell "cd ssm;C:\scripts\Run-AllUnitTests.ps1"

Once you've got all of this setup, you can run your unit tests with

docker-compose run pestertester

One the container starts up you'll see your test results

Experience

We've been running linux containers in production for a couple of years now, but we're just starting to pilot windows containers. According to the documentation they're not production ready yet

Docker is a full development platform for creating containerized apps, and Docker Desktop for Windows is the best way to get started with Docker on Windows.

Running our unit tests inside of windows containers has been a good way to get some experience with them without risking production impact.

A couple final thoughts

Windows containers are large, even server core and nano server are gigabytes.

The container we landed on is 11GB

If you need to run windows containers, and you can't stick to .NET core and get onto nano server, you're going to be stuck with pretty large images.

Start up times for windows containers will be a few minutes

Especially the first time on a machine while resources are getting loaded.

Versatile Pattern

This pattern of unit testing inside of a container is pretty versatile. You can use it with any unit testing framework, and any operating system you can run inside a container.

*no actual game of thrones references will be in this blog post