Skip to main content

A test script that fails the build

test.ps1 has been growing since the output module: it runs the suite, prints detailed results and writes both artifacts. It needs one more line before a pipeline can rely on it — the one that makes it fail.

The missing line​

test.ps1
$config = New-PesterConfiguration
$config.Run.Path = './Planetarium'
$config.Run.Exit = $true
$config.Output.Verbosity = 'Detailed'
$config.TestResult.Enabled = $true
$config.TestResult.OutputPath = './testResults.xml'
$config.CodeCoverage.Enabled = $true
$config.CodeCoverage.Path = './Planetarium'
$config.CodeCoverage.OutputPath = './coverage.xml'

Invoke-Pester -Configuration $config

That is the finished script: run the tests, write both artifacts, exit non-zero if anything failed.

Why the exit code is the important line​

A failing Pester test does not, by itself, fail your build.

By default Invoke-Pester reports failures and returns normally. Whether that becomes a red build then depends entirely on how the script is invoked — which is a horrible thing to leave to chance:

# Without Run.Exit, one failing test:
pwsh -NoProfile -File ./test.ps1   # exit code 0 — the build goes green

Pester does set $LASTEXITCODE either way, so some runners catch it regardless. GitHub Actions is one: its pwsh shell appends an exit $LASTEXITCODE for you, so the step fails even without Run.Exit. Run the same script under pwsh -File — a scheduled task, a Makefile, most other CI systems — and the failure vanishes.

Run.Exit = $true removes the guesswork by making the script itself exit non-zero. Prove both directions rather than trusting it:

pwsh -NoProfile -File ./test.ps1
$LASTEXITCODE
Tests Passed: 26, Failed: 0, Skipped: 0, Inconclusive: 0, NotRun: 0
0

Now break something on purpose — change a Should-Be 8 to Should-Be 9 — and run it again:

Tests Passed: 25, Failed: 1, Skipped: 0, Inconclusive: 0, NotRun: 0
1

Exit code 1. That is what makes CI red. Put the 8 back.

The code is not merely non-zero, it is the number of failures: five failing tests exit with 5. Failed blocks count too, so a BeforeAll that throws while wrapping two tests exits with 3 — the two failed tests plus the failed block. Handy at a glance in a job list, though read the results file rather than doing arithmetic on an exit code.

Invoke-Pester -CI is the shorthand — but not for this

Pester has a -CI switch that sets exactly two things:

TestResult.Enabled = $true
Run.Exit = $true

It does not enable code coverage. Since this script wants coverage too — along with control over the output paths and verbosity — it sets those options directly instead. If you only need test results and a correct exit code, Invoke-Pester -Path ./Planetarium -CI is the one-liner version of most of this page.

It still works locally​

You have been running this script since the output module, and adding Run.Exit does not change that:

./test.ps1
$LASTEXITCODE

Run.Exit is safe interactively. It ends the script rather than your session, and leaves $LASTEXITCODE behind so you can check the result.

Running the identical script locally and in CI removes an entire genre of problem — the one where a build fails remotely and cannot be reproduced because CI was doing something subtly different.

Artifacts​

Two files come out of every run:

  • testResults.xml — NUnit format, per-test results
  • coverage.xml — JaCoCo format, coverage data

Both are build output, so keep them out of version control:

testResults.xml
coverage.xml

They exist for the machine that runs the build, not for you. The next page hands them to GitHub.

Before you move on

0/6

Next: running it on every push.