JUnit support¶
Nextest can produce output in the JUnit/XUnit XML format. This format is widely understood by test analysis tools and libraries.
To enable JUnit support, add this to your nextest configuration:
[profile.ci.junit] # this can be some other profile, too
path = "junit.xml"
If --profile ci is selected on the command line, a JUnit report will be written out to target/nextest/ci/junit.xml within the workspace root.
Some notes about the JUnit support:
- There are several slightly different formats all called "JUnit" or "XUnit". Nextest adheres to the Jenkins XML format.
- Every test binary forms a single
<testsuite>. Every test forms a single<testcase>. - Standard output and standard error are included for failed and retried tests. (However, invalid XML characters are stripped out.)
Configuration¶
Configuration options supported for JUnit reports, within the junit section:
report-name- The name of the report. Defaults to
"nextest-run". store-success-output- Whether to store output for successful tests in the
<system-out>and<system-err>elements. Defaults to false. store-failure-output- Whether to store output for failing tests in the
<system-out>and<system-err>elements. Defaults to true.
The store-success-output and store-failure-output configuration can also be configured as per-test overrides.
Example configuration¶
[profile.default.junit]
path = "junit.xml"
# These are the default values, specified for clarity.
store-success-output = false
store-failure-output = true
[[profile.default.overrides]]
filter = 'test(important-test)'
junit.store-success-output = true
In this example, the JUnit report will contain the output for all failing tests, and for successful tests that contain "important-test" in the name.
Post-processing¶
Some tools that read JUnit files don't follow the Jenkins standard. You can post-process the JUnit file in such cases. Here's some recommendations for post-processing tools written by community members:
- CircleCI:
circleci-junit-fix
Example¶
Here's an example JUnit file generated by cargo-nextest.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="nextest-run" tests="3" failures="1" errors="0" uuid="45c50042-482e-477e-88a2-60cfcc3eaf95" timestamp="2024-01-09T07:50:12.664+00:00" time="0.023">
<testsuite name="nextest-tests::basic" tests="3" disabled="0" errors="0" failures="1">
<testcase name="test_cwd" classname="nextest-tests::basic" timestamp="2024-01-09T07:50:12.665+00:00" time="0.004">
</testcase>
<testcase name="test_failure_assert" classname="nextest-tests::basic" timestamp="2024-01-09T07:50:12.665+00:00" time="0.004">
<failure type="test failure">thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace</failure>
<rerunFailure timestamp="2024-01-09T07:50:12.670+00:00" time="0.004" type="test failure">thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</rerunFailure>
<rerunFailure timestamp="2024-01-09T07:50:12.676+00:00" time="0.004" type="test failure">thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</rerunFailure>
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at tests/basic.rs:19:5:
assertion `left == right` failed: this is an assertion
left: 4
right: 5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</testcase>
<testcase name="test_flaky_mod_4" classname="nextest-tests::basic" timestamp="2024-01-09T07:50:12.683+00:00" time="0.004">
<flakyFailure timestamp="2024-01-09T07:50:12.665+00:00" time="0.004" type="test failure">thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 1 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_flaky_mod_4 ... FAILED
failures:
failures:
test_flaky_mod_4
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 1 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</flakyFailure>
<flakyFailure timestamp="2024-01-09T07:50:12.671+00:00" time="0.004" type="test failure">thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 2 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_flaky_mod_4 ... FAILED
failures:
failures:
test_flaky_mod_4
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 2 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</flakyFailure>
<flakyFailure timestamp="2024-01-09T07:50:12.676+00:00" time="0.005" type="test failure">thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 3 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_flaky_mod_4 ... FAILED
failures:
failures:
test_flaky_mod_4
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_flaky_mod_4' panicked at tests/basic.rs:43:9:
Failed because attempt 3 % 4 != 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</flakyFailure>
</testcase>
</testsuite>
</testsuites>