How to contribute to the µTest++ Testing Framework

This page is designed for developers who intend to contribute new features or resolve bugs within the µTest++ Testing Framework project and provides documentation on building and testing the package.

The project is written in modern C++ and, for reproducibility reasons, utilises the xPack Framework tools during development.

Prerequisites

@micro-os-plus/micro-test-plus is an xpm package that provides a C++ source code library; development can be conducted on macOS, GNU/Linux and Windows (although some npm scripts must be executed within a Git Bash terminal).

The prerequisites are:

• git
• node >= 18.0.0
• npm

To execute the native tests with the system tools, a C++ development environment is required. On macOS, install Command Line Tools; on Ubuntu, install build-essential.

For details on installing the prerequisites, please refer to the Prerequisites page.

Obtain project sources

The project is hosted on GitHub:

• https://github.com/micro-os-plus/micro-test-plus-xpack.git

Branches

This project utilises multiple branches:

• master, not actively used
• xpack, containing the latest stable version (default)
• xpack-development, containing the current development version
• website, containing the current website content; pushes to this branch automatically trigger publication of the main website
• webpreview, containing the current preview website content; pushes to this branch automatically trigger publication of the preview website

All development is conducted in the xpack-development branch, and contributions via Pull Requests should be directed to this branch.

When new releases are published, the xpack-development branch is rebased onto xpack.

To clone the stable branch (xpack), execute the following commands in a terminal (on Windows use the Git Bash console):

rm -rf ~/Work/micro-os-plus/micro-test-plus-xpack.git && \
mkdir -p ~/Work/micro-os-plus && \
git clone \
  --branch xpack \
  https://github.com/micro-os-plus/micro-test-plus-xpack.git \
  ~/Work/micro-os-plus/micro-test-plus-xpack.git

For development purposes, clone the xpack-development branch.

rm -rf ~/Work/micro-os-plus/micro-test-plus-xpack.git && \
mkdir -p ~/Work/micro-os-plus && \
git clone \
  --branch xpack-development \
  https://github.com/micro-os-plus/micro-test-plus-xpack.git \
  ~/Work/micro-os-plus/micro-test-plus-xpack.git

Alternatively, if the repository has already been cloned:

git -C ~/Work/micro-os-plus/micro-test-plus-xpack.git pull

tip

To contribute Pull Requests, fork the project and ensure the Copy the master branch only is disabled.

Utilise the xpack-development branch and ensure you contribute the Pull Requests back to the xpack-development branch.

Add links for development

During development, it is convenient to maintain a writable instance of the library to enable changes in parallel with the parent project.

To facilitate the use of a writable instance of this library in other projects, add a link from the user's global xPacks store to this local development folder:

xpm link -C ~/Work/micro-os-plus/micro-test-plus-xpack.git

And in the projects referring it:

xpm link @micro-os-plus/micro-test-plus

Obtain the writable helper sources (optional, for development purposes)

The project has a dependency on a common helper, which is normally installed as a read-only dependency; for development purposes, to enable modifications to the scripts located within the helper, clone the xpack-development branch and link it to the user's global xPacks store:

rm -rf ~/Work/micro-os-plus/build-helper-xpack.git && \
mkdir -p ~/Work/micro-os-plus && \
git clone \
  --branch xpack-development \
  https://github.com/micro-os-plus/build-helper-xpack.git \
  ~/Work/micro-os-plus/build-helper-xpack.git 

Alternatively, if the repository has already been cloned:

git -C ~/Work/micro-os-plus/build-helper-xpack.git pull

If a writable instance of this library is required in another project, add a link from the user's global xPacks store to it:

xpm link -C ~/Work/micro-os-plus/build-helper-xpack.git

Satisfy dependencies

Navigate to the project folder:

cd ~/Work/xpack/micro-os-plus/micro-test-plus-xpack.git

and execute the following commands:

npm install
npm --prefix tests install
xpm install -C tests

Language standard compliance

The library is written in modern C++ and should be compiled with -std=c++20.

Code formatting

For formatting style, the library employs a .clang-format configuration file based on the GNU style.

Code formatting is performed using clang-format --style=file, either manually via a script, or automatically through Visual Studio Code, or the Eclipse CppStyle plug-in.

info

Visual Studio Code can directly utilise the .clang-format file within the Format Document command.

tip

Always reformat the source files that have been modified before committing them to the repository.

Tests

The µOS++ testing strategy involves compiling the sources with as many toolchains as possible and executing them on a wide range of platforms.

Host Platforms

The tests may be executed on any of the supported host platforms:

• GNU/Linux (x64 and arm64, GLIBC>=2.28); to execute the native tests, a C++ development environment is required (on Ubuntu install build-essential)
• macOS (>= 11.0); as C++ development environment utilise Command Line Tools
• Windows 7 with the Universal C Runtime (UCRT), Windows 8, Windows 10, Windows 11

Target Platforms

The supported target test platforms are:

• platforms/native - execute the test applications as native processes on the development machine
• platforms/qemu-cortex-m0 - execute the tests as fully semihosted Cortex-M0 applications on a QEMU mps2-an385 emulated board (an Arm Cortex-M3 development board)
• platforms/qemu-cortex-m3 - execute the tests as fully semihosted Cortex-M3 applications on a QEMU mps2-an385 emulated board (an Arm Cortex-M3 development board)
• platforms/qemu-cortex-m4f - execute the tests as fully semihosted Cortex-M4 applications on a QEMU mps2-an386 emulated board (an Arm Cortex-M4 development board)
• platforms/qemu-cortex-m7f - execute the tests as fully semihosted Cortex-M7 applications on a QEMU mps2-an500 emulated board (an Arm Cortex-M7 development board)
• platforms/qemu-cortex-a15 - execute the tests as fully semihosted Cortex-A15 applications on a QEMU virt emulated board
• platforms/qemu-cortex-a72 - execute the tests as fully semihosted Cortex-A72 (64-bit) applications on a QEMU virt emulated board
• platforms/qemu-riscv-rv32imac - execute the tests as fully semihosted RISC-V RV32IMAC applications on a QEMU virt emulated board
• platforms/qemu-riscv-rv64imafdc - execute the tests as fully semihosted RISC-V RV64IMAFDC applications on a QEMU virt emulated board

The build configurations utilise exactly the same source files on all platforms, without changes.

Semihosting

On embedded platforms, the applications interact with the host via the Arm semihosting mechanism.

Toolchains

To enhance source code portability, the builds are repeatedly conducted utilising various toolchains, including multiple versions of the same toolchain.

The following toolchains are utilised:

• gcc (native)
• clang (native)
• arm-none-eabi-gcc (Cortex-M, AArch32)
• aarch64-none-elf-gcc (AArch64)
• risc-none-elf-gcc (RISC-V 32/64)

Test Details

minimal-test

The minimal-test.cpp file is a concise application that illustrates how to construct the simplest possible test for verifying a condition.

> xpm run test --config native-cmake-sys-release
> cd build/native-cmake-sys-release && ctest -V
UpdateCTestConfiguration  from :/home/ilg/Work/micro-os-plus/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ilg/Work/micro-os-plus/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/DartConfiguration.tcl
Test project /home/ilg/Work/micro-os-plus/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
...
test 13
      Start 13: minimal-test-reporter-tap-verbose

13: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/minimal-test "one" "two" "--verbose" "--reporter" "tap" "--output-file" "minimal-test-verbose.tap"
13: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
13: Test timeout computed to be: 10000000
13:
13: # Running: minimal-test one two --verbose --reporter tap --output-file minimal-test-verbose.tap
13: # Built with clang Apple LLVM 17.0.0 (clang-1700.6.4.2), with exceptions
13: TAP version 14
13: # Subtest: Minimal
13:
13:     # Subtest: Check various conditions
13:         ok 1 - answer is 42
13:         ok 2 - condition is true
13:         1..2
13:     ok 1 - Check various conditions # { passed, 2 checks }
13:
13:     1..1
13: ok 1 - Minimal # { passed, 2 checks in 1 test case, time: 0.025 ms }
13:
13: 1..1
13: # { total: 2 checks passed, 0 failed, in 1 test case, 1 test suite, time: 0.074 ms }
13: Test output written to 'minimal-test-verbose.tap'.
13/63 Test #13: minimal-test-reporter-tap-verbose ...........   Passed    0.00 sec
test 14
      Start 14: minimal-test-reporter-tap-verbose-compare

14: Test command: /bin/bash "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../scripts/compare-files.sh" "minimal-test-verbose.tap" "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../assets/tap/minimal-test.tap"
14: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
14: Test timeout computed to be: 10000000
14: minimal-test-verbose.tap ok
14/63 Test #14: minimal-test-reporter-tap-verbose-compare ...   Passed    0.02 sec
test 15
      Start 15: minimal-test-reporter-tap

15: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/minimal-test "one" "two" "--reporter" "tap" "--output-file" "minimal-test.tap"
15: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
15: Test timeout computed to be: 10000000
15:
15: # Running: minimal-test one two --reporter tap --output-file minimal-test.tap
15: # Built with clang Apple LLVM 17.0.0 (clang-1700.6.4.2), with exceptions
15: TAP version 14
15: # Subtest: Minimal
15:
15:     # Subtest: Check various conditions
15:         1..0
15:     ok 1 - Check various conditions # { passed, 2 checks }
15:
15:     1..1
15: ok 1 - Minimal # { passed, 2 checks in 1 test case, time: 0.077 ms }
15:
15: 1..1
15: # { total: 2 checks passed, 0 failed, in 1 test case, 1 test suite, time: 0.164 ms }
15: Test output written to 'minimal-test.tap'.
15/63 Test #15: minimal-test-reporter-tap ...................   Passed    0.00 sec
test 16
      Start 16: minimal-test-reporter-tap-compare

16: Test command: /bin/bash "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../scripts/compare-files.sh" "minimal-test.tap" "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../assets/tap/minimal-test.tap"
16: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
16: Test timeout computed to be: 10000000
16: minimal-test.tap ok
16/63 Test #16: minimal-test-reporter-tap-compare ...........   Passed    0.02 sec
test 17
      Start 17: minimal-test-reporter-tap-quiet

17: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/minimal-test "one" "two" "--quiet" "--reporter" "tap" "--output-file" "minimal-test-quiet.tap"
17: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
17: Test timeout computed to be: 10000000
17:
17: TAP version 14
17:
17: 1..0
17: # { total: 2 checks passed, 0 failed, in 1 test case, 1 test suite, time: 0.049 ms }
17: Test output written to 'minimal-test-quiet.tap'.
17/63 Test #17: minimal-test-reporter-tap-quiet .............   Passed    0.00 sec
test 18
      Start 18: minimal-test-reporter-tap-quiet-compare

18: Test command: /bin/bash "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../scripts/compare-files.sh" "minimal-test-quiet.tap" "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../assets/tap/minimal-test.tap"
18: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
18: Test timeout computed to be: 10000000
18: minimal-test-quiet.tap ok
18/63 Test #18: minimal-test-reporter-tap-quiet-compare .....   Passed    0.02 sec
test 19
      Start 19: minimal-test-reporter-tap-silent

19: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/minimal-test "one" "two" "--silent" "--reporter" "tap" "--output-file" "minimal-test-silent.tap"
19: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
19: Test timeout computed to be: 10000000
19: Test output written to 'minimal-test-silent.tap'.
19/63 Test #19: minimal-test-reporter-tap-silent ............   Passed    0.00 sec

All verbosity modes are exercised: default, --verbose, --quiet, and --silent.

sample-test

The sample-test.cpp file is a representative application that demonstrates how to use the principal primitives provided by the library to construct a practical test.

A typical execution produces the following output:

test 39
      Start 39: sample-test-reporter-tap-verbose

39: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/sample-test "one" "two" "--verbose" "--reporter" "tap" "--output-file" "sample-test-verbose.tap"
39: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
39: Test timeout computed to be: 10000000
39:
39: # Running: sample-test one two --verbose --reporter tap --output-file sample-test-verbose.tap
39: # Built with clang Apple LLVM 17.0.0 (clang-1700.6.4.2), with exceptions
39: TAP version 14
39: # Subtest: Sample
39:
39:     # Subtest: Check various conditions
39:         ok 1 - answer eq 42
39:         ok 2 - answer ne 43
39:         ok 3 - answer lt 43
39:         ok 4 - answer le 42
39:         ok 5 - answer gt 43
39:         ok 6 - answer ge 42
39:         ok 7 - condition is true
39:         1..7
39:     ok 1 - Check various conditions # { passed, 7 checks }
39:
39:     # Subtest: Check various conditions with operators
39:         ok 1 - answer == 42 (with literal)
39:         ok 2 - answer == 42 (with cast)
39:         ok 3 - answer != 43
39:         ok 4 - answer < 43
39:         ok 5 - answer <= 42
39:         ok 6 - answer > 43
39:         ok 7 - answer >= 42
39:         1..7
39:     ok 2 - Check various conditions with operators # { passed, 7 checks }
39:
39:     # Subtest: Check strings
39:         ok 1 - ultimate answer is 'fortytwo'
39:         1..1
39:     ok 3 - Check strings # { passed, 1 check }
39:
39:     # Subtest: Check strings with operators
39:         ok 1 - ultimate answer == 'fortytwo'
39:         1..1
39:     ok 4 - Check strings with operators # { passed, 1 check }
39:
39:     # Subtest: Check compound conditions
39:         ok 1 - logical 'and' expression
39:         1..1
39:     ok 5 - Check compound conditions # { passed, 1 check }
39:
39:     # Subtest: Check compound conditions with operators
39:         ok 1 - logical 'and' expression with operators
39:         1..1
39:     ok 6 - Check compound conditions with operators # { passed, 1 check }
39:
39:     # Subtest: Check multiple function invocations
39:         ok 1 - lambda returns 42
39:         ok 2 - lambda returns 43
39:         1..2
39:     ok 7 - Check multiple function invocations # { passed, 2 checks }
39:
39:     # Subtest: Check args
39:         ok 1 - argc >= 2
39:         ok 2 - argv[1] is 'one'
39:         ok 3 - argv[2] is 'two'
39:         1..3
39:     ok 8 - Check args # { passed, 3 checks }
39:
39:     # Subtest: Check complex logic
39:         ok 1 - xyz passed
39:         1..1
39:     ok 9 - Check complex logic # { passed, 1 check }
39:
39:     # Subtest: Check if exceptions are thrown
39:         ok 1 - exception thrown
39:         ok 2 - std::runtime_error thrown
39:         1..2
39:     ok 10 - Check if exceptions are thrown # { passed, 2 checks }
39:
39:     # Subtest: Check if exceptions are not thrown
39:         ok 1 - exception not thrown
39:         1..1
39:     ok 11 - Check if exceptions are not thrown # { passed, 1 check }
39:
39:     # Subtest: Check containers
39:         ok 1 - vector{ 1, 2 } eq vector{ 1, 2 }
39:         ok 2 - vector{ 1, 2, 3 } ne vector{ 1, 2, 4 }
39:         1..2
39:     ok 12 - Check containers # { passed, 2 checks }
39:
39:     # Subtest: Check containers with operators
39:         ok 1 - vector{ 1, 2 } == vector{ 1, 2 }
39:         ok 2 - vector{ 1, 2, 3 } != vector{ 1, 2, 4 }
39:         1..2
39:     ok 13 - Check containers with operators # { passed, 2 checks }
39:
39:     1..13
39: ok 1 - Sample # { passed, 31 checks in 13 test cases, time: 0.295 ms }
39:
39: # Subtest: Args suite
39:
39:     # Subtest: args
39:         ok 1 - ic is 42
39:         ok 2 - iv is 43
39:         ok 3 - ir is 44
39:         ok 4 - *ip1 is 45
39:         ok 5 - *ip2 is 46
39:         1..5
39:     ok 1 - args # { passed, 5 checks }
39:
39:     1..1
39: ok 2 - Args suite # { passed, 5 checks in 1 test case, time: 0.017 ms }
39:
39: # Subtest: Explicit namespace suite
39:
39:     # Subtest: Check one
39:         ok 1 - Passed
39:         1..1
39:     ok 1 - Check one # { passed, 1 check }
39:
39:     # Subtest: Check two
39:         ok 1 - Passed
39:         1..1
39:     ok 2 - Check two # { passed, 1 check }
39:
39:     1..2
39: ok 3 - Explicit namespace suite # { passed, 2 checks in 2 test cases, time: 0.026 ms }
39:
39: # Subtest: Implicit namespace suite
39:
39:     # Subtest: Check one
39:         ok 1 - Passed
39:         1..1
39:     ok 1 - Check one # { passed, 1 check }
39:
39:     # Subtest: Check two
39:         ok 1 - Passed
39:         1..1
39:     ok 2 - Check two # { passed, 1 check }
39:
39:     1..2
39: ok 4 - Implicit namespace suite # { passed, 2 checks in 2 test cases, time: 0.026 ms }
39:
39: 1..4
39: # { total: 40 checks passed, 0 failed, in 18 test cases, 4 test suites, time: 0.438 ms }
39: Test output written to 'sample-test-verbose.tap'.
39/63 Test #39: sample-test-reporter-tap-verbose ............   Passed    0.00 sec

All possible verbosity modes are exercised: default, --verbose, --quiet, and --silent.

unit-test

The unit-test.cpp file constitutes the comprehensive test suite, designed to verify that all library functions behave as intended. Some checks are deliberately expected to fail, in order to validate the failure-reporting paths.

A typical execution produces the following output:

test 60
      Start 60: unit-test-reporter-human

60: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/unit-test "--reporter" "human" "--output-file" "unit-test.txt"
60: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
60: Test timeout computed to be: 10000000
60:
60: Running: unit-test --reporter human --output-file unit-test.txt
60: Built with clang Apple LLVM 17.0.0 (clang-1700.6.4.2), with exceptions
60: µTest++ human report
60:
60: • Static top suite
60:
60: µTest++ unit tests; some checks are expected to fail.
60:
60:     ✓ assume - passed (1 check)
60:     ✓ Initial counters - passed (0 checks)
60:     ✓ expect(true) - passed (5 checks)
60:
60:     • expect(false)
60:         ✗ FAILED (main.cpp:127)
60:         ✗ false FAILED (main.cpp:130)
60:     ✗ expect(false) - FAILED (0 checks passed, 2 failed)
60:
60:     ✓ reflection::type_name() - passed (4 checks)
60:     ✓ Splits - passed (8 checks)
60:     ✓ to_* modern wrappers - passed (4 checks)
60:     ✓ reflection::short_name() - passed (2 checks)
60:         ✓ inner subtest - passed (1 check)
60:     ✓ Nested subtests - passed (0 checks)
60:
60: ✗ Static top suite - FAILED (25 checks passed, 2 failed, in 10 test cases), time: 0.157 ms
60:
60: • Dynamic suite
60:
60:     ✓ check in dynamic suite - passed (1 check)
60:
60: ✓ Dynamic suite - passed (1 check in 1 test case), time: 0.018 ms
60:
60: • Combinatorial floats suite
60:
60:     ✓ Combinatorial floats - passed (21 checks)
60:     ✓ Combinatorial doubles - passed (21 checks)
60:
60: ✓ Combinatorial floats suite - passed (42 checks in 2 test cases), time: 0.063 ms
60:
60: • Combinatorial integrals suite
60:
60:     ✓ Combinatorial integrals with signed long long - passed (20 checks)
60:     ✓ Combinatorial integrals unsigned long long - passed (20 checks)
60:     ✓ Combinatorial integrals with signed int - passed (20 checks)
60:     ✓ Combinatorial integrals with unsigned int - passed (20 checks)
60:     ✓ Combinatorial integrals with signed short - passed (20 checks)
60:     ✓ Combinatorial integrals with unsigned short - passed (20 checks)
60:     ✓ Combinatorial integrals with signed char - passed (20 checks)
60:     ✓ Combinatorial integrals with unsigned char - passed (20 checks)
60:
60: ✓ Combinatorial integrals suite - passed (160 checks in 8 test cases), time: 0.223 ms
60:
60: • Comparisons suite
60:
60:     ✓ Integer comparisons - passed (8 checks)
60:
60:     • Failed integer comparisons
60:         ✗ actual != 42 FAILED (comparisons.cpp:115, 42 != 42)
60:         ✗ actual+1 == 42 FAILED (comparisons.cpp:118, 43 == 42)
60:         ✗ actual-1 >= 42 FAILED (comparisons.cpp:122, 41 >= 42)
60:         ✗ actual > 42 FAILED (comparisons.cpp:126, 42 > 42)
60:         ✗ actual+1 <= 42 FAILED (comparisons.cpp:129, 43 <= 42)
60:         ✗ actual < 42 FAILED (comparisons.cpp:133, 42 < 42)
60:         ✗ 42_i != 42_i FAILED (comparisons.cpp:139, 42 != 42)
60:         ✗ 42 != 42_i FAILED (comparisons.cpp:142, 42 != 42)
60:     ✗ Failed integer comparisons - FAILED (0 checks passed, 8 failed)
60:
60:     ✓ Float comparisons - passed (27 checks)
60:
60:     • Failed float comparisons
60:         ✗ actual != 42.0 FAILED (comparisons.cpp:264, 42f != 42f)
60:         ✗ actual == 43.0 FAILED (comparisons.cpp:267, 42f == 43f)
60:         ✗ actual != 42 FAILED (comparisons.cpp:270, 42f != 42)
60:         ✗ actual == 43 FAILED (comparisons.cpp:273, 42f == 43)
60:         ✗ 42.101f != 42.101_f FAILED (comparisons.cpp:279, 42.101f != 42.101f)
60:         ✗ 42.101f epsilon 0.01f != 42.10_f FAILED (comparisons.cpp:282, 42.101f != 42.1f)
60:         ✗ 42.101f epsilon 0.1f == 42.10_f FAILED (comparisons.cpp:286, 42.101f == 42.1f)
60:         ✗ 42.1010001f epsilon 0.1f != 42.1_f FAILED (comparisons.cpp:290, 42.101f != 42.1f)
60:         ✗ 42.101f == 42.10_f FAILED (comparisons.cpp:294, 42.101f == 42.1f)
60:         ✗ 42.101f == 42.100_f FAILED (comparisons.cpp:297, 42.101f == 42.1f)
60:         ✗ 42.10f == 42.1_f FAILED (comparisons.cpp:300, 42.1f != 42.1f)
60:         ✗ 42.42f == 42.42_f FAILED (comparisons.cpp:303, 42.42f != 42.42f)
60:         ✗ 42.42 == 42.420_d FAILED (comparisons.cpp:306, 42.42 != 42.42)
60:         ✗ 42.0 == 42.0_d FAILED (comparisons.cpp:309, 42 != 42)
60:         ✗ 42. == 42._d FAILED (comparisons.cpp:312, 42 != 42)
60:         ✗ 42.42 == 42.42_ld FAILED (comparisons.cpp:315, 42.42l != 42.42l)
60:         ✗ 1234._f == 1234.f FAILED (comparisons.cpp:319, 1234f != 1234f)
60:         ✗ 1234.56_f == 1234.56f FAILED (comparisons.cpp:322, 1234.56f != 1234.56f)
60:         ✗ 12345678.9f == 12345678.9_f FAILED (comparisons.cpp:325, 12345679f != 12345679f)
60:         ✗ 111111.42f == 111111.42_f FAILED (comparisons.cpp:329, 111111.42f != 111111.42f)
60:         ✗ 1111111111.42 == 1111111111.42_d FAILED (comparisons.cpp:333, 1111111111.42 != 1111111111.42)
60:     ✗ Failed float comparisons - FAILED (0 checks passed, 21 failed)
60:
60:     ✓ String comparisons - passed (6 checks)
60:
60:     • Failed string comparisons
60:         ✗ actual_sv != abc_sv FAILED (comparisons.cpp:392, abc != abc)
60:         ✗ actual_sv == abx_sv FAILED (comparisons.cpp:396, abc == abx)
60:         ✗ actual_sv >= abd_sv FAILED (comparisons.cpp:400, abc >= abd)
60:         ✗ actual_sv > abc_sv FAILED (comparisons.cpp:404, abc > abc)
60:         ✗ actual_sv <= abb_sv FAILED (comparisons.cpp:408, abc <= abb)
60:         ✗ actual_sv < abc_sv FAILED (comparisons.cpp:412, abc < abc)
60:     ✗ Failed string comparisons - FAILED (0 checks passed, 6 failed)
60:
60:     ✓ Pointer comparisons - passed (9 checks)
60:
60:     • Failed pointer comparisons
60:         ✗ ptr1 != &one FAILED (comparisons.cpp:494, 0x7ff7b396c45c != 0x7ff7b396c45c)
60:         ✗ ptr1 != ptr2 FAILED (comparisons.cpp:497, 0x7ff7b396c45c != 0x7ff7b396c45c)
60:         ✗ ptr1 > ptr2 FAILED (comparisons.cpp:500, 0x7ff7b396c45c > 0x7ff7b396c45c)
60:         ✗ ptr1 < ptr2 FAILED (comparisons.cpp:503, 0x7ff7b396c45c < 0x7ff7b396c45c)
60:         ✗ ptr1 == a_non_nullptr FAILED (comparisons.cpp:506, 0x7ff7b396c45c == 0x7ff7b396c450)
60:         ✗ pfunc != afunc FAILED (comparisons.cpp:509, 0x10c5eb7c0 != 0x10c5eb7c0)
60:         ✗ pfunc == a_non_nullptr FAILED (comparisons.cpp:512, 0x10c5eb7c0 == 0x7ff7b396c450)
60:         ✗ pone >= ptwo FAILED (comparisons.cpp:524, 0x7ff7b396c448 >= 0x7ff7b396c44c)
60:         ✗ ptwo <= pone FAILED (comparisons.cpp:527, 0x7ff7b396c44c <= 0x7ff7b396c448)
60:     ✗ Failed pointer comparisons - FAILED (0 checks passed, 9 failed)
60:
60:     ✓ Null pointer comparisons - passed (3 checks)
60:
60:     • Failed null pointer comparisons
60:         ✗ a_nullptr != nullptr FAILED (comparisons.cpp:575, 0x0 != nullptr)
60:         ✗ a_non_nullptr == nullptr FAILED (comparisons.cpp:578, 0x7ff7b396c468 == nullptr)
60:         ✗ pfunc != nullptr FAILED (comparisons.cpp:581, 0x0 != nullptr)
60:     ✗ Failed null pointer comparisons - FAILED (0 checks passed, 3 failed)
60:
60:     • Logical operations
60:         ✓ not (actual != 42)
60:         ✗ not (actual == 42) FAILED (comparisons.cpp:602, not 42 == 42)
60:         ✓ (actual == 42) and (actual == 42.0)
60:         ✗ (actual == 42) and (actual != 42.0) FAILED (comparisons.cpp:611, (42 == 42 and 42f != 42))
60:         ✗ (actual != 42) and (actual == 42.0) FAILED (comparisons.cpp:616, (42 != 42 and 42f == 42))
60:         ✗ (actual != 42) and (actual != 42.0) FAILED (comparisons.cpp:621, (42 != 42 and 42f != 42))
60:         ✓ (42 == 42 and 42f == 42)
60:         ✗ FAILED (comparisons.cpp:630, (42 == 42 and 42f != 42))
60:         ✓ (actual == 42) or (actual == 42.0)
60:         ✓ (actual == 42) or (actual != 42.0)
60:         ✓ (actual != 42) or (actual == 42.0)
60:         ✗ (actual != 42) or (actual != 42.0) FAILED (comparisons.cpp:649, (42 != 42 or 42f != 42))
60:     ✗ Logical operations - FAILED (6 checks passed, 6 failed)
60:
60: ✗ Comparisons suite - FAILED (59 checks passed, 53 failed, in 11 test cases), time: 0.309 ms
60:
60: • Exceptions suite
60:
60:     ✓ thrown exceptions - passed (2 checks)
60:
60:     • Failed thrown exceptions
60:         ✗ exception thrown FAILED (exceptions.cpp:91, throws)
60:         ✗ std::runtime_error thrown FAILED (exceptions.cpp:101, throws<std::runtime_error>)
60:     ✗ Failed thrown exceptions - FAILED (0 checks passed, 2 failed)
60:
60:     ✓ Not thrown exceptions - passed (1 check)
60:
60:     • Failed not thrown exceptions
60:         ✗ exception not thrown FAILED (exceptions.cpp:142, nothrow)
60:     ✗ Failed not thrown exceptions - FAILED (0 checks passed, 1 failed)
60:
60: ✗ Exceptions suite - FAILED (3 checks passed, 3 failed, in 4 test cases), time: 0.159 ms
60:
60: • Failed combinatorial floats suite
60:
60:     • Combinatorial floats
60:         ✗ 42.0 == 42 signed long long FAILED (float-failed-combinations.cpp:36, 42f != 42ll)
60:         ✗ 42.0 == 42 unsigned long long FAILED (float-failed-combinations.cpp:41, 42f != 42ull)
60:         ✗ 42.0 == 42 signed long FAILED (float-failed-combinations.cpp:46, 42f != 42l)
60:         ✗ 42.0 == 42 unsigned long FAILED (float-failed-combinations.cpp:50, 42f != 42ul)
60:         ✗ 42.0 == 42 signed int FAILED (float-failed-combinations.cpp:54, 42f != 42)
60:         ✗ 42.0 == 42 unsigned int FAILED (float-failed-combinations.cpp:58, 42f != 42u)
60:         ✗ 42.0 == 42 signed short FAILED (float-failed-combinations.cpp:62, 42f != 42s)
60:         ✗ 42.0 == 42 unsigned short FAILED (float-failed-combinations.cpp:66, 42f != 42us)
60:         ✗ 42.0 == 42 signed char FAILED (float-failed-combinations.cpp:71, 42f != 42c)
60:         ✗ 42.0 == 42 unsigned char FAILED (float-failed-combinations.cpp:75, 42f != 42uc)
60:         ✗ 42.0 <= 42 signed int FAILED (float-failed-combinations.cpp:81, 42f > 42)
60:         ✗ 42.0 >= 42 signed int FAILED (float-failed-combinations.cpp:85, 42f < 42)
60:         ✗ 43.0 != 42 signed int FAILED (float-failed-combinations.cpp:89, 43f == 42)
60:         ✗ 41.0 < 42 signed int FAILED (float-failed-combinations.cpp:94, 41f >= 42)
60:         ✗ 43.0 > 42 signed int FAILED (float-failed-combinations.cpp:99, 43f <= 42)
60:         ✗ 42.0 <= 42 unsigned int FAILED (float-failed-combinations.cpp:106, 42f > 42u)
60:         ✗ 42.0 >= 42 unsigned int FAILED (float-failed-combinations.cpp:110, 42f < 42u)
60:         ✗ 43.0 != 42 unsigned int FAILED (float-failed-combinations.cpp:114, 43f == 42u)
60:         ✗ 41.0 < 42 unsigned int FAILED (float-failed-combinations.cpp:119, 41f >= 42u)
60:         ✗ 43.0 > 42 unsigned int FAILED (float-failed-combinations.cpp:124, 43f <= 42u)
60:         ✗ 42 == 42.0 FAILED (float-failed-combinations.cpp:131, 42 != 42f)
60:     ✗ Combinatorial floats - FAILED (0 checks passed, 21 failed)
60:
60:     • Combinatorial doubles
60:         ✗ 42.0 == 42 signed long long FAILED (float-failed-combinations.cpp:36, 42 != 42ll)
60:         ✗ 42.0 == 42 unsigned long long FAILED (float-failed-combinations.cpp:41, 42 != 42ull)
60:         ✗ 42.0 == 42 signed long FAILED (float-failed-combinations.cpp:46, 42 != 42l)
60:         ✗ 42.0 == 42 unsigned long FAILED (float-failed-combinations.cpp:50, 42 != 42ul)
60:         ✗ 42.0 == 42 signed int FAILED (float-failed-combinations.cpp:54, 42 != 42)
60:         ✗ 42.0 == 42 unsigned int FAILED (float-failed-combinations.cpp:58, 42 != 42u)
60:         ✗ 42.0 == 42 signed short FAILED (float-failed-combinations.cpp:62, 42 != 42s)
60:         ✗ 42.0 == 42 unsigned short FAILED (float-failed-combinations.cpp:66, 42 != 42us)
60:         ✗ 42.0 == 42 signed char FAILED (float-failed-combinations.cpp:71, 42 != 42c)
60:         ✗ 42.0 == 42 unsigned char FAILED (float-failed-combinations.cpp:75, 42 != 42uc)
60:         ✗ 42.0 <= 42 signed int FAILED (float-failed-combinations.cpp:81, 42 > 42)
60:         ✗ 42.0 >= 42 signed int FAILED (float-failed-combinations.cpp:85, 42 < 42)
60:         ✗ 43.0 != 42 signed int FAILED (float-failed-combinations.cpp:89, 43 == 42)
60:         ✗ 41.0 < 42 signed int FAILED (float-failed-combinations.cpp:94, 41 >= 42)
60:         ✗ 43.0 > 42 signed int FAILED (float-failed-combinations.cpp:99, 43 <= 42)
60:         ✗ 42.0 <= 42 unsigned int FAILED (float-failed-combinations.cpp:106, 42 > 42u)
60:         ✗ 42.0 >= 42 unsigned int FAILED (float-failed-combinations.cpp:110, 42 < 42u)
60:         ✗ 43.0 != 42 unsigned int FAILED (float-failed-combinations.cpp:114, 43 == 42u)
60:         ✗ 41.0 < 42 unsigned int FAILED (float-failed-combinations.cpp:119, 41 >= 42u)
60:         ✗ 43.0 > 42 unsigned int FAILED (float-failed-combinations.cpp:124, 43 <= 42u)
60:         ✗ 42 == 42.0 FAILED (float-failed-combinations.cpp:131, 42 != 42)
60:     ✗ Combinatorial doubles - FAILED (0 checks passed, 21 failed)
60:
60: ✗ Failed combinatorial floats suite - FAILED (0 checks passed, 42 failed, in 2 test cases), time: 0.079 ms
60:
60: • Failed combinatorial integrals suite
60:
60:     • Combinatorial integrals with signed long long
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42ll != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42ll != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42ll != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42ll != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42ll != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42ll != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42ll != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42ll != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42ll != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42ll != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42ll > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42ll < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43ll == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41ll >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43ll <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42ll > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42ll < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43ll == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41ll >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43ll <= 42u)
60:     ✗ Combinatorial integrals with signed long long - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals unsigned long long
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42ull != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42ull != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42ull != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42ull != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42ull != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42ull != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42ull != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42ull != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42ull != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42ull != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42ull > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42ull < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43ull == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41ull >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43ull <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42ull > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42ull < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43ull == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41ull >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43ull <= 42u)
60:     ✗ Combinatorial integrals unsigned long long - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with signed int
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42 != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42 != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42 != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42 != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42 != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42 != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42 != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42 != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42 != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42 != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42 > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42 < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43 == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41 >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43 <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42 > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42 < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43 == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41 >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43 <= 42u)
60:     ✗ Combinatorial integrals with signed int - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with unsigned int
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42u != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42u != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42u != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42u != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42u != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42u != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42u != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42u != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42u != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42u != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42u > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42u < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43u == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41u >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43u <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42u > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42u < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43u == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41u >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43u <= 42u)
60:     ✗ Combinatorial integrals with unsigned int - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with signed short
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42 != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42 != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42 != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42 != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42 != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42 != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42 != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42 != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42 != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42 != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42 > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42 < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43 == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41 >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43 <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42 > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42 < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43 == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41 >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43 <= 42u)
60:     ✗ Combinatorial integrals with signed short - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with unsigned short
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42u != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42u != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42u != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42u != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42u != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42u != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42u != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42u != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42u != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42u != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42u > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42u < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43u == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41u >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43u <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42u > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42u < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43u == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41u >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43u <= 42u)
60:     ✗ Combinatorial integrals with unsigned short - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with signed short
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42s != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42s != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42s != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42s != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42s != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42s != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42s != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42s != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42s != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42s != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42s > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42s < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43s == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41s >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43s <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42s > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42s < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43s == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41s >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43s <= 42u)
60:     ✗ Combinatorial integrals with signed short - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with unsigned short
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42us != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42us != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42us != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42us != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42us != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42us != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42us != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42us != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42us != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42us != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42us > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42us < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43us == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41us >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43us <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42us > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42us < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43us == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41us >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43us <= 42u)
60:     ✗ Combinatorial integrals with unsigned short - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with signed char
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42c != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42c != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42c != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42c != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42c != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42c != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42c != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42c != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42c != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42c != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42c > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42c < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43c == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41c >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43c <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42c > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42c < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43c == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41c >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43c <= 42u)
60:     ✗ Combinatorial integrals with signed char - FAILED (0 checks passed, 20 failed)
60:
60:     • Combinatorial integrals with unsigned char
60:         ✗ ne matches signed long long FAILED (integral-failed-combinations.cpp:36, 42uc != 42ll)
60:         ✗ ne matches unsigned long long FAILED (integral-failed-combinations.cpp:41, 42uc != 42ull)
60:         ✗ ne matches signed long FAILED (integral-failed-combinations.cpp:46, 42uc != 42l)
60:         ✗ ne matches unsigned long FAILED (integral-failed-combinations.cpp:51, 42uc != 42ul)
60:         ✗ ne matches signed int FAILED (integral-failed-combinations.cpp:56, 42uc != 42)
60:         ✗ ne matches unsigned int FAILED (integral-failed-combinations.cpp:60, 42uc != 42u)
60:         ✗ ne matches signed short FAILED (integral-failed-combinations.cpp:65, 42uc != 42s)
60:         ✗ ne matches unsigned short FAILED (integral-failed-combinations.cpp:70, 42uc != 42us)
60:         ✗ ne matches signed char FAILED (integral-failed-combinations.cpp:75, 42uc != 42c)
60:         ✗ ne matches unsigned char FAILED (integral-failed-combinations.cpp:80, 42uc != 42uc)
60:         ✗ gt matches signed int FAILED (integral-failed-combinations.cpp:87, 42uc > 42)
60:         ✗ lt matches signed int FAILED (integral-failed-combinations.cpp:91, 42uc < 42)
60:         ✗ eq matches signed int FAILED (integral-failed-combinations.cpp:95, 43uc == 42)
60:         ✗ ge matches signed int FAILED (integral-failed-combinations.cpp:100, 41uc >= 42)
60:         ✗ le matches signed int FAILED (integral-failed-combinations.cpp:105, 43uc <= 42)
60:         ✗ gt matches unsigned int FAILED (integral-failed-combinations.cpp:110, 42uc > 42u)
60:         ✗ lt matches unsigned int FAILED (integral-failed-combinations.cpp:115, 42uc < 42u)
60:         ✗ eq matches unsigned int FAILED (integral-failed-combinations.cpp:120, 43uc == 42u)
60:         ✗ ge matches unsigned int FAILED (integral-failed-combinations.cpp:125, 41uc >= 42u)
60:         ✗ le matches unsigned int FAILED (integral-failed-combinations.cpp:130, 43uc <= 42u)
60:     ✗ Combinatorial integrals with unsigned char - FAILED (0 checks passed, 20 failed)
60:
60: ✗ Failed combinatorial integrals suite - FAILED (0 checks passed, 200 failed, in 10 test cases), time: 0.486 ms
60:
60: • Miscellaneous suite
60:
60:     ✓ Arrays - passed (4 checks)
60:
60:     • Arrays bad
60:         ✗ array{ 42 } ne array{ 42 } FAILED (miscellaneous.cpp:68, {42} != {42})
60:         ✗ array{ 42 } eq array{ 43 } FAILED (miscellaneous.cpp:72, {42} == {43})
60:         ✗ array{ 1, 2 } ne array{ 1, 2 } FAILED (miscellaneous.cpp:76, {1, 2} != {1, 2})
60:         ✗ array{ 1, 2, 3 } eq array{ 1, 2, 4 } FAILED (miscellaneous.cpp:80, {1, 2, 3} == {1, 2, 4})
60:     ✗ Arrays bad - FAILED (0 checks passed, 4 failed)
60:
60:     ✓ Vectors - passed (6 checks)
60:
60:     • Vectors bad
60:         ✗ vector{ } != vector{ } FAILED (miscellaneous.cpp:132, {} != {})
60:         ✗ FAILED (miscellaneous.cpp:136, {} != {})
60:         ✗ vector{ 42 } != vector{ 42 } FAILED (miscellaneous.cpp:139, {42} != {42})
60:         ✗ vector{ 42 } == vector{ 43 } FAILED (miscellaneous.cpp:143, {42} == {43})
60:         ✗ vector{ 1, 2 } != vector{ 1, 2 } FAILED (miscellaneous.cpp:147, {1, 2} != {1, 2})
60:         ✗ vector{ 1, 2, 3 } == vector{ 1, 2, 4 } FAILED (miscellaneous.cpp:151, {1, 2, 3} == {1, 2, 4})
60:     ✗ Vectors bad - FAILED (0 checks passed, 6 failed)
60:
60: ✗ Miscellaneous suite - FAILED (10 checks passed, 10 failed, in 4 test cases), time: 0.082 ms
60:
60: • Operators suite
60:
60:     • Operators
60:         ✓ actual == 42
60:         ✗ actual+1 == 42 with scalar FAILED (operators.cpp:48)
60:         ✗ actual+1 == 42 with _i literal FAILED (operators.cpp:52, 43 == 42)
60:         ✗ actual+1 == 42 with _i() FAILED (operators.cpp:56, 43 == 42)
60:         ✓ actual+1 != 42
60:         ✓ actual-1 < 42
60:         ✓ actual <= 42
60:         ✓ actual+1 > 42
60:         ✓ actual >= 42
60:         ✓ not (actual != 42)
60:         ✗ not (actual == 42) FAILED (operators.cpp:78, not 42 == 42)
60:         ✓ (actual == 42) and (actual == 42.0)
60:         ✗ (actual == 42) and (actual != 42.0) FAILED (operators.cpp:86, (42 == 42 and 42f != 42))
60:         ✗ (actual != 42) and (actual == 42.0) FAILED (operators.cpp:91, (42 != 42 and 42f == 42))
60:         ✗ (actual != 42) and (actual != 42.0) FAILED (operators.cpp:96, (42 != 42 and 42f != 42))
60:         ✓ (42 == 42 and 42f == 42)
60:         ✗ FAILED (operators.cpp:105, (42 == 42 and 42f != 42))
60:         ✓ (actual == 42) or (actual == 42.0)
60:         ✓ (actual == 42) or (actual != 42.0)
60:         ✓ (actual != 42) or (actual == 42.0)
60:         ✗ (actual != 42) or (actual != 42.0) FAILED (operators.cpp:124, (42 != 42 or 42f != 42f))
60:     ✗ Operators - FAILED (12 checks passed, 9 failed)
60:
60:     • String view operators
60:         ✓ equal string views with ==
60:         ✗ unequal string views with == FAILED (operators.cpp:147, hello == world)
60:         ✓ unequal string views with !=
60:         ✗ equal string views with != FAILED (operators.cpp:155, hello != hello)
60:     ✗ String view operators - FAILED (2 checks passed, 2 failed)
60:
60: ✗ Operators suite - FAILED (14 checks passed, 11 failed, in 2 test cases), time: 0.061 ms
60:
60: • Timings suite
60:
60:     ✓ timestamp::has_clock() - passed (3 checks)
60:     ✓ timestamp copy semantics - passed (5 checks)
60:     ✓ timestamps::has_timestamps() state - passed (5 checks)
60:     ✓ timestamps::timestamp_begin() idempotency - passed (2 checks)
60:     ✓ timestamps::timestamp_end() idempotency - passed (2 checks)
60:     ✓ timestamps::compute_elapsed_time() - passed (10 checks)
60:
60: ✓ Timings suite - passed (27 checks in 6 test cases), time: 0.077 ms
60:
60: • Utility suite
60:
60:     ✓ utility::extract_file_name() - passed (5 checks)
60:     ✓ utility::is_match() — passing - passed (11 checks)
60:
60:     • utility::is_match() — failing
60:         ✗ "" matches abc FAILED (utility.cpp:152)
60:         ✗ abc matches b?? FAILED (utility.cpp:156)
60:         ✗ abc matches a*d FAILED (utility.cpp:160)
60:         ✗ abc matches *C FAILED (utility.cpp:164)
60:         ✗ ab matches ??? FAILED (utility.cpp:168)
60:     ✗ utility::is_match() — failing - FAILED (0 checks passed, 5 failed)
60:
60: ✗ Utility suite - FAILED (16 checks passed, 5 failed, in 3 test cases), time: 0.055 ms
60:
60: ✗ Total: 357 checks passed, 326 failed, in 63 test cases, 12 test suites, time: 1.899 ms
60: Overall, the µTest++ unit tests were successful!
60:
60: Test output written to 'unit-test.txt'.
60/63 Test #60: unit-test-reporter-human ....................   Passed    0.01 sec
test 61
      Start 61: unit-test-reporter-human-compare

61: Test command: /bin/bash "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../scripts/compare-files.sh" "unit-test.txt" "/Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/platforms/native/../../assets/txt/unit-test.txt"
61: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
61: Test timeout computed to be: 10000000
61: unit-test.txt ok
61/63 Test #61: unit-test-reporter-human-compare ............   Passed    0.03 sec
test 62
      Start 62: unit-test-reporter-human-quiet

62: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/unit-test "--quiet" "--reporter" "human"
62: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
62: Test timeout computed to be: 10000000
62:
62: µTest++ human report
62:
62: ✗ Total: 357 checks passed, 326 failed, in 63 test cases, 12 test suites, time: 0.758 ms
62/63 Test #62: unit-test-reporter-human-quiet ..............   Passed    0.00 sec
test 63
      Start 63: unit-test-reporter-human-silent

63: Test command: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin/unit-test "--silent" "--reporter" "human"
63: Working Directory: /Users/ilg/MyProjects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/tests/build/native-cmake-sys-release/platform-bin
63: Test timeout computed to be: 10000000
63/63 Test #63: unit-test-reporter-human-silent .............   Passed    0.00 sec

All verbosity modes are evaluated: default, verbose, quiet, and silent (for conciseness, the complete verbose output is not presented here).

Manual tests

There are predefined xpm actions available to manually execute various tests on the local development machine.

It is recommended to commence by performing some clean-ups (not necessary following the initial git clone):

npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests install
xpm run deep-clean -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

tip

During the initial execution, the installation step may require a considerable amount of time, as it needs to download the toolchain archives, which can be relatively large, up to several hundred megabytes.

To execute the tests with the system compiler (usually not available on Windows):

xpm run install -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests
xpm run test -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

To execute a selection of tests with the latest versions of toolchains:

xpm run install-selected -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests
xpm run test-selected -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

To execute all tests with the latest versions of toolchains:

xpm run install-latest -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests
xpm run test-latest -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

To execute all tests with all toolchain versions:

xpm run install-all -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests
xpm run test-all -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

Clean-ups

Executing all tests may require several gigabytes of space.

To clean up the tests folder, utilise:

npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests install
xpm run deep-clean -C ~/Work/micro-os-plus/micro-test-plus-xpack.git/tests

A considerable amount of space may also be utilised by the toolchains. When no longer required, they can be removed with xpm uninstall.

xpm clean-ups

For a comprehensive clean-up, please note that xpm utilises only two folders:

Windows

• %APPDATA%\Roaming\xPacks
• %APPDATA%\Local\Caches\xPacks

macOS

• ${HOME}/Library/xPacks
• ${HOME}/Library/Caches/xPacks

GNU/Linux

• ${HOME}/.local/xPacks
• ${HOME}/.cache/xPacks

They can be removed at any time and space reclaimed; xpm will recreate them during new installations.

However, projects linking to the user's global xPacks store will fail with broken paths.

Continuous Integration (CI)

The library is subject to continuous integration (CI) testing with every push utilising GitHub Actions. This ensures compatibility and stability across Ubuntu, macOS, and Windows operating systems.

Testing is conducted on both 32-bit and 64-bit bare-metal platforms, including Arm Cortex-M0, Cortex-M3, Cortex-M4F, Cortex-M7F, Cortex-A15, Cortex-A72, RISC-V RV32IMAC, and RV64IMAFDC. Additionally, native testing is performed utilising GCC and LLVM/clang compilers, ensuring comprehensive validation across various environments.

The website

The project includes the source files for its website. The main pages are generated with Docusaurus, and the reference pages are generated with Doxygen.

Regenerate the Doxygen pages

Execute the generate-website-doxygen npm script:

npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/website run generate-website-doxygen

At present, the doxygen programme must be installed on the host, as it is not available as an xpm dependency.

With the current configuration, the static HTML pages are generated within the website/static/doxygen folder and converted to docusaurus markdown in docs/api.

Prepare the website

The documentation site is built using Docusaurus and published on the project GitHub Pages.

• Switch to the xpack-development branch
• Execute the website-generate-commons npm script in the website/package.json

  npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/website run website-generate-commons

• Commit all changes

Build the static website locally

Validate the website content by performing a local build using the npm build script:

npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/website run build

Resolve any broken links that may be identified during the build process.

Start the local web server

Execute the npm script clear followed by start in the website sub-project, or execute the following within the project folder:

npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/website run clear
npm --prefix ~/Work/micro-os-plus/micro-test-plus-xpack.git/website run start

Navigate to the Maintainer Info page, the Start the local web server section.