Getting Started with the µTest++ Testing Framework

Dedicated Pages for Different Audiences

To cater to the diverse requirements of its audience, this site offers distinct pages: User's Guide, Contributor's Guide and Maintainer's Guide. Each guide is tailored to ensure users can efficiently access the information and resources most pertinent to their specific roles or needs.

Overview

The µTest++ Testing Framework project (micro test plus plus) is a C++ source code library that offers a testing framework characterised by a minimal memory footprint, primarily intended for executing unit tests on embedded platforms in a reproducible manner.

What are reproducible operations?

To be reproducible, an operation must remain stable over time and across different environments. In other words, if the same operation is repeated after some time, possibly on a different machine or platform, the resulting behaviour must be functionally equivalent.

info

In this context, the term embedded pertains specifically to bare-metal systems, excluding embedded Linux. The latter requires an entirely distinct methodology for developing Linux user-space applications.

The open-source µTest++ Testing Framework project is available on GitHub under the repository name micro-os-plus/micro-test-plus-xpack. Additionally, the content is published on the npmjs.com repository as @micro-os-plus/micro-test-plus, making it accessible for integration into projects as an xpm dependency.

What are xpm packages? (xPacks)

xpm packages, also known as xPacks, are highly adaptable and language-neutral software solutions. They have been effectively employed in C and C++ projects, demonstrating their versatility and practicality.

What the heck are xpm packages (xPacks)? Please, do not introduce another package format!

At first glance, the use of xpm packages may appear complex; however, in practice, they are remarkably straightforward. Their design philosophy is centred on automating frequent tasks in software development, such as dependency installation, and ensuring reproducibility.

xpm packages are managed by xpm (the xPack Project Manager), a utility that complements the npm CLI, the widely-used JavaScript package manager, by introducing language-neutral features.

The xPacks Framework does not establish a new package format. Instead, it adopts the standard npm package format, which comprises a collection of files and directories, along with a package.json file containing the metadata.

xpm is capable of installing packages from the same repositories as npm, whether they are public or private.

Packages, which are typically regular archives or git repositories, are extracted into distinct folders within the project.

Based on their contents, xpm packages fall into two categories:

• Source xpm packages: These contain source libraries.
• Binary xpm packages: These typically consist of executables, often platform-specific binary tools.

Source xpm packages are standard npm packages that incorporate library source files, usually written in C/C++, although they are not restricted to any particular programming language.

Binary xpm packages are also standard npm packages but have minimal contents, generally limited to a package.json file. Since these packages provide executables for multiple platforms — many of which can be considerably large — it is impractical to include all binaries within the npm package itself. Instead, the package.json file includes references (URLs) to platform-specific binary archives (e.g., .tar.gz for Unix or .zip for Windows).

These archives, along with the package metadata, are extracted. Additionally, links or forwarders to the executables are created in a .bin folder, thereby eliminating the need to add multiple directories to the system PATH.

In conclusion, xpm packages simplify and streamline the installation process for source libraries and tools, enhancing efficiency and automation. For further information, please refer to the xpm Concepts page.

Features

The principal attributes of the µTest++ Testing Framework, largely derived from Boost UT, are as follows:

• Designed to facilitate testing of both C and C++ projects.
• Written in modern C++20 (which consequently elevated the entire µOS++ project to require C++20).
• Macro-free implementation (whilst retaining the valuable feature of reporting the file name and line number for failed tests).
• Support for expectations, assumptions, and exceptions.
• Provision of test cases and test suites.
• Automatic registration of test suites.

Key distinctions from Boost UT include:

• Reduced memory footprint, owing to the absence of dependencies on the standard C++ stream library.
• A slightly simplified API.

Benefits

• Versatility in testing both C and C++ projects: Delivers a streamlined testing process suitable for projects encompassing both languages.
• Adoption of modern C++20 features for optimal performance: Improves code readability and maintainability.
• Reduced complexity and enhanced clarity through the absence of macros: Simplifies the testing workflow.
• Comprehensive testing capabilities: Efficiently addresses a wide range of scenarios.
• Structured methodology for organising and executing tests: Facilitates straightforward management and scalability of test cases.
• Simplified management of test suites via automated registration: Minimises manual intervention and enhances efficiency.
• Superior performance on embedded platforms with constrained memory resources: Optimises utilisation and overall system performance.

Install

The source code library can be installed either automatically via xpm or manually. Manual installation can be performed by copying the necessary files directly into the target project or by linking the entire project as a Git submodule.

Comprehensive instructions for installing the µTest++ Testing Framework are available in the Install Guide.

Status

The µTest++ Testing Framework project is robust, reliable, and fully operational, making it well-suited for deployment in production environments.

Testing

The library undergoes continuous integration (CI) testing with every push using GitHub Actions. This ensures compatibility and stability across Ubuntu, macOS, and Windows operating systems.

The tests are conducted on both 32-bit and 64-bit bare-metal platforms, including Arm Cortex-M0, Cortex-M3, Cortex-M4F, Cortex-M7F, Cortex-A15, Cortex-A72, RISC-V RV32IMAC, and RV64IMAFDC. Additionally, native testing is performed using GCC and LLVM/clang compilers, ensuring thorough validation across various environments.

Release schedule

The project follows a continuous release strategy, with updates being rolled out progressively as development advances, rather than adhering to a fixed release schedule.

Releases

A complete list of releases can be found on the Releases pages.

Change log

The release notes and change log can be accessed in the repository's CHANGELOG.md file.

Support & feedback

For prompt support and to provide feedback, it is recommended to utilise GitHub Discussions.

For more detailed information, please visit the Help Centre page.

Compatibility notices

According to SemVer rules:

Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API.

The breaking changes introduced in each major release are documented in reverse chronological order. This information provides a comprehensive overview of significant modifications and is readily available in the respective release notes.

v3.x (2022-04-03)

Significant revision, introducing a comprehensive set of comparators, exception handling, function templates for test cases, and class templates for test suites.

v2.3.x (2022-02-19)

The run_test_case(func, name) function has been deprecated in favour of run_test_case(name, func), in preparation for the adoption of variadic templates.

v2.x (2021-03-01)

The C++ namespace was changed from os to micro_os_plus.

v1.x (2021-02-04)

The initial codebase, noted for its simplicity and inspired by node-tap, was extracted from the mono-repo µOS++ project.

License

Unless explicitly mentioned otherwise, the original content is distributed under the terms of the MIT License, with all rights reserved by Liviu Ionescu.

The code incorporated from Boost UT is distributed under the terms of the Boost Software License, Version 1.0.

Credits

Grateful acknowledgement is extended to:

• Node tap – for providing an exemplary demonstration that testing frameworks need not be overly complex, and for inspiring the initial versions of this project.
• Boost UT – for illustrating that, in C++, it is possible to obtain file names and line numbers without resorting to cumbersome C macros, and for contributing significant portions of the codebase.
• Doxygen – for the documentation site generator.
• doxygen-awesome-css – for the attractive Doxygen theme.
• SVG REPO – for the SVG icons.

Thanks to Shields IO for the badges.

Enjoyed Using This Project? Let Us Know!

If you've found this project valuable, we'd love to hear from you! Here are some ways you can show your support:

• Donate: Every contribution, no matter the size, helps sustain the xPack and µOS++. Your generosity keeps the development alive and thriving.
• Star the project on GitHub: Help others discover our work by giving it a star — it makes a difference!
• Follow Us on X/Twitter: Stay informed about our latest updates, releases, and news by following our account.

Your support is greatly appreciated and motivates us to continue improving!