Skip to main content

How to integrate the µTest++ Testing Framework

GitHub package.json version NPM Version NPM Downloads NPM Downloads licence

As an xpm package, the most convenient method to integrate the µTest++ Testing Framework into a project is to add it as a dependency via xpm.

However, as with any source code library, this project may also be integrated into another project in the traditional manner, either by copying the relevant files into the target project, or by linking the entire project as a Git submodule.

xpm packages (xPacks) refresher

xpm packages, abbreviated as xPacks, are general-purpose, language-neutral software packages. They use the same format as npm packages, which comprises a collection of files/folders and a package.json file with the package metadata.

xpm can install source and binary packages.

For more details, see the explanation on the Getting Started page.

Automated installation​

Alongside the source files, this project also includes a package.json file containing the metadata that identifies it as an xpm/npm package, enabling direct installation from GitHub or from the npmjs.com registry as @micro-os-plus/micro-test-plus.

Prerequisites​

The only requirement for an automated installation is a recent xpm, which is a portable Node.js command line application that complements npm with several additional features specific to C/C++ projects.

To install xpm, follow the instructions on the xpm install page.

If already installed, it is recommended to update it to the latest version with:

npm install --location=global xpm@latest
tip

Although not mandated by xpm, it is also recommended to upgrade npm to the latest version, and node to a currently supported LTS version.

Local installations

One of the xPack design goals is to allow each project to select the exact versions of the libraries it requires.

Similarly to npm being able to install specific versions of the JavaScript tools into each project, xpm was also designed to be able to install specific versions of the required libraries locally into each project.

Therefore, similarly to the way npm installs the JavaScript packages into node_modules, xpm installs the libraries into xpacks. Here there will be separate folders with the installed packages, for example xpacks/@micro-os-plus/micro-test-plus.

Initialise the project​

The project needs a package.json file in its root folder.

Create it by running xpm init in the project folder (replace my-project with the name of the folder):

cd my-project
xpm init
Under the hood

The primary purpose of xpm init is to create a package.json file, if not already present.

In addition to name and version, the minimal package.json must include a property named xpacks, even if empty. This property is mandatory to identify the package as an xpm package.

Installation via xpm​

The next step is to install the micro-test-plus package into the project.

The command to install the latest available version of micro-test-plus is:

xpm install @micro-os-plus/micro-test-plus@latest --verbose

To install a specific version, specify it explicitly:

xpm install @micro-os-plus/micro-test-plus@6.0.0 --verbose
Installation details

The above xpm install command will perform the following:

  • identify the desired version of @micro-os-plus/micro-test-plus, download it into a cache and unpack it into a versioned folder within the user's global xPacks store (if not already present); check the output of the xpm install command for the actual folder used on your platform;
  • create a local symbolic link such as xpacks/@micro-os-plus/micro-test-plus pointing to the versioned folder in the user's global xPacks store;
  • add @micro-os-plus/micro-test-plus to package.json as a dependency; this associates a specific version of µTest++ Testing Framework with the current project (details below).
tip

The installation location may be configured using the XPACKS_STORE_FOLDER environment variable; for more details, see the xpm folders page.

Reproducibility and dependencies

To ensure reproducibility, it is essential for each project to always use the exact desired versions of the required libraries, regardless of the libraries installed in the system.

To achieve this objective, xpm records all locally installed packages as dependencies in the project package.json file.

The result appears as follows:

  "xpack": {
  "minimumXpmRequired": "0.23.3",
  "dependencies": {
    "@micro-os-plus/micro-test-plus": {
      "specifier": "6.0.0",
      "local": "link",
      "platforms": "all"
    }
  },
  "devDependencies": {},
  "properties": {},
  "actions": {},
  "buildConfigurations": {}
}

If the package.json is saved in the revision system, the above definition acts as a firm reference to the specific version of µTest++ Testing Framework.

After cloning the project into a different location, the command xpm install may be used to install all dependencies.

This is particularly useful for CI/CD environments.

Install folder hierarchy​

Following the xpacks/@micro-os-plus/micro-test-plus link, the installed package is a folder hierarchy that includes only the source files of the library, the include folder with the public headers, and the build system files (CMakeLists.txt, meson.build, and xcdl-package.jsonc) at the top level.

% tree -l
.
├── CHANGELOG.md
├── CMakeLists.txt
├── include
│   └── micro-os-plus
│       ├── micro-test-plus
│       │   ├── check-result.h
│       │   ├── counters.h
│       │   ├── deprecated.h
│       │   ├── detail.h
│       │   ├── exceptions.h
│       │   ├── expression-formatter.h
│       │   ├── extension.h
│       │   ├── function-comparators.h
│       │   ├── inlines
│       │   │   ├── check-result-inlines.h
│       │   │   ├── counters-inlines.h
│       │   │   ├── deprecated-inlines.h
│       │   │   ├── detail-inlines.h
│       │   │   ├── exceptions-inlines.h
│       │   │   ├── expression-formatter-inlines.h
│       │   │   ├── function-comparators-inlines.h
│       │   │   ├── math-inlines.h
│       │   │   ├── reflection-inlines.h
│       │   │   ├── reporter-inlines.h
│       │   │   ├── runner-inlines.h
│       │   │   ├── runner-options-inlines.h
│       │   │   ├── stopwatch-inlines.h
│       │   │   ├── test-inlines.h
│       │   │   └── utility-inlines.h
│       │   ├── math.h
│       │   ├── README.md
│       │   ├── reflection.h
│       │   ├── reporter-human.h
│       │   ├── reporter-tap.h
│       │   ├── reporter.h
│       │   ├── runner-options.h
│       │   ├── runner.h
│       │   ├── stopwatch.h
│       │   ├── test.h
│       │   └── utility.h
│       └── micro-test-plus.h
├── LICENSE
├── LICENSE-Boost
├── meson.build
├── package.json
├── README.md
├── src
│   ├── check-result.cpp
│   ├── counters.cpp
│   ├── deprecated.cpp
│   ├── expression-formatter.cpp
│   ├── reporter-human.cpp
│   ├── reporter-tap.cpp
│   ├── reporter.cpp
│   ├── runner-options.cpp
│   ├── runner.cpp
│   ├── stopwatch.cpp
│   ├── test.cpp
│   └── utility.cpp
└── xcdl-package.jsonc

6 directories, 55 files

The original Git repository contains hundreds of additional files, such as the website files and the multi-platform tests; therefore, the installed package is a much smaller subset of the original project and is recommended.

Integration​

This project is implemented in C++. Tests are also expected to be written in C++, although the code under test may be written in standard C.

The source code has been compiled natively using GCC and LLVM/clang, and cross-compiled for embedded Arm and RISC-V targets, with no warnings.

For execution on embedded platforms, the test framework requires only minimal system support: the ability to write to an output stream. Any environment providing this capability is suitable; for standalone tests, the most common approach is semihosting.

Pre-configured CMake and Meson build files are provided to simplify integration into user projects, as described below. For alternative build systems, refer to the sections that follow.

Include folders​

The following folder should be passed to the compiler during the build process:

  • include

The primary header file to include in user projects is:

#include "micro-os-plus/micro-test-plus.h"
caution

No other header file should be included explicitly by the application.

Source files​

All source files in the src folder should be incorporated into user projects. The files are:

  • check-result.cpp
  • counters.cpp
  • deprecated.cpp
  • expression-formatter.cpp
  • reporter-human.cpp
  • reporter-tap.cpp
  • reporter.cpp
  • runner-options.cpp
  • runner.cpp
  • stopwatch.cpp
  • test.cpp
  • utility.cpp

The library must be built with the same exception setting as the application: the layout of the object returned by expect() and require() depends on whether exceptions are enabled, so the translation units compiled with and without exceptions cannot be mixed.

Compiler options​

The following options must be supplied to the compiler:

  • -std=c++20 or later for C++ source files

Dependencies​

This library has no required dependencies.

The µOS++ Tracing Infrastructure is optional, and is used only to output trace messages in Debug mode. The library includes micro-os-plus/diag/trace.h only if it is available, so to enable the trace messages, install the @micro-os-plus/diag-trace package, add it to the build of the application, and define the trace macros (see Configuration). The CMake and Meson files of this library do not link it.

Static suites​

The test suites of the application, defined as static_suite objects, must not be packed in a static library, however µTest++ itself is built (see the FAQ).

CMake​

To integrate the micro-test-plus library into a CMake application, add the folder where this project is located to the build:

add_subdirectory("xpacks/@micro-os-plus/micro-test-plus")

The result is an interface library, whose sources are compiled in each target that links it, and that may be added as an application dependency with:

target_link_libraries(your-target PRIVATE
  micro-os-plus::micro-test-plus
)

CMake language standard​

The library requires C++20, but its CMake files do not select it. Set the CMAKE_CXX_STANDARD variable in the project, before the add_subdirectory() call shown above:

set (CMAKE_CXX_STANDARD 20)
set (CMAKE_CXX_STANDARD_REQUIRED ON)

Without CMAKE_CXX_STANDARD_REQUIRED, CMake may silently fall back to an older standard if the compiler does not support C++20.

CMake library targets​

The CMakeLists.txt file defines three libraries, which differ in how the µTest++ sources are built, and in the options that the sources get; all of them provide the same headers and the same definitions.

TargetLibrarySources
micro-os-plus::micro-test-plusinterfacecompiled in each target that links it
micro-os-plus::micro-test-plus-objects-libraryobjectcompiled once, and the object files are shared by the targets that link it
micro-os-plus::micro-test-plus-static-librarystaticcompiled once, into an archive (named micro-test-plus) linked into each target

Use the interface library, as shown above, for a single application. When several executables use µTest++, as the tests of this project do, the object or the static library avoids compiling the sources again for each executable. Like the interface library, they are excluded from the default build; they are built when a target links them.

CMake common options​

The interface library is compiled in the target of the application, so its sources automatically get the include folders, the compile definitions, and the compile options of that target. The object and the static libraries are separate targets, which do not inherit them: they must be given exactly the same common options as the application, otherwise the library and the application are built differently. This matters in particular for the folder of the micro-os-plus/micro-test-plus-defines.h header, which enables the library (see Configuration), since the sources of the library are empty without it, and the application then fails to link, with undefined symbols. It matters also for the options that change the generated code, such as -fno-exceptions (see above).

A simple way to give the same options to the application and to the library is to collect them in an interface library of the project, and link it to both. The library must be linked by its real name, micro-os-plus-micro-test-plus-objects-library or micro-os-plus-micro-test-plus-static-library, since CMake does not allow the aliases to be modified:

add_library (app-common-options INTERFACE)
# The folder with micro-os-plus/micro-test-plus-defines.h.
target_include_directories (app-common-options INTERFACE "include")
# The common compile options for the application and the library.
target_compile_options (app-common-options INTERFACE -fno-exceptions ...)

add_subdirectory ("xpacks/@micro-os-plus/micro-test-plus")

target_link_libraries (
  micro-os-plus-micro-test-plus-static-library PRIVATE app-common-options
)

add_executable (your-target main.cpp)
target_link_libraries (
  your-target PRIVATE app-common-options
                      micro-os-plus::micro-test-plus-static-library
)

Meson​

To integrate the micro-test-plus library into a Meson application, add the folder where this project is located to the build:

subdir('xpacks/@micro-os-plus/micro-test-plus')

The result is a dependency object that may be added to an application with:

exe = executable(
  'your-target',
  sources: ['main.cpp'],
  dependencies: [
    micro_os_plus_micro_test_plus_dependency,
  ]
)

Due to differences in the build model, the Meson configurations currently do not support the object and static library targets for µTest++. They rely on the interface library approach, where the sources are compiled directly into each target that links them, so the options of the application, including the folder of the micro-os-plus/micro-test-plus-defines.h header, apply to them automatically.

Configuration​

The build can be configured using the following preprocessor definitions:

  • MICRO_OS_PLUS_MICRO_TEST_PLUS_ENABLED - to include and enable the µTest++ library in the build; it is required, and must be defined for both the library sources and the application: without it, the headers and the sources compile to nothing, and the application fails to compile, since none of the µTest++ definitions are declared
  • MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_ENABLED - to enable tracing messages specific to the µTest++ library
  • MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_CONSTRUCTORS_ENABLED - to enable tracing messages for object constructors and destructors
  • MICRO_OS_PLUS_MICRO_TEST_PLUS_DEPRECATED_ENABLED - to enable the deprecated definitions, which are otherwise not available; define it for the library sources too

The recommended way to define the micro-test-plus specific macros is to use a separate header file, micro-os-plus/micro-test-plus-defines.h, and define the macros there (uncomment the desired lines).

micro-os-plus/micro-test-plus-defines.h
// ----------------------------------------------------------------------------

#ifndef MICRO_OS_PLUS_MICRO_TEST_PLUS_DEFINES_H_
#define MICRO_OS_PLUS_MICRO_TEST_PLUS_DEFINES_H_

// ----------------------------------------------------------------------------

// µTest++, a lightweight testing framework for embedded platforms.
#if !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_ENABLED)
#define MICRO_OS_PLUS_MICRO_TEST_PLUS_ENABLED
#endif // !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_ENABLED)

// Display trace messages in the µTest++ Testing Framework when TRACE is
// enabled.
#if !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_ENABLED)
// #define MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_ENABLED
#endif // !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_ENABLED)

// Display trace messages in constructors and destructors in the µTest++
// testing framework when TRACE is enabled.
#if !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_CONSTRUCTORS_ENABLED)
// #define MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_CONSTRUCTORS_ENABLED
#endif // !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_TRACE_CONSTRUCTORS_ENABLED)

// Enable the deprecated definitions of the µTest++ Testing Framework, which
// are otherwise not available; the compiler reports their use.
#if !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_DEPRECATED_ENABLED)
// #define MICRO_OS_PLUS_MICRO_TEST_PLUS_DEPRECATED_ENABLED
#endif // !defined(MICRO_OS_PLUS_MICRO_TEST_PLUS_DEPRECATED_ENABLED)

// ----------------------------------------------------------------------------

#endif // MICRO_OS_PLUS_MICRO_TEST_PLUS_DEFINES_H_

// ----------------------------------------------------------------------------

Once the xCDL tool is available, for xCDL enabled projects this file will be generated automatically.

The cdlSettings to enable them are:

xcdl-project.jsonc
    "cdlConfigurations": {
      //
      "<name>": {
        ...
        "cdlSettings": {
          ...
          "micro-os-plus.micro-test-plus.trace": true,
          "micro-os-plus.micro-test-plus.trace-constructors": true
        }
      },
      ...
    }

Alternative installation​

Installation via npm​

The package may also be installed with npm or related tools, but the features specific to C/C++ projects will not be available, so it is recommended to use xpm.

Copy files​

The traditional method of using source libraries is to copy them into the user's project.

The obvious advantage is simplicity, but the disadvantage is that the library becomes part of the project, and any improvements or bug fixes must be manually copied into the project.

Add as a Git submodule​

Another common solution is to add the entire project as a Git submodule, for example beneath an xpacks folder:

cd my-project
git init # Unless already a Git project
mkdir -p xpacks

git submodule add https://github.com/micro-os-plus/micro-test-plus-xpack.git \
  xpacks/@micro-os-plus/micro-test-plus
info

Note that linking the project as a submodule brings the entire project, with tests and documentation, not only the source files.

Installing via xpm filters the content, and only the relevant files are brought into the project.

Uninstall​

The xpm packages do not use any form of system installer; instead they are distributed as portable archives; therefore no uninstaller is needed: removing the links, and possibly the user's global xPacks store folder and the user xPack cache folder, is sufficient.

To remove the links created by xpm in the current project, navigate to the project folder:

cd my-project

and request xpm to uninstall the package:

xpm uninstall @micro-os-plus/micro-test-plus --verbose

To completely remove the package from the user's global xPacks store:

xpm uninstall --global @micro-os-plus/micro-test-plus --verbose
Clean-ups

For a complete clean-up, note that xpm uses two folders:

  • %APPDATA%\xPacks
  • %LOCALAPPDATA%\Caches\xPacks

They can be removed at any time and space reclaimed; xpm will recreate them during new installations.

However, the projects that link to the user's global xPacks store will have broken links.

Status​

The micro-test-plus library is production-ready and is used to test multiple projects within the µOS++ framework.

Last updated on