Writing checks

A check is an expectation or a requirement, written as an expression with the function comparators, which report their operands when the check fails. This page describes how to write the checks for the common types of values; the checks for the types of the application are described in Extending the framework.

Namespaces

The definitions are organised within several namespaces under micro_os_plus:

• micro_os_plus::micro_test_plus (commonly aliased as mt)
• micro_os_plus::micro_test_plus::utility
• micro_os_plus::micro_test_plus::extension (see Custom Comparators)
• micro_os_plus::micro_test_plus::reflection (the source_location of the signatures, rarely named by the applications)

micro_os_plus is the primary µOS++ namespace, and micro_test_plus is the namespace for µTest++.

In complex projects, the definitions within the micro_test_plus namespace may conflict with those of the application, so refer to the function comparators with explicit namespaces, and not with a using namespace directive. For brevity, declare a short alias for the namespace, at the appropriate scope, and prefix all the references with it:

namespace mt = micro_os_plus::micro_test_plus;

ts.test ("Check answer", [] (auto& t) {
  t.expect (mt::eq (compute_answer (), 42)) << "answer is 42";
});

Expectations and requirements

Expectations and requirements are checks that evaluate expressions yielding boolean values; they differ in what happens after a failure.

Both are created with member functions of the test case objects, and return an object to which a message can be appended with <<; the check is reported at the end of the statement, so the returned object must not be named (see below). The declarations below show the parameters that an application passes; the return type is internal:

template <class Expr_T>
  requires extension::checkable<Expr_T>
auto expect (const Expr_T& expr,
             const reflection::source_location& sl
             = reflection::source_location::current ());

template <class Expr_T>
  requires extension::checkable<Expr_T>
auto require (const Expr_T& expr,
              const reflection::source_location& sl
              = reflection::source_location::current ());

The source location defaults to that of the call, so a check helper that takes a const mt::reflection::source_location& parameter, with the same default, and passes it on, makes the failures report the line of the caller of the helper.

The expression is either a framework expression (a comparator, an operator, or a logical function), whose operands are reported on failure, or any value that the condition of an if statement accepts, including the types with an explicit operator bool, such as std::optional, std::unique_ptr, and std::function (except the wrappers of a bool, which are rejected, see below):

auto buffer = std::make_unique<std::byte[]> (size);
t.require (buffer) << "buffer allocated";

The arguments that are always true are rejected, since they make the check pass silently: the arrays, including the string literals, the functions, the pointers to functions and to members, and the pointers to the character types. They usually come from writing the description instead of the condition, or a function name without the call; write the condition, with mt::ne (p, nullptr) for a character pointer. The pointers to other objects are accepted, so t.expect (ptr) checks for non-null:

t.expect ("answer () == 42");        // Rejected, was always true.
t.expect (answer);                   // Rejected, a function name.
t.expect (mt::eq (answer (), 42));   // Correct.

The wrappers of a bool are rejected too: std::optional<bool>, and the smart pointers std::unique_ptr<bool> and std::shared_ptr<bool>. The reason is that the condition of such a wrapper does not test the bool that it holds, but the wrapper itself (a value is present, or the pointer is not null), while a reader of t.expect (flag) assumes that the check tests the flag. For std::optional<bool>{ false }, the check would pass, and it would be reported as true, like the arguments that are always true, which are rejected for the same reason: a check that passes silently, for another reason than the one the author had in mind, is worse than one that does not compile. Write flag.has_value () or ptr != nullptr to test the wrapper, or compare the value, mt::eq (*flag, true). The wrappers of the other types are accepted, so t.expect (value) with a std::optional<int> checks that a value is present, as there is no other meaning for it:

std::optional<bool> enabled = read_flag ();
t.expect (enabled);                  // Rejected, passes also for false.
t.expect (enabled.has_value ());     // Correct, a value is present.
t.expect (mt::eq (*enabled, true));  // Correct, the value is true.

A failed expectation is reported and counted, and the test case continues, so that all its checks are evaluated and reported in one run.

The result of t.expect (...) and t.require (...) lives until the end of the statement, where the check is reported, so that the checks are reported in the order of the statements, so it must never be named or stored. A variable of its type, auto r = t.expect (...), does not compile; a reference, auto&& r = t.expect (...), compiles, but dangles after the statement, since the temporary is destroyed there, and the check is reported there, not later. The message operator and value() can be called only on the temporary, so the named result cannot be used, and forcing it with std::move() is undefined. Append the message in the same statement, with <<.

The message accepts the same values as the operands of the function comparators, including the custom types with an operator<< for the formatter, and any number of them can be appended. They are written as in the reports of the comparators, except that the numbers have no type suffixes and the char values are not quoted, since the message is text; the wide characters (u'x', U'x', L'x') are written as in the reports, since they cannot be appended to the text unchanged:

t.expect (mt::eq (parse (input), expected))
    << "input " << input << ", " << input.size () << " bytes";

A part of the message that is valid only after a failure can be given as a callable object, such as a lambda, that returns a value accepted by the message; it is called only if the check fails, and its result is appended, while after a success it appends nothing:

// std::expected requires C++23.
std::expected<config, std::string> result = parse ("config.json");
t.require (result.has_value ())
    << "parse failed: " << [&result] { return result.error (); };

The function pointers are not called; they are written as addresses.

An argument that is neither of these is rejected with a message that states what a message argument is, not with the trace of the overload resolution: a type without an operator<< for the formatter (define one in the namespace of the type, as for the comparators), or a callable object that takes arguments, returns void, or returns a value that cannot be written. The last case would otherwise convert a lambda without captures to a function pointer and then to bool, and write true.

Requirements

A failed requirement is reported and counted as a failed expectation, and then the rest of the test case is skipped, since the following checks would be meaningless; the run continues with the next test case:

ts.test ("Parse configuration", [] (auto& t)
  {
    auto config = parse ("config.json");
    t.require (config.has_value ()) << "configuration parsed";

    // Not reached if the configuration could not be parsed.
    t.expect (config->version == 2) << "version 2";
  });

A failed requirement in a nested test case stops only the nested test case; the outer test case continues, and fails because of it.

The test case is stopped by throwing an exception, at the end of the statement, after the message was appended. It is not derived from std::exception, so a catch (const std::exception&) in the test case body does not intercept it; a catch (...) does, and must rethrow it with throw;. The test case body must not be noexcept, since the exception would call std::terminate().

The human-readable reporter marks the failed requirement with - test case stopped, after the location; the TAP reporter reports it like any other failed check, as node-tap does, and the stop is shown by the plan of the test case, which follows it:

    • Parse configuration
        ✗ configuration parsed FAILED (main.cpp:4) - test case stopped

Without exceptions (for example with -fno-exceptions, common on embedded targets), the test case cannot be stopped, and a failed requirement aborts the whole test execution, as described in Aborting the test execution: the results of the test case are displayed, ending with the failed requirement, but the remaining test cases do not run and no totals are written. The other test frameworks make the same distinction between the fatal and the non-fatal checks (see Comparison with other frameworks).

Refer to the Expectations and Requirements reference pages.

Function comparators

To report the differences between the expected and the actual values in failed conditions, several generic comparator functions are provided.

template <class Lhs_T, class Rhs_T>
auto eq(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T>
auto ne(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T>
auto lt(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T>
auto le(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T>
auto gt(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T>
auto ge(Lhs_T&& lhs, Rhs_T&& rhs);

template <class Lhs_T, class Rhs_T, class Epsilon_T>
auto approx(Lhs_T&& lhs, Rhs_T&& rhs, Epsilon_T&& epsilon);

The rules for the operands (the strings, the arrays, the integers, and the copies) are in How the operands are compared and stored, at the end of this page.

The approx() comparator succeeds if its first two operands differ by at most the third one, the tolerance, so a tolerance of zero requires equal operands and a negative one always fails. The operands are numbers, integral or floating-point, possibly of different types, but not bool or the character types, which are rejected. For the integers, the distance is computed exactly, also for operands of different signedness, without wrap-around; for the other types, in their common floating-point type. A failure is reported as lhs ~ (rhs +/- epsilon):

t.expect (mt::approx (read_adc (), 512, 3)) << "ADC reading";
// When it fails: 508 ~ (512 +/- 3)

Refer to the Function Comparators reference page.

Logical functions

Complex expressions may be evaluated in a single statement using the logical functions all_of(), any_of(), and negate(), the function forms of the &&, ||, and ! operators; any_of() is an inclusive or, true also when all the operands are true.

template <class First_T, class Second_T, class... Rest_T>
auto all_of (First_T&& first, Second_T&& second, Rest_T&&... rest);

template <class First_T, class Second_T, class... Rest_T>
auto any_of (First_T&& first, Second_T&& second, Rest_T&&... rest);

template <class Expr_T>
auto negate (Expr_T&& expr);

The operands must be accepted by expect(): the values that are always true, such as string literals, C strings, arrays, and pointers to functions, are rejected at compile time, also when wrapped in one of these functions.

all_of() and any_of() take two or more operands. A chain of the same operation is reported as a single group, for example (1 == 1 and "abc" == "abd" and 3 < 2).

All the operands are evaluated before the check, so, unlike the standard algorithms of the same names, the functions do not stop at the first operand that decides the result. After a using namespace std, an unqualified call with three or more operands of the same type selects std::all_of() or std::any_of(); call them as mt::all_of() and mt::any_of().

Refer to the Logical Functions reference page.

Checking exceptions

It is also possible to check various exception-related conditions.

// Check for any exception.
template <class Callable_T>
  requires std::invocable<Callable_T&>
auto throws (Callable_T&& func);

// Check for a specific exception.
template <class Exception_T, class Callable_T>
  requires std::invocable<Callable_T&>
auto throws (Callable_T&& func);

// Check for no exception at all.
template <class Callable_T>
  requires std::invocable<Callable_T&>
auto nothrow (Callable_T&& func);

The callable is invoked once, immediately, with no arguments, as an lvalue, so it can be a lambda (including a mutable one), a functor, or a function; a value that cannot be called without arguments is rejected by the constraint.

The first form passes if the callable throws an exception of any type, the second if it throws an exception of the given type or of a type derived from it (so throws<std::exception> passes for any standard exception), and nothrow() if it throws no exception:

t.expect (mt::throws<std::runtime_error> ([] { parse (""); }))
    << "an empty input is rejected";
t.expect (mt::nothrow ([] { parse ("config.json"); })) << "a valid input is accepted";

These functions are available only when exceptions are enabled, that is, when __cpp_exceptions is defined. A program that is also built with -fno-exceptions must guard them:

#if defined(__cpp_exceptions)
t.expect (mt::throws ([] { parse (""); })) << "an empty input is rejected";
#endif // defined(__cpp_exceptions)

Refer to the Checking Exceptions reference page.

Strings

Strings are compared with eq() and ne(), which compare them by content, and show them when the check fails (the standard == and != operators return a plain bool, so a failed check could not show the strings):

t.expect (mt::eq (actual, "fortytwo"))
    << "actual eq 'fortytwo'";

Containers

Containers are compared with eq() and ne(), which use the standard operators, and show their elements when the check fails. The containers must be comparable with the standard operators, which for the standard containers means the same type; a C array is also accepted against a container:

t.expect (mt::eq (std::vector<int>{ 1, 2 }, std::vector<int>{ 1, 2 }))
    << "vector{ 1, 2 } eq vector{ 1, 2 }";

The elements are shown as {1, 2}, for any type that a range-based for statement can iterate, with begin() and end() members, like the standard containers, the spans, and the views, or functions, found by argument-dependent lookup. The arrays passed to the comparators are shown the same way. The exceptions are the strings, which are shown as text, and the types whose elements are of their own type, like std::filesystem::path, which cannot be shown element by element, and are rejected by the comparators; compare p.string () instead, or, for an application type, define an operator<< for it, as for the custom types. The elements must themselves be printable.

Other standard types

The function comparators also accept the scoped enumerations (including std::byte), and the std::pair, std::tuple, and std::optional objects, so also the maps, whose elements are pairs. These are accepted only if their elements are printable (and a duration only if its count is a number), so a pair with a member that has no operator<< is rejected like that member. When a check fails, they are shown as follows:

• a scoped enumeration by its underlying value, written as that type is, for example 1 for int and (unsigned char)1 for std::byte, whose underlying type is unsigned char; to show the names of the enumerators instead, define an operator<< for the enumeration, as for the custom types; the report then shows the text that the function writes, as for any other type of the application, also for the elements of the containers;
• a pair or a tuple as (a, b), and a map as {(1, 2), (3, 4)};
• an optional by its value, or as nullopt when it has no value;
• a std::chrono duration as the count followed by the unit, for example 5ms, 250us, 2s, 3min, or 1h, and, for the other periods, by the ratio, as 3[1/30]s.

enum class colour { red, green };

t.expect (mt::eq (paint (), colour::red)) << "colour is red";
t.expect (mt::ne (find (42), std::nullopt)) << "42 is found";
t.expect (mt::lt (elapsed (), 5ms)) << "completed within 5ms";

The strings are shown quoted, whether they are std::string or std::string_view values, or const char* elements of a container, pair, tuple, or optional (a null pointer is shown as nullptr), so that an empty string or a trailing space is visible. The values that the operator<< of an application type writes are shown as that function writes them, as in the messages, without quotes or type suffixes: its std::string, std::string_view, and const char* members are not quoted by the report, so write the quotes in the operator if an empty string or a trailing space must be visible.

A char array that the operator<< of an application type, or a message, writes (for example a fixed-size name field of a structure) is written up to its first null character, or up to its end if it has none, so an array that is full, without a terminator, is read only within its bounds. A char pointer is a null-terminated string, read up to its terminator; when only the array is available, write the array itself, not a pointer to its first element.

How the operands are compared and stored

These rules apply to all the function comparators described above.

The C strings (the arrays, including the string literals, and the pointers of char, wchar_t, char8_t, char16_t, and char32_t) are compared by content, also with the standard strings and string views of the same character type, so mt::eq (actual, "abc") checks the characters, not the addresses; a null pointer is equal only to another null pointer (or nullptr), not to an empty string. The other arrays (including uint8_t buffers) are compared element by element, also with a container such as a std::vector, and are written as ranges on failure; arrays of different sizes are never equal, and their elements are compared as single values are (see below), so an int array holding -1 is not equal to an unsigned array holding UINT_MAX. The other pointers are compared by address.

The volatile operands, such as a buffer filled by a DMA controller, are accepted: an array of volatile elements is compared element by element, and written as a range, like the other arrays; this includes a volatile char array, which is a buffer of characters, not a string, so it is not compared with a string literal (compare it with a std::array<char, N>). A pointer to volatile characters is compared by address, like the other pointers.

Two integers of different signedness are compared by value, not by converting the signed operand to the unsigned type, so mt::lt (-1, 1u) succeeds and mt::eq (-1, UINT_MAX) fails. The character types are compared as the numbers that they hold, with the same rule, as the integers of the same size and signedness (the signedness of char and wchar_t is that of the platform), so mt::gt (U'a', -1) succeeds, and a char with the value -1 is less than 1u where char is signed. Two characters of different types are compared in the same way, so mt::eq ('a', U'a') succeeds. This does not apply to bool, which is compared as the language does.

The wide and Unicode strings are reported as text, converted to UTF-8 and quoted as the std::string values are; the messages write the text of the pointers and arrays of these types in the same way. A C string of one character type is not comparable with a string of another type, for example L"abc" with a std::string; the comparison is rejected at compile time.

Operands that cannot be compared with the operator of the comparator, such as a structure and an integer, or that cannot be written in the report, such as an application type without an operator<< for the formatter, are rejected with a message that names the comparator and the operator, or says that the operand cannot be written, not with the trace of the concepts.

The comparators copy the named operands and move the temporaries into the expression, so that an expression built in a helper function from its parameters and returned, or kept in a variable, stays valid after the operands are destroyed. The copy is made for every check, passing or failing, so a large container or string costs a copy and an allocation, which matters on a small target. When the cost matters, compare a view, which copies only a pointer and a size: mt::eq (std::string_view{ text }, "abc") makes no allocation. The operands whose class cannot be copied (an application type with a deleted copy constructor, which needs an operator<< like any other operand) are referred to, not copied, and so are the arrays, including the C strings held in arrays: the expression refers to the array, which must outlive it. This matters only for an expression that is returned from a helper or kept in a variable, after the array is destroyed, since the report is written from the array; the check in the same statement is safe. For such an expression, compare a copy of the array, of the same type as the other operand: mt::eq (std::to_array (buffer), expected_array) for a numeric array, or mt::eq (std::string{ name }, "abc") for a character array.

A smart pointer cannot be written in a report, so it is not a valid operand: compare the pointer that it holds, mt::ne (p.get (), nullptr), or pass it as a condition, t.expect (p), as shown above.

Comparison with other frameworks

The plain assert() macro, mentioned in the Rationale, terminates the whole program at the first failure. Test frameworks distinguish the non-fatal checks, which only report the failure, from the fatal ones, which also stop the test; most of them stop only the current test case, which is also the behaviour of require():

Framework	Non-fatal	Fatal	After a fatal failure
µTest++	expect()	require()	the test case is stopped
GoogleTest	EXPECT_*()	ASSERT_*()	the current function returns
Catch2	CHECK()	REQUIRE()	the test case is stopped
doctest	CHECK()	REQUIRE()	the test case is stopped
Boost.Test	BOOST_CHECK()	BOOST_REQUIRE()	the test case is stopped
Boost UT	expect()	expect (...) << fatal	the program exits
Unity (C)	—	TEST_ASSERT_*()	the test is stopped
JUnit	—	assert*()	the test is stopped
C library	—	assert()	the program is aborted

Some differences are worth noting:

• in GoogleTest, ASSERT_*() returns only from the function in which it is used; if this is a helper function, the test continues after the call;
• Catch2, doctest, and Boost.Test also stop the test case with an exception; without exceptions, Catch2 calls std::terminate(), and doctest rejects REQUIRE() at compile time or, with a configuration macro, makes it behave as CHECK();
• Boost UT reports the fatal failure and calls std::exit(), so the remaining tests do not run, as require(), then named assume(), did before version 6.0;
• Unity, a C framework, stops the test with longjmp(), which does not run the destructors of the C++ objects;
• in JUnit, an unmet assume*() aborts the test, which is reported as skipped, not as failed.

A broader, design-level comparison is in the Contributor's Guide.