Test runners

The runner initialises the framework and returns the exit code. More...

Classes Index

class	counters
	Aggregated check and test case counters for a node in the test tree. More...
class	runner
	The test runner for the µTest++ framework. More...
class	runner_options
	The configuration of a test session, and its command-line parser. More...

Enumerations Index

enum class	colour_mode { ... }
	The use of colours in the console output. More...
enum class	reporter_kind { ... }
	The built-in reporters. More...
enum class	verbosity { ... }
	The verbosity levels for test reporting. More...

Description

The runner initialises the framework and returns the exit code.

A test program usually has one runner, a global object, which main() uses in three steps:

1. runner::initialise() reads the command-line options, from argc and argv (see Command-line options); on platforms without a command line, the overload that takes the options, runner_options, sets them directly. The call is optional; without it, the default options are used;
2. runner::add_top_suite() returns the top suite, on which main() creates its own test cases (see Test suites). The call is optional, for programs whose test cases are all in other suites;
3. runner::run_suites() runs the other suites, writes the totals, and returns the exit code, which main() returns to the system.

Several runners may coexist, global or local, each with its own suites: a runner runs only the suites that refer to it, so the runners do not interfere (see runner).

The exit code is:

• 0 when all the checks passed, when the test cases were only listed, or when the usage was displayed by --help;
• 1 when a check failed, or when the program used the runner wrongly (for example, it added no suite at all);
• 2 when the command line had an error, when the output file could not be opened, or when the filters selected no test case.

Aborting the session

runner::abort() ends the session immediately: the report is completed (with a Bail out! line with the TAP reporter, in place of the totals), then the handler is called, if there is one, and, if there is none, or if it returns, the process is terminated with ::abort(). A failed require() calls it in the builds without exceptions (see Requirements). runner::set_abort_handler() sets the handler, which can, for example, reset an embedded target or signal the result to a debugger; it returns the previous handler, and nullptr restores the default behaviour. With exceptions, the handler may throw, to return control to the application, but the runner cannot be used afterwards.

Reading the session

These functions of the runner are useful to the code of main():

• runner::totals() returns the totals of the session, complete after run_suites() returned;
• runner::verbosity_level() returns the verbosity of the console, which the tests may use to decide whether to display their own messages;
• runner::is_listing() tells whether the test cases are only listed, when their bodies do not run;
• runner::application_arguments() returns the command-line arguments left to the application.

Example

 #include <micro-os-plus/micro-test-plus.h>

 

 namespace mt = micro_os_plus::micro_test_plus;

 

 mt::runner tr;

 

 int

 main (int argc, char* argv[])

 {

  tr.initialise (argc, argv);

 

  auto& ts = tr.add_top_suite ("Minimal");

 

  ts.test ("Check truth", [] (auto& t) {

  t.expect (true);

  });

 

  return tr.run_suites ();

 }

The program writes (the passed checks are displayed only with the --verbose option):

 # Running: minimal

 # Built with ...

 TAP version 14

 # Subtest: Minimal

  ok 1 - Check truth # { passed, 1 check }

 

  1..1

 ok 1 - Minimal # { passed, 1 check in 1 test case, time: X.XXX ms }

 

 1..1

 # { total: 1 check passed, 0 failed, in 1 test case, 1 test suite, time: X.XXX ms }

The name of the top suite is optional. Without it, the suite is named after the program, the file name of argv[0] without its extension (minimal above); without argv[0], it is named default suite.

Without a command line

On the platforms that have no command line, such as many embedded targets, main() defines the options with runner_options, starting from the defaults, and passes them to initialise():

 #include <micro-os-plus/micro-test-plus.h>

 

 namespace mt = micro_os_plus::micro_test_plus;

 

 mt::runner tr;

 

 int

 main (void)

 {

  mt::runner_options options{};

  options.selected_reporter = mt::reporter_kind::human;

  options.verbosity_level = mt::verbosity::verbose;

 

  tr.initialise (options);

 

  auto& ts = tr.add_top_suite ("Embedded");

 

  ts.test ("Check truth", [] (auto& t) {

  t.expect (mt::eq (6 * 7, 42));

  });

 

  return tr.run_suites ();

 }

The members of runner_options are the options of the command line (see Command-line options): the reporter, the verbosity, the colours, the output file, the filter patterns, the listing of the test cases, and the display of the usage.

Enumerations

colour_mode

enum class micro_os_plus::micro_test_plus::colour_mode

strong

The use of colours in the console output.

Enumeration values

automatic	Colours are used when stdout is a terminal; on the platforms without isatty(), such as the embedded ones, they are not used.
always	Colours are always used, for example in the CI systems that render the ANSI escape sequences but do not provide a terminal, or on embedded targets whose output is displayed by a terminal.
never	Colours are never used.

Selected with the --colour command-line option (or its American spelling --color, or its alternative --use-colour, also spelled --use-color), or with runner_options::colour; the default is automatic. The names follow the values of --color in the GNU tools.

Only the human-readable reporter uses colours; the TAP reporter never does, since the ANSI escape sequences are not part of the TAP syntax. The output file never receives colours, whatever the mode.

Since

6.0.0

Definition at line 152 of file runner-options.h.

153 {

154 automatic,

157 always,

161 never

162 };

reporter_kind

enum class micro_os_plus::micro_test_plus::reporter_kind

strong

The built-in reporters.

Enumeration values

tap	The TAP reporter, compliant with TAP version 14.
human	The human-readable reporter, with optional colours.

Selected with the --reporter command-line option, or with runner_options::selected_reporter; the default is tap.

Since

6.0.0

Definition at line 127 of file runner-options.h.

128 {

129 tap,

130 human

131 };

verbosity

enum class micro_os_plus::micro_test_plus::verbosity

strong

The verbosity levels for test reporting.

Enumeration values

silent	No console output; only the exit code is returned. (= 0)
quiet	Displays only the failed checks and the overall session summary; the TAP reporter also displays one result line per test suite, counted by the plan, and the failed checks as comments. (= 1)
normal	Displays test suite and test case results; the individual checks are shown only for failures, the passed checks being counted, and, in the TAP blocks, numbered by bare ok lines. (= 2)
verbose	Displays all results, including passed checks, for maximum detail. (= 3)

The level is selected with the --silent, --quiet, and --verbose command-line options, or with runner_options::verbosity_level; the default is normal. The console output is intended for the people who watch the tests run, while the optional output file is intended for the integration with other tools. The level affects only the console output; the output file always receives the full report.

The quiet level displays the failed checks, with the names of the suite and of the test cases that contain them, and the totals; the passed checks and the passed test cases are not displayed. To also see them, use the verbose level, or keep quiet and use an output file, which receives them.

Definition at line 100 of file runner-options.h.

101 {

102 silent = 0,

103 quiet = 1,

107 normal = 2,

111 verbose = 3

113 };

---

Generated via doxygen2docusaurus 2.2.8 by Doxygen 1.18.0.