Run your Boost Tests in parallel with CMake

I was looking for a Test Library to run eddic tests in parallel to replace Boost Test Library. I posted my question on StackOverflow and an awesome solution has been posted. With CMake and a little CMake additional file, it is possible to run the tests written with Boost Test Library in parallel without changing anything in the tests code !

CTest is the test runner that is shipped with CMake. This runner can run tests in parallel using the -j X option (X is the numbers of threads). However, it can only run the tests that are declared in the CMakeLists.txt file. In my case, this means only one (the executable with Boost Test Library). If you have T tests, a solution would be create T executable files. Then, they can be run in parallel by ctest. However, this is not very practical. The solution proposed in this article is better.

Integrate Boost Test Library in CMake

Ryan Pavlik provides a series of CMake modules in its Github repository. One of this module is named BoostTestTargets. It automatically generates the CTest commands to run all the tests that you have. The small drawback is that you to list all the tests.

To start, you have to download these files:

• BoostTestTargets.cmake
• GetForceIncludeDefinitions.cmake
• CopyResourcesToBuildTree.cmake
• BoostTestTargetsStatic.h
• BoostTestTargetsDynamic.h
• BoostTestTargetsIncluded.h

These files must be placed next to your CMakeLists.txt file. Then, you have to modify your CMakeLists.txt file to enable testing and enable the new module. For example, if you have two test suites and five tests in each:

INCLUDE(CTest)

ENABLE_TESTING()

file(
    GLOB_RECURSE
    test_files
    test/*
)

include(BoostTestTargets.cmake)

add_boost_test(eddic_boost_test
    SOURCES ${test_files}
    TESTS 
    TestSuiteA/test_1
    TestSuiteA/test_2
    TestSuiteA/test_3
    TestSuiteA/test_4
    TestSuiteA/test_5
    TestSuiteB/test_1
    TestSuiteB/test_2
    TestSuiteB/test_3
    TestSuiteB/test_4
    TestSuiteB/test_5
    )

All the test files are searched in the test directory and used in the SOURCES variable. Then all the tests are declared.

The main test file has to include a specific header file:

#define BOOST_TEST_MODULE eddic_test_suite
#include <BoostTestTargetConfig.h>

This file will be automatically detected by BoostTestTargets and configured correctly. And that's it !

You can run CMake again in your build directory to use the new test system:

[bash]cmake .[/bash]

If the configuration has been successful, you will see a message indicating that. For example, I see that:

-- Test 'eddic_boost_test' uses the CMake-configurable form of the boost test framework - congrats! (Including File: /home/wichtounet/dev/eddi/eddic/test/IntegrationTests.cpp)
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/ramdrive/dev/eddic

Run tests in parallel

You can then run your tests in parallel with ctest. For instance, with 9 threads:

ctest -j 8

In my case, my tests are completed 6x faster ! This is very valuable when you often run your tests.

For more information on how to integrate your Boost Test Library tests with CMake, you can consult the The cmake-modules repository

Related articles