Unit testing with cpptest

:: testing, libraries, cpptest, unit tests

Unit testing is a fundamental activity in software development, even if not as widespread as it should. As C++ has been around for quite some time, several libraries are available for carrying out this activity. This week I’ve had a look at cpptest. In the future, I plan to compare compare it to another pretty popular test library, googletest.

The case in point

I was playing with the Dijkstra algorithm this week, because I needed to write an algorithm to solve the following problem:

Find whether there exists a sequence of words between two given words, such that the Hamming distance between each word and the following is 1

For example, I can go from milk to typo in 6 steps through the sequence:

milk -> mile -> male -> tale -> tape -> type -> typo

To do this, we can organize a dictionary into an undirected graph whose nodes are connected if and only if they have Hamming distance of 1.

Unit tests

In cpptest, we organize tests in test suites. A suite is declared as a class deriving publicly from Test::Suite. For example, I have a header file containing declarations of functions that are not related strictly to graphs. The header’s name is Util.hpp, and the following is a snippet extracted from it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
int getHammingDistance(const string& s1, const string& s2);

class UtilTestSuite : public Test::Suite
{
public:
  UtilTestSuite()
  {
    TEST_ADD(UtilTestSuite::test_get_hamming_distance);
  }
private:
  void test_get_hamming_distance();
};

As you can see, there is the function, and then the test suite. Its constructor only TEST_ADDs methods for testing, in this case to test the function getHummingDistance(); the method itself is then declared in the private section of the class.

The implementation is pretty straightforward, and introduces another little concept: Assertions:

1
2
3
4
5
6
7
8
void UtilTestSuite::test_get_humming_distance()
{
  TEST_ASSERT(getHammingDistance("wind", "wand") == 1);
  TEST_ASSERT(getHammingDistance("wind", "wane") == 2);
  TEST_ASSERT(getHammingDistance("wind", "list") == 3);
  TEST_ASSERT(getHammingDistance("wind", "crow") == 4);
  TEST_ASSERT(getHammingDistance("wind", "winds") == Infinity);
}

Assertions work kind of like the function assert in the library header <cassert>: they check whether the condition is satisfied. The difference is that, in the case of TEST_ASSERT, the failed assertion does not stop the execution of the program, so to allow the collection of data about the execution of all the tests, instead of stopping at each error, giving the developer the chance to decide where to start concentrating his or her debugging efforts.

I have found that declaring the test suite in the header of the code we want to test is neat and useful for documentation.

Running the tests

In order to run the test, you instantiate a test suite and run it, possibly on an output handler. In my case, I have used a text output, that delivers the results of the tests to the console:

1
2
3
Test::TextOutput output(Test::TextOutput::Verbose);
UtilTestSuite uts;
uts.run(output);

When I run the program, I get the following results:

UtilTestSuite: 1/1, 100% correct in 0.000078 seconds<br />
Total: 1 tests, 100% correct in 0.000078 seconds

Now imagine the sheer power you have at your fingertips: for any change you make at the function getHammingDistance, you just re-run the tests and you become confident about the quality of your changes. Of course, be sure to test your tests sometimes!

Fixtures

Sometimes we might want to have some data setup before running the tests, maybe because we want to use the same data for all of them, and initializing them might be expensive. In my case, I wanted to share a graph between two methods of the class Graph. The methods setup() and tear_down() are called by the CppTest library before and after all the tests are executed. Imagine the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
template <typename T>
class Graph {
  unordered_map<T, vector<T>> adjlist;
  vector<T> nodes;

public:
  Graph(const string& filename);
  void AddNode(const T& n);
  void Dijkstra(const T& start);

  unordered_map<T, int> dist;
  unordered_map<T, T> parent;
  friend class GraphTestSuite;
};

Of course you might want to rely on a well-reputed library and use the Boost Graph Library, that we have treated in another post. Here, I have done things manually, as part of my keeping fit for my job.

This graph class has a method for adding a node, and a method for computing the Dijkstra algorithm on it. We want to have access to the private data of the class in our test suite, so we declare it as a friend (we need to make a forward declaration of the suite itself before doing this).

Now we declare the Graph test suite: c++ class GraphTestSuite : public Test::Suite { public: GraphTestSuite() { TEST_ADD(GraphTestSuite::test_AddNode); TEST_ADD(GraphTestSuite::test_Dijkstra); } protected: virtual void setup(); virtual void tear_down(); private: void test_AddNode(); void test_Dijkstra(); Graph<string>* g; }; void GraphTestSuite::setup() { g = new Graph<string>("t/test2.txt"); } void GraphTestSuite::tear_down() { delete g; }

Everything in the test suite class is as before, only we have a protected section declaring the fixture methods, and we have a private pointer to a Graph of strings, that is needed for the setup. In the setup method, we just create a Graph with new (this is generally bad, but I’ll keep it simple here — think RAII for production code).

Now, I want to use that graph pointer to test both AddNode and Dijkstra:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
void GraphTestSuite::test_AddNode()
{
  string s1("file");  // "mile" is in the dictionary
  g->AddNode(s1);
  TEST_ASSERT(std::find(g->adjlist["mile"].begin(),
	                    g->adjlist["mile"].end(), s1) !=
	          g->adjlist["mile"].end());
  string s2("five");
  g->AddNode(s2);
  TEST_ASSERT(std::find(g->adjlist["mile"].begin(),
	                    g->adjlist["mile"].end(), s2) ==
	          g->adjlist["mile"].end());
}

void GraphTestSuite::test_Dijkstra()
{
  g->Dijkstra("rice");  // rice is in the file
  TEST_ASSERT(g->dist["typo"] == 7);
  g->Dijkstra("typo");
  TEST_ASSERT(g->dist["rice"] == 7);
}

As you see, both tests refer to a g variable, which is of course the member field of the GraphTestSuite class. The output, in my case, is:

UtilTestSuite: 1/1, 100% correct in 0.000018 seconds
Total: 1 tests, 100% correct in 0.000018 seconds
GraphTestSuite: 2/2, 50% correct in 0.002941 seconds
> Test: test_Dijkstra
> Suite: GraphTestSuite
> File: ./graph.hpp
> Line: 113
> Message: g->dist["rice"] == 7

Apparently, my implementation of Dijkstra is not symmetrical: it is possible to get to rice from typo, but apparently not the other way around. Sigh. But what’s important here, is that we know which test has gone wrong, even thoug this doesn’t prevent me from going bug-hunting in the next few hours.

Summary

I have explored the basic features of a nice library called CppTest. It is extremely pleasant to use, and provides a service that is of paramount importance in a software project, namely, unit testing. The library-related code is not invasive at all, and plays nicely with the application code. Tests can be organized and run together, and the results can be presented in different output formats (including HTML).

As you have seen, I have still some problems with my implementation of Dijkstra. The code is a micro project on Github; if you spot the error and want to help me, I’ll be glad to receive PRs.