Unit testing with Google Test

In a previous post, I have written a brief introduction to CppTest, and given some motivation as to why you should be using unit tests if you are not already. This week, I wanted to throw a glance at the competition, so I tried Google Test. I still don’t like macros, but these look actually useful.

Tests by example

As in the previous installment, I have considered the problem of finding a sequence of words that whould connect two words given in input, with the rule that to switch from one to the other we can change only one letter. The code is still buggy (sigh), and still lives on GitHub.

The class defining the graph hasn’t changed, but this time I have had to mention explicitly the names of the methods that are supposed to access the private section of Graph

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
template <typename T>
class Graph {

// Private section here

public:
friend class GraphTestSuite;
FRIEND_TEST(GraphTestSuite, TestAddNode);
FRIEND_TEST(GraphTestSuite, TestDijkstra);

Graph(const string& filename);
void AddNode(const T& n);
void Dijkstra(const T& start);

// other public stuff
};

Google Test has a concept of test suite, which is a class that groups together logically related tests. You don’t actually need to have a class with the name GraphTestSuite, unless you want to start writing code that is common to the tests in the suite, as it happened in CppTest.

In this case, we have declared that the tests TestAddNode and TestDijkstra will need access to the private members of Graph; we can write other tests in the same suite that don’t need such access. One difference with respect to CppTest is that we don’t have to register the tests: the magic macro RUN_ALL_TESTS will automatically find the tests to run, and run them.

In our example, we want to create a graph to be used in both the tests declared, so we actually declare a class like this:

1
2
3
4
5
6
class GraphTestSuite : public ::testing::Test {
protected:
virtual void SetUp();
virtual void TearDown();
Graph<string>* g;
};

Compare SetUp and TearDown with CppTest’s setup and tear_down. As I said, no need to have a constructor only to add tests to the suite.

The test for the method AddNode is almost the same as the one we previously wrote for it using CppTest, with the difference of course of the name of the macros. Another noticeable difference comes from an interesting presentation on testing at Google (which apparently I’m not able to find again!), which convinced me of testing one thing per test. Compare with the multiple tests in one line, properly ANDed, of the post on CppTest. Here’s the result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
TEST_F(GraphTestSuite, TestAddNode) {
EXPECT_EQ(g->adjlist["typo"].size(), 1);
EXPECT_STREQ(g->adjlist["typo"][0].c_str(), "type");

// other tests

string s1("file");
g->AddNode(s1);
EXPECT_EQ(g->adjlist["mile"].size(), 5);
EXPECT_TRUE(std::find(g->adjlist["mile"].begin(),
g->adjlist["mile"].end(), s1) !=
g->adjlist["mile"].end());
// other tests
}

EXPECT vs. ASSERT

In Google Test, we have to types of macros: EXPECT and ASSERT. After an EXPECT condition fails, the execution continues; after a failed ASSERT, it stops, because it would not make sense to continue. In this test, none of the single failures might be a serious error, and we have the advantage of being able to see more errors at a time, speeding up our development cycle.

EXPECT_TRUE tests the general case when a condition has to be true for the test to pass. EXPECT_EQ is a shortcut for testing for equality, and is suggested over using EXPECT_TRUE whenever possible, because it will print more useful debugging information in case of failure (as you can read for example in the comments here).

Running the tests

In the main function, we need to perform two operations:

  1. Initialize the library
  2. call RUN_ALL_TESTS

like this:

1
2
3
4
int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
RUN_ALL_TESTS();
}

Given this use of EXPECT, the previous tests (in their complete form, without the parts I have commented out to keep the code snippets clear) give the following output:

[==========] Running 3 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from UtilTestSuite
[ RUN      ] UtilTestSuite.TestGetHammingDistance
[       OK ] UtilTestSuite.TestGetHammingDistance (0 ms)
[----------] 1 test from UtilTestSuite (0 ms total)

[----------] 2 tests from GraphTestSuite
[ RUN      ] GraphTestSuite.TestAddNode
[       OK ] GraphTestSuite.TestAddNode (1 ms)
[ RUN      ] GraphTestSuite.TestDijkstra
[  FAILED  ] GraphTestSuite.TestDijkstra (2 ms)
[----------] 2 tests from GraphTestSuite (3 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 2 test cases ran. (4 ms total)
[  PASSED  ] 2 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] GraphTestSuite.TestDijkstra

1 FAILED TEST

Er, no, it still doesn’t work. But I’m close!

CAVEAT EMPTOR

To use Google Test, link your code with both gtest and pthread.

Intermezzo: installing on Ubuntu

I have installed the library on my Ubuntu box (13.10), which is kind of different from the usual apt-get procedure. You have to apt-get the package libgtest-dev; then, go to /usr/src/gtest and compile it by issuing the following commands (see also this AskUbuntu answer):

1
2
3
sudo cmake CMakeList.txt
sudo make
sudo cp *.a /usr/lib

The last command copies the resulting static libraries you have just compiled into the proper system location. Wonder why they do things this way these days The answer linked before seems to give a hint at why things are done this way.

Death tests

A death test is a test that checks whether a program terminates. This type of tests do not include termination by exception.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
void ExitingFunction() {
std::cerr << "[MR3432] Too bad, we are exiting";
exit(0);
}

TEST(FirstDeathTest, KillingAround) {
EXPECT_EXIT(ExitingFunction(), ::testing::ExitedWithCode(0),
"[MR*] *");
}

int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
RUN_ALL_TESTS();
std::cout << "I am still here :)\n";
}

The test above uses EXPECT_EXIT, which is one type of death test. The macro takes as arguments:

  1. a statement, in this case a call to the exiting function
  2. a predicate, in this case a canned Google Test predicate that expects the function to just exit with code 0
  3. a regular expression that should match what is printed by the function

The result is the following:

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from FirstDeathTest
[ RUN      ] FirstDeathTest.KillingAround
[       OK ] FirstDeathTest.KillingAround (5 ms)
[----------] 1 test from FirstDeathTest (5 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (5 ms total)
[  PASSED  ] 1 test.
I am still here :)

Summary

In this post, I have reported on my early experiences with Google Test. I will keep on testing it, and I’ll throw in also Google Mock, which is useful when we need to provide Mock classes in order to be able to test larger systems.

As a result, I have also subscribed to the Google Testing blog, hoping to reach the illumination on testing that might help me become a better engineer.