Here's the video walkthrough, if you'd rather watch it:
Why I Started Using This Tool
In my last post I set up unit tests for my C
project with Unity and CTest, and one command gave me a lovely green line: 100% tests passed.
I was proud of it for about a day.
Then I looked at what I'd actually tested. My arithmetic.c has five functions, and I'd written
tests for one: add. The other four, including divide and modulo with
their divide-by-zero handling, had never been called by a single test. CTest didn't care. It counts tests
that pass, not code that gets tested.
I wanted a number that would tell me the truth, plus a list of the exact lines my tests had never touched, coming out of the same one command I already use. That's code coverage, and GCC already ships the tool for it: gcov.
What It Does
gcov answers one question: which lines of my code ran while my tests ran? It works in three steps, each leaving a file behind:
- Compile with
--coverage. GCC adds counters to the program and writes a.gcnofile, a map of the code. - Run the tests. The counters tick up, and on exit the counts are saved to a
.gcdafile. - Run
gcov. It combines the two into a readable.gcovreport.
In CMakeLists.txt that's a handful of lines. First, the flag on the test program only, for
both compiling and linking, because the link step needs gcov's runtime library:
target_compile_options(test_arithmetic PRIVATE --coverage)
target_link_options(test_arithmetic PRIVATE --coverage)
Then the trick I liked most: CTest will run any command as a test, so I registered gcov itself as a test:
add_test(NAME coverage
COMMAND gcov CMakeFiles/test_arithmetic.dir/src/arithmetic.c.gcda
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
Running from the build folder makes the path work and keeps the report out of my source folder.
Last, gcov needs the .gcda file to exist, so the tests must run first. CTest
fixtures guarantee that:
set_tests_properties(test_arithmetic PROPERTIES FIXTURES_SETUP arithmetic_run)
set_tests_properties(coverage PROPERTIES FIXTURES_REQUIRED arithmetic_run)
Now coverage always runs after test_arithmetic, even in parallel. If I run only
the coverage test with ctest -R coverage, CTest pulls in the real tests automatically.
One command runs the lot:
cmake --workflow --preset test-verbose
1/2 Test #1: test_arithmetic .................. Passed 0.09 sec
...
2: File 'C:/git/09_cmake/src/arithmetic.c'
2: Lines executed:20.00% of 10
2: Creating 'arithmetic.c.gcov'
2/2 Test #2: coverage ......................... Passed 0.03 sec
100% tests passed, 0 tests failed out of 2
And there it was, in the middle of the output: Lines executed:20.00% of 10, right above
100% tests passed. The .gcov report showed counts next to add's
lines and ##### next to every line of subtract, multiply,
divide and modulo:
20: 3:int add(int a, int b)
-: 4:{
20: 5: return a + b;
-: 6:}
#####: 7:int subtract(int a, int b)
-: 8:{
#####: 9: return a - b;
-: 10:}
That's a to-do list I didn't have before.
Final Verdict
If you're already using GCC, CMake and CTest, adding gcov is close to free. It's two flags and one extra
test, and you get an honest answer about what your tests really cover. For a beginner, seeing the
##### lines is the clearest lesson there is in why "all tests passed" isn't the end of the
story.
A few honest caveats. The coverage test always "passes", because gcov doesn't fail on low numbers, so this
is a report and not a gate (tools like gcovr can enforce a minimum). The .gcda
counts add up across runs, which confused me at first: my report said Runs:10. Delete
build/ for fresh numbers. The .gcda path is hardcoded, which is fine for a small
project but fragile for a big one. And none of this works with MSVC.
My biggest takeaway: coverage doesn't make your tests better, it shows you where they're missing. Mine said 20%. Next episode, I'm going to make that number climb.