Exercise: CMake

Outline

Introduction

Recall from class that CMake is a tool that enables you to configure and construct a build process for a project. This exercise will provide you first hand experience in using basic features of CMake that you may find useful (required) in order to complete your term project. You will be constructing and modifying the CMake configuration files of a small project in order to produce the final desired results. You should feel free to look at outside sources such as the CMake Documentation for help, as long as you do not copy or share your work with another student. To access help for any command from the command line, you may use

cmake --help-command <command> | less

for example, you may print the documentation for the find_library command via

cmake --help-command find_library | less

Notes and Preparation:

export PATH=/usr/shared/CMPT/faculty/wsumner/base/bin/:$PATH

Step 1: Compiling a Program

To start, create a new directory called cmakeExercise/. This will be the root directory of your project for the exercise. Spelling and capitalization matter, as exercises will be graded automatically.

A project built using CMake is configured and controlled through lists of CMake commands in CMakeLists.txt files. To simply compile a program using CMake, you need to define a CMakeLists.txt file containing basic information about the project in the root directory of the project. Our initial project contains the three files below:

main.cpp 
#include <iostream>
#include <vector>

#include "SortIntegers.h"

int
main() {
  std::vector integers = {1, 13, 89, 2, 55, 21, 8, 5, 3, 1, 34};
  sortIntegers(integers);
  for (auto integer : integers) {
    std::cout << integer << "\n";
  }
  return 0;
}
SortIntegers.h 
#pragma once

void sortIntegers(std::vector<int> &numbers);
SortIntegers.cpp 
#include <algorithm>
#include <vector>

void
sortIntegers(std::vector<int> &numbers) {
  std::sort(std::begin(numbers), std::end(numbers));
}

The most basic CMakeLists.txt for a project usually includes three commands: cmake_minimum_required, project, and add_executable. Recall that add_executable creates a target to build1. After that, you need to tell cmake which sources are associated with the target in order to do anything meaningful. Note that most exercise work we do in this class uses C++17. To set the version of the language used to compile a target, you can use set_target_properties for a target and set the CXX_STANDARD to 17. In your term project and later exercises, you'll have the opportunity to use C++20 if you choose to.

Use the documentation for these commands to create a CMakeLists.txt script that defines a project called sorter and compiles the above two files into a single program called sorter. The minimum cmake version should be 3.19. Recall that you will want to use an out-of-source build process to make sure that your script works. You will lose points for any build artifacts found in your source directories. You can also choose to use clang and clang++ as the compilers for your project by using the CMAKE_C_COMPILER and CMAKE_CCC_COMPILER variables:

cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ /path/to/project/

If you were to run the program under a debugger, you might notice that it does not have any debugging symbols. This certainly makes debugging a challenge! CMake allows you to build a program in different modes or types. Building in Debug mode enables debugging, while building in Release mode does not enable debugging symbols and might, for instance, do extra optimization. The default mode is often essentially Release mode, but it can vary with the project. In order to select Debug mode, you must set the CMAKE_BUILD_TYPE variable when running CMake. Once again, you do this on the command line:

cmake -DCMAKE_BUILD_TYPE=Debug /path/to/project/

When compiling with gcc or clang, this will automatically add the -g option to the standard CFLAGS variable used when invoking the compiler. In practice, clang often provides clearer error messages, so compiling with clang during development and using gcc for release builds is not uncommon.

Step 2: Organizing the Project

As much as possible, we would like to organize our source directory structure in order to reflect the different components of our project. We also want to decompose the project into potentially reusable components as much as possible. Finally, we want the directory structure of the separate build directory to also reflect the structure of the different components we are building.

In order to break our source program structure into different components, we will perform two tasks simultaneously. From the perspective of design, we will break the program up into (1) a library that encapsulates related features and (2) an executable program that uses the functionality of the library to achieve a specific goal. From the perspective of file organization, we will separate the files for individual libraries and executables into their own subdirectories.

Inside the project root directory, create the following directories:

Following the above descriptions (1) move main.cpp into tools/fibSorter/, (2) move SortIntegers.cpp into lib/sortIntegers/, and (3) move SortIntegers.h into lib/sortIntegers/include/. Now you must change the CMakeLists.txt in the project's root directory in order to use the new directory structure and design.

CMake supports recursively processing subdirectories in order to decompose project structure. Commands need to be added to build the sortIntegers library and the fibSorter application. Use the add_subdirectory command to include targets in the lib/ and tools/ subdirectories. Inside these subdirectories, you will need additional CMakeLists.txt files that tell CMake to descend further into the lib/sortIntegers/ and tools/fibSorter/ directories.

To build the library, create a CMakeLists.txt file in lib/sortIntegers/ that builds a library called sortIntegers using the add_library command. We also need to declare the location of public interface headers that will be included by clients of the library. This can be done using the target_include_directories command to specify PUBLIC headers in the include directory with $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>. This last complex generator expression sets the include path to use when building the project. To determine the path at install time, we can also add $<INSTALL_INTERFACE:include>. The first portion of the generator expression is a condition determining when it applies, and the latter portion is the path.

Similarly, create a CMakeLists.txt file in tools/fibSorter/ that builds a program called fibSorter using the add_executable command along with the target_link_libraries command. Notice that the location of the include directories for the library are automatically propagated to the fibSorter that uses it. This means you could change where the include directories for the library are, and the fibSorter build process would not need to be modified. If you happen to test that out, make sure to change the file paths back, as they will be checked.

Now try building your project to make sure that it compiles. Look at the directory structure in the build directory. Where is the fibSorter program? Try running it to make sure that it works. Where is the sortIntegers library?

The build directory would be more organized if all programs were built in a single bin/ subdirectory and all libraries were built in a 'lib/' subdirectory. This is again done by setting specific CMake variables in the top level CMakeLists.txt:

set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}/bin")
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}/lib")

You should be aware that the order of operations matters. Ask yourself, "Where in the CMakeLists.txt file do these lines belong?"

Note that the sortIntegers library we built is called an archive or a static library. If we instead wanted to build a shared library, we would instead set the CMAKE_ARCHIVE_OUTPUT_DIRECTORY variable.

From within your build directory, run make clean to remove your previously compiled files and then run make in order to check that the system correctly compiles with your modifications2.

Step 3: Using Other Libraries

Now that you know how to build and organize your own projects in isolation, you can start to incorporate other external projects and libraries into your own project. For instance, if you want to use an external library for managing networking or cryptographic protocols, you can include CMake commands to find such libraries and link against them automatically.

External libraries and development APIs are included in a project using the find_library and find_package commands. In many cases, such libraries and development packages can be found and used by your CMake project even if it may reside in different locations for different build environments or even different operating systems. Many libraries are even specially detected by CMake in order to make development easier. CMake can also automatically build a completely separate external project and allow you to use it as a part of your build process.

For this next step, you will create a new tool/program that uses the ncurses library for presenting nice text based user interfaces. You will use CMake to recognize (1) whether or not the build environment contains the necessary dependencies and (2) determine which libraries need to be linked against in order to build a program that uses ncurses functionality.

Start by creating a new subdirectory called tools/cursesExample/. Inside this directory create the following source file:

main.cpp 
#include <ncurses.h>

int
main() {   
  initscr();
  if(has_colors() == FALSE) {
    endwin();
    printf("Your terminal does not support color\n");
    return 1;
  }
  start_color();
  init_pair(1, COLOR_RED, COLOR_BLACK);

  attron(COLOR_PAIR(1));
  attron(A_BOLD);
  printw("Hello World !!!");
  attroff(A_BOLD);
  attroff(COLOR_PAIR(1));

  refresh();
  getch();
  endwin();

  return 0;
}

Now add the necessary changes to the various CMakeLists.txt files in order to tell CMake to build a program called cursesExample using this source file.

Try to build to project. You should now see that it fails because the project uses ncurses, but we haven't told it that we need to link against the curses library. In order to do so, in the CMakeLists.txt that configures the build of cursesExample, we need to use the find_package command to find the required ncurses libraries. You should use find_package to search for the Curses package as a required dependency of the project. This uses CMake's built in support for recognizing the curses-like libraries. Notice from the given documentation that using ncurses requires the command

set(CURSES_USE_NCURSES TRUE)

before calling find_package. Also notice that find_package results in variables being set that need to be used in order to make the project build correctly. CURSES_LIBRARIES is given the value of the necessary library names for linking a project that uses ncurses. You can use this variable as ${CURSES_LIBRARIES} to pass its values as arguments to another command. Use the commands you have already seen in order to make the project compile using this variables. Notice that the program uses header files, but you don't have to say anything about where they are! You should understand at this point why that is.

Notice, if we wanted to, we could make the project compile by explicitly passing different project commands the names of the libraries we want to link against and the paths to the headers. However, if we did so, changing those values in the future would be more cumbersome. In addition, using find_package as we do in this case means that if curses is not present in the build environment, we will get a clearly explained error as soon as CMake runs, instead of a potentially confusing compilation error much later. I may check for this when grading.

Once you can build and run cursesExample, move on to the next step.

Step 4: Installing

Building the project is important, but you also want to be able to install it once the build process is complete. Installing all of the desired files once they are built can be done using the install command. install copies the selected TARGETS and FILES into paths rooted at the directory pointed to by the CMAKE_INSTALL_PREFIX variable. For instance, if you wanted to install the project to ~/testing/, you would set the install prefix when invoking CMake using a command like

cmake -DCMAKE_INSTALL_PREFIX=~/testing/ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ /path/to/project/

Notice that this is different than the invocation we used before. The argument -DCMAKE_INSTALL_PREFIX=~/testing/ determines where the project will be installed. Afterward, you can install the project by running make and then make install.

There may be many different things that we want to install. For instance, we presently build a library and a program, both of which can be installed. In order to use the library, however, the header file for sortIntegers must also be installed. If we wanted to install documentation for the user, that would also be done using the install command.

install allows you to specify a subdirectory inside the install prefix where each installed file will be placed. For a Linux system there are general patterns/rules to follow:

Add rules to your CMakeLists.txt files to install the header, the library, and the programs to the appropriate directories. You may find the PUBLIC_HEADER property for set_target_properties useful for doing this in the cleanest way, but it is not required to complete the task.

If you are interested in creating a more professional installer, e.g. a Debian package or a graphical installer for Windows or OS X, you can do so using CPack and its CMake commands.

Step 5: Sanity Checking and Submission

You should now have some experience with the fundamentals of CMake as discussed in class. As a part of the project, you will also start to use the add_custom_command and add_custom_target commands along with find_package in order to define a build process for the project documentation using Sphinx as well as to coordinate unit testing.

If you have written things successfully, you should be able to run the following from the project directory without errors.

mkdir ../tmpbuild
cd ../tmpbuild
cmake -DCMAKE_INSTALL_PREFIX=../tmpinstall/ ../cmakeExercise
make
make install
echo '#include <iostream>
#include <vector>
#include "SortIntegers.h"
int main() { 
  std::vector integers = {6, 3, 1, 15, 11, 16};
  sortIntegers(integers);
  for (auto integer : integers) { std::cout << integer << "\n"; }
  return 0;
}' > test.cpp
clang++ test.cpp -std=c++17 -I../tmpinstall/include -L../tmpinstall/lib/ -lsortIntegers
./a.out
../tmpinstall/bin/fibSorter
../tmpinstall/bin/cursesExample

One you are comfortable with your exercise, create an archive of your solution by changing into the directory containing your project and running:

tar zcvf e1.tar.gz cmakeExercise/

Note that this should not contain your build directory.

You can then upload this archive to CourSys.

Bonus Step: Using Other Tools

Ninja

As discussed in class, using CMake provides many useful features. For instance, you are not required to use make in order to build your project. You can have CMake instead generate, e.g., an Eclipse project, a Visual Studio project, an XCode project, and more. When building large projects, I like to use Ninja because it is faster than many other build systems and thus allows me to wait less for compilations to complete. To build a project using Ninja, you can use

cmake -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=on /path/to/project/
ninja
ninja install

clang-tidy

In addition, the JSON compilation databases that CMake can export allow you to more easily use advanced analysis tools like clang-tidy. As demonstrated during class, such tools can provide great aid in both uncovering bugs and in identifying potential stylistic or design issues in a project. To use clang-tidy, you must pass it a generated compilation database and tell it what checks you want to run as well as which files you wish to analyze.

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=on /path/to/project/
cd /path/to/project/
clang-tidy -p /path/to/build/compile_commands.json -checks='*' lib/*/*.cpp tools/*/*.cpp

The last two lines can also be simplified to:

run-clang-tidy -p ./ -checks='*'

cmake-init

This exercise has you write the CMakeLists.txt files and organize a project yourself to help you develop your mental model of how these activities work. In practice, many of these tasks can be automated and performed for you, requiring only fine-tuning later to meet your needs.

One such tool for generating cmake project structures is cmake-init. It can generate the CMakeLists.txt files and project structures that include tasks like static or dynamic analysis integration and even the use of package managers like conan or vcpkg.

There are many more features available in CMake. You should feel free to explore the official CMake tutorial and other documentation online and see what else is available that may be helpful in your project.

  1. A library is another type of target. ↩

  2. There are other ways that a project can be organized. One approach is to break a project into individual subprojects, each in its own directory. CMake provides additional commands that can allow projects to use/communicate with each other. You can find more information on advanced uses of CMake here. ↩