Speed up C++ build times by means of parallel compilation

Everyone who has worked on a fair-sized C/C++ project surely knows these scenarios: sometimes it’s unavoidable to change or introduce a new #define or declaration that is used nearly everywhere. When you hit the ‘Build’ button next time, you’ll end up recompiling nearly the whole thing. Or you just came to the office, updated your working copy and want to start the day with a clean build.

The complexity of the C++ language in combination with the preprocessor makes compilation orders of magnitude slower, compared to modern languages such as C#. Precompiled headers help here a bit, but it’s not a solution to the problem inherent to the language itself, only a mere optimization. There are coding practices that help a lot, not only in making robust and maintainable software, but also helping to improve build times. They go along the way of “minimize dependencies between modules” or “#include only what you use directly”. There are also tools that visualize #include trees and help you identify hot-spots. These are all clever tricks, which I may discuss later. However, this article is about raw, brute force :) You just got a new, powerful, N-core workstation? Well, let’s get those cores busy…

C++ translation units (.cpp files) are independent during the compilation phase and indeed are compiled in isolation. Therefore, the speed of compilation scales almost linearly with the number of processors. Most IDEs and build tools nowadays come with an option to enable parallel compilation. However, this option is almost never enabled by default. I will show you how to enable parallel compilation in build systems with which I have some experience:

  • Makefiles (Linux and Windows)
  • Qt’s Qtcreator IDE (Linux and Windows)
  • MS Visual Studio, MSBuild

Makefiles – gnu make

Telling the make program to compile in parallel could not be simpler. Just specify the -j N (or –jobs=N) option when calling make, where N is the number of things you want make to run in parallel. Good choice is to use the number of CPU cores as N. Warning: if you use -j but do not specify N, make will spawn as many parallel jobs as there are targets in the Makefile. This is neither efficient nor desirable.

Makefiles on Windows – nmake, jom

On Windows, Visual Studio comes with its own version of the make program called nmake.exe. Nmake does not know the -j option and can’t do parallel jobs. Luckily, thanks to the Qt community, there is an open source program called “jom”, which is compatible with nmake and adds the -j option. You can download the source and binary from here: http://qt.gitorious.org/qt-labs/jom. Installation is very simple, just extract the .zip file anywhere, optionally add it to %PATH%. Use it like you would use nmake.

Qt’s Qtcreator

First, let me say that Qtcreator is a very promising cross-platform IDE for (not only Qt) C++ projects, completely free and open source. Not surprisingly, Qtcreator uses the Qt’s qmake build tool first to generate a Makefile from a project description .pro file. Then it simply runs make on the generated Makefile. Qtcreator allows you to pass additional arguments to the build commands: Projects -> Build Settings -> Build Steps -> Make -> Make arguments: here you can specify the -j N option.

Project build settings in Qtcreator on Linux.

Qtcreator on Windows

If you use Qtcreator on Windows, the story is almost the same with only minor differences. On Windows platforms Qtcreator uses the MinGW32 build toolchain. Unfortunately due to the way (bug) MinGW’s make works on Windows and the way Qt’s qmake generates Makefiles, the -j option doesn’t work. The reason why and various workarounds are described in this discussion. One easy way is to override the mingw32-make.exe and use jom.exe instead.

Project build settings in Qtcreator on Windows.

MS Visual Studio, MSBuild

Not surprisingly, the Visual Studio/C++ IDE uses a completely different build system than the GNU toolchain, called MSBuild (formerly VCBuild). If you only work within the IDE and do not wander into the command line world very often, you probably haven’t even bumped into this tool. Yet it is invoked behind the scenes whenever you press the build button. In short, the process is as follows: Visual Studio keeps the list of project source files, compiler and linker options in a .vc(x)proj file. At the start of each build, the MSBuild tool then crunches the .vcxproj file and outputs a list of commands for invoking the compiler, the linker and any other tools involved in the build process.

The MS Visual C++ compiler (cl) can compile multiple source files in parallel, if you tell it to using the /MP switch. It will then spawn as many parallel processes as there are installed CPU cores in the system. You can set this option conveniently from the IDE: Project -> Properties -> Configuration Properties -> C/C++ -> General -> Multi-processor Compilation: Yes (/MP). This option will be saved into the .vcxproj file, so multi-process compilation will be used regardless if you build in the IDE or from the command line.

Enable parallel compilation for a MSVC project.

Multiple simultaneous builds

In Visual Studio, you can go even a little further and tell the IDE to build multiple projects in parallel. To enable this, go to: Tools -> Options -> Projects and Solutions -> Build and Run: and set the maximum number of parallel project builds. When building a solution from the command line, pass this option to MSBuild: /maxcpucount[:n]. This can be useful, if your solution consists of many small, independent projects. If your solution contains just a single or a couple of big projects, you’ll probably do best with the /MP option only.

Setting maximum number of parallel builds.

In closing

Modern machines come with a lot of horsepower, the trend is that the number of CPU cores will be increasing. Why not leverage this and turn your workspace builds from a lunch break into “only” a coffee break? Parallel compilation speeds up the build process almost linearly with the number of CPU cores.

However compilation is only one part of the story. Then there’s linking. It is not uncommon that a project, which compiles in seconds, takes minutes to link. I will point you to some articles on how to speed up linking in my next post.


3 thoughts on “Speed up C++ build times by means of parallel compilation

  1. Amir Gonnen

    It’s worth mentioning that make -j is much stronger than msbuild parallelization options since make can build different *targets* in parallel, while msbuild cannot.
    Msbuild is limited to either building different projects in parallel, or compiling C code in parallel using CL. If you define different targets on your project file, they cannot be built in parallel.
    (At least up until Visual Studio 2010, I hope they change it in the future)

  2. martin.kutny Post author

    Hi Amir. It should be easily possible to spawn two instances of MSBuild (from two consoles), each building a different target of the same project/solution concurrently. Only take care that different targets use different intermediate and output directories.

  3. Amir Gonnen

    That’s true, but it’s a “Hack”.
    You could also run two parallel instances of “Make” with the same makefile and different targets, but no one does this. It’s much easier to add “-j” option and let Make do the work. Make will automatically identify the Target’s dependencies and it will know which targets can run in parallel.
    What you suggest is that we (humans) do that dependencies calculation manually in order to know which targets can be spawned in parallel. In complex build systems this can be difficult, but this is a classic task for an automated build system.

    My point is that Microsoft missed something. They did not think about parallelism thoroughly when designing their build system and they didn’t try to learn from mature build systems such as Make.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.