Precompiled headers

Microsoft Visual C++ (VC++) can compile code very quickly, if it is setup properly. Unfortunately, the precompiled header settings are peculiar, badly documented, and frequently have inappropriate defaults. Therefore, many projects using VC++ have their precompiled header files setup incorrectly and they build slowly. It is not unusual to have projects building four to six times slower than they need to, all because of bad precompiled header setup. Even Microsoft seems to have difficulty with these settings - the DirectX samples (up to version 8 at least) don't use precompiled header files, and therefore build slower.

Once you understand your Visual C++'s precompiled header files work, and how to analyze their performance, optimizing your build times is easy. Here's what some happy readers had to say:

Thanks for your excellent page about precompiled headers in VC++. My rebuild times went from roughly 11 minutes to 1.44! Modifying all files to use pre-compiled headers took only 35 minutes, so it was well worth it.
Mikael Sterner
I just set up my D3D/Python project to use precompiled headers...the full rebuild went from 2min to 21.5 secs!! Never knew I was wasting so much build time on loading headers...
Benjamin Crane
Game Programmer
Just writing to say a quick "thanks" for the page about precompiled headers.
Build time on my project is now 15% of what it was before!
Richard Mitton
Xbox programmer

Let's be scientific

Anytime you try and optimize something - whether it's your memory footprint, your frame rate or your build time - it is important to do measurements. Otherwise you may make incorrect decisions about what actually makes things better. Also, measurements let you more precisely brag to your co-workers - you can say "I made the build run 4.5 times faster!" Luckily VC++ has an option to measure build times.

With Visual C++ 6.0 you have to run devenv.exe with the undocumented /y3 option.

In Visual Studio .Net you can get similar functionality by going to Tools->Options->Projects->VC++ Build and setting Build Timing to Yes.

They will then dump out build times for each part of your build. Here's sample output from VC++ 6.0:

--------------------Configuration: InterlaceSimDDraw - Win32 Debug--------------------
Compiling...
InterlaceSimDDraw.cpp
Spawn Time 0:01.9
Linking...
Spawn Time 0:00.2

InterlaceSimDDraw.exe - 0 error(s), 0 warning(s)
Build Time 0:02.5

Because of the disk cache and other issues it is important that you ignore the timing results of your first build - it will usually be substantially longer.

That's great for measuring your progress, but it doesn't tell you if your build times are great or terrible, and it doesn't tell you how to improve them. It turns out that the worst problems with precompiled header files are when the .pch file is being rebuilt multiple times. Normally it is very difficult to detect this - watching to see if the file date changes in the middle of a build is pretty tedious.

However there's a wonderful diagnostic that I discovered quite accidentally. If you put a #pragma message into a header file then it will only be displayed when that file is actually being compiled - not when it is being pulled out of the .pch file! This is an incredibly useful diagnostic tool. If you add this pragma to a few key source files that should always be precompiled - like windows.h - then you will quickly find out which projects have problems.

I add some variation on this line to all copies of windows.h on my machine:

#pragma message("Compiling windows.h - this should happen just once per project.\n")

While modifying windows.h is obviously a bad habit to get into, I think this is one of those times when breaking the rules is the right thing to do.

The first project I tested this on was spitting out that message on 91 of the 102 source files. In other words, the precompiled header file was being built so often that it was effectively useless. I even tested this by turning precompiled header files off completely, and the build time stayed virtually the same.

The contents of stdafx.h

Before going any further, add the /y3 option to your Developer Studio shortcut, add a #pragma message to windows.h or some other huge and unchanging header file that should be precompiled. Do a couple of rebuilds and record how long they take, and how many times the message shows up. If it shows up just once then congratulations - your precompiled header files are setup correctly. Proceed to the "physical design" section.

If you're still here then the message must have showed up a few times. The good news is, the more times the message showed up, the more room there is for improvement. In a few minutes your project will be building faster.

The useful version of Microsoft's precompiled header system relies on a header file that contains all of the preprocessing that should be put into the .pch file. This file, typically called stdafx.h or precompiled.h, must be included by every file in your project, as the first preprocessor directive. Any code - including other #includes or defines - will be silently ignored if it comes before this include. Watch out for that.

If you forget to include your precompiled header file then you will get the following rather cryptic message:

fatal error C1010: unexpected end of file while looking for precompiled header directive

Just remember that the "precompiled header directive" it is looking for is the include of the header file you specified.

The precompiled header file should include the big header files which slow down your builds. The most obvious candidate is windows.h, the 800 lb gorilla of the header file world. Other possible candidates are STL header files.

What you shouldn't put in here is header files from your project. Any file that is included from here will cause a total rebuild of your project when it is changed. Your own header files should be included from an absolute bare minimum of header files in order to reduce the number of files that rebuild when you change them.

The one exception to this rule that I allow is my "core.h" file. This file defines a few types and macros that I like to have available in all of my projects, and putting it in the precompiled header file is a convenient way of doing this. This is an okay violation of the rule because I want all of my source code to have access to this file, and I very rarely change it.

If you have small header files that don't include much, there is no advantage to putting them in the precompiled header file. If you have a large header file that is only included from a few places you shouldn't put it in your precompiled header file because this will cause more total rebuilds. Finally, if you have a large header file that is needed everywhere then you need to fix the physical design of your program - and you can't do that if you are including this file from your precompiled header file.

Setting up precompiled headers

The only precompiled header settings that are efficient to use are the Create/Use pair of settings. The documentation for these settings tells you what each setting does, but it fails to give you the big picture - it doesn't tell you how to setup a project to use these two together.

The basic idea is that one cpp file is specified as creating the precompiled header file and the other cpp files uses that precompiled header file. Because only one source file ever creates the precompiled header file it is guaranteed that it will only get built once per total rebuild. Unlike the "automatic" setting, which punishes mistakes by silently rebuilding the precompiled header file and slowing your build, the create/use pair gives you an error message if you use it incorrectly, thus ensuring that you will fix the problem and continue to have fast builds. The error messages tend to be exquisitely cryptic, but they're better than nothing.

First you need to have one header file that every source file in your project will include. This is typically called stdafx.h or precompiled.h. If you don't have such a file, create it now. Check it to make sure it is including the appropriate set of header files - big header files that never change. Now make sure that every source file in your project is including this header file as the first non-comment thing that they do. A Python script to automatically insert this at the beginning is my preferred way of making this change to large projects.

Now you need one source file whose only job is to create the precompiled header file. It's best to have a source file exclusively for this purpose because every time this source file is modified, the .pch file will be regenerated. Generating the .pch file is an expensive operation that we are trying to avoid. This source file will typically be called stdafx.cpp or precompiled.cpp - to match the header file.

Now it's time to go to project settings.

First, make sure you select "All Configurations" so that your fixes will affect both debug and release builds of your project.

Now, go to the C/C++ tab, precompiled headers category. Select "Use precompiled header file" and type in the name of your precompiled header file (stdafx.h or precompiled.h).

If you try to build now your build will fail, giving a cryptic error about how it doesn't know how to create the .pch file. You need to tell it how. In the explorer view in project settings, open up your project and navigate down to the .cpp file that you chose for creating the precompiled header file (in VS.Net you will select that file from the Solution Explorer). Select that file, with "All Configurations" still selected, and select "Create precompiled header file", and type in the name of your precompiled header file.

It kind of makes sense once you go through the steps. You give VC++ one source file to create the precompiled header file, and the other files all use it. This guarantees that it won't be rebuilt unecessarily. You have to #include this file from all of your source files so that if you turn precompiled headers off, your program will compile in the same manner.

The one fly in the ointment, the worrisome aspect of this feature, is that there is actually no guarantee that your program will behave identically, because VC++ doesn't enforce that your #include of the precompiled header file comes first. It enforces that you include the precompiled header file in each source file, but it will happily ignore includes, defines, and other code before that include. If you notice yourself getting some strange errors after changing your precompiled header files, that's probably why. Go through all of your source files and make sure that your precompiled header file is included first.

That's it. If you have any problems then double check all of the steps, check the problematic source files for any preprocessor directives or code before the include of the precompiled header file, and maybe do a rebuild-all for good measure. If you want an example of a properly setup project to use as a reference, use the AppWizard to create an MFC project.

Let me know how it works.

Advanced precompiled header usage

If you have a library that creates a precompiled header file, it seems a shame for a project that uses that library to have to recreate its own .pch file from scratch. It turns out that it doesn't have to. It is possible to use one .pch file while creating another one.

The disclaimer on this technique is that I haven't used it, and I'm not convinced it's worth the hassle and confusion. The IDE doesn't support it so you'll have to type some of these options in yourself. For all the gory details, read this MSDN article - Two Choices for Precompiling Code.

Good physical design matters also

C++ encourages a style of programming that has more dependencies between source files than in C. Inline functions and other implementation details in headers lead to many source files including many header files. In the worst case, rebuilding after changing a header file can take time proportional to the square of the number of source files (every source file rebuilds, and every source file includes virtually all header files). On a project that pays attention to physical design it is typically possible to rebuild after modifying a header file in "constant time" - one or two source files recompile, and they only include a few header files so they compile quickly. On a project with a hundred source files this can potentially mean a rebuild time that is several hundred times faster!

Some aspects of physical design are easy. Use forward declarations to avoid including header files that aren't strictly necessary. Simple steps like doing a #define of WIN32_LEAN_AND_MEAN can improve your build times by stripping out rarely used parts of windows.h. Other methods can get much more complicated, and it's important to decide how to balance programming convenience, build time, and run time performance.

For a detailed coverage of physical design principles, read:
Large Scale C++ Software Design, by John Lakos

Make those source files bigger

If you have a project that has bad build times then there is one unfortunate technique that will frequently help you. I say unfortunate because it goes against the usual rules for what is appropriate. First, some background:

C++ source files tend to include a lot of files. I can advise against this all I want, but the reality is that it happens. You can fight it, and minimize it, but you'll still end up with a fair number of includes. The trouble with these includes is that they get reprocessed for every source file. That's expensive. Header files are such a huge chunk of C++ compilation time that on many projects all source files take the same amount of time to compile, regardless of size. That leads us to this suggestion...

Let's say you're working on a project and you have dozens of different types of related classes. Normally you would put each type in its own .cpp file. However these different classes have a lot in common - including header files that they need - so I am going to suggest that you should seriously consider putting multiple class definitions in one file. Sure, it's a bit messier, and if your source file ends up being 10,000 lines long it may be completely unmanageable. But if each source file takes one second to compile, and if you can reduce the number of source files by ten, then your build time may improve by ten times!

One concrete example of this technique was on a Game Cube game I was working on. The build was rather slow, at least partially because the main project had 150 source files, with an average length of just a few hundred lines. As an experiment I wrote a script that generated 25 source files, each of which included six of the real source files. Then I created a project that compiled those 25 'meta source files'. It worked the first time and the project built roughly four times faster! Now, you're not always doing a full rebuild, and this technique slightly increases the time to build a single source file, but it doesn't increase it much - typically only by a fraction of a second. And, in this particular case, due to some quirks to do with the handling of debug information, having fewer object files dramatically improved the link time, so even the build time when just a single file had changed was improved.

I don't actually recommend the "#include " technique, but I would recommend not creating hundreds of tiny source files. While large source files can be unmanageable to edit, having too many source files can also be unmanageable - and slow to build.

I think that having an average .cpp length of about 1,000 lines is probably reasonable. You can have shorter files on small projects, but may want larger files on large projects. If your object definitions are small, and if you space them out well, there is no reason why a single source file can't have five to ten class definitions in it.

I've tried everything - it's still too slow!

One interesting technique that some people have used to analyze their builds is using ntfilemon from sysinternals to track down which .cpp files and header files are taking the most time, and to see how many header files you're actually including. You may be surprised. With Visual Studio .Net you can get a hierarchical list of header files used by setting Properties->C/C++->Advanced->Show Includes to yes.

If all these tricks don't work you can always throw money at the problem. Buy everybody faster computers with lots of memory, and then buy Incredibuild to let you do network builds. I haven't used this product, but I know smart people who swear buy it.

The defaults are dangerous

For some projects - typically non-MFC projects, but the specifics depend on what version of VC++ you are using - the project wizard's set the precompiled header option to the seductive sounding "Automatic use of precompiled headers." Microsoft itself has admitted that this option is inefficient - yet they make it the default for many projects. Unless you are exquisitely careful this "automatic" use of precompiled headers translates to "rebuilding the precompiled headers for every source file." It is difficult to avoid this and keep it working properly. Don't use it. Either disable precompiled headers, or use them properly.

If you use the DirectX AppWizard to create a non-MFC project, the newly created project will use the "Automatic use of precompiled headers" setting. Because of this these projects will take approximately twice as long to build as they need to. You need to manually fix them to get good build performance.

What about portable code, that shouldn't be including windows.h everywhere?

If you're trying to write code that isn't Windows dependent you don't want to put windows.h in your precompiled header file. Instead, just include it from the one or two files that actually need it. If more than one or two files need it, move them into a separate library with windows.h in the precompiled header file for just that library.

Summary

Add /y3 to your msdev startup shortcut (for Visual Studio .Net go to Tools->Options->Projects->VC++ Build and set Build Timing to Yes)
Add #pragma message("Compiling windows.h") to windows.h
Use the Create/Use precompiled headers settings everywhere.
Don't include project files from your precompiled header file - just include system files that will never change.
Prime candidates for inclusion in the precompiled header file are windows.h, vector, or any expensive header files that never change and that are needed by a significant number of your source files.
Other things can slow down your builds also.

Let me know what sort of build time improvements you get.

Please contact me if you have any questions.