Sponsors: KDAB and Whole Tomato Software

12 April 2013

C++ Include test: full matrix

Finally I am able to present experiment results about including headers in C++! Previously I wrote about Code Generator and now I can actually run this tool and get some numbers out of it. I compared VC11.0 (Visual Studio 2012 For Desktop) and GCC 4.7.2 (using MinGW32 4.7.2 and DevCpp).

Basics of experiment

  • VC11.0 - Visual Studio 2012 For Desktop, Release Mode, 32bit, no optimizations.
    • I turned build timings on to have detailed build performance.
  • GCC 4.7.2 32bit - MinGW 4.7.2 version and run from DevCpp 5.4.1
    • g++.exe -c testHeaders.cpp -o testHeaders.o -ftime-report
  • The machine: Core i5, 4 cores, 4GB RAM, Windows 8 64bit
    • compilation will probably take place only on one core only (there will be one translation unit only)
I run each test 3 times and then compute the average. At the end of post there is a link to detailed spreadsheet.

Code structure

The overall code structure is not a real case scenario. It was my first attempt to do such experiment and it was actually easy to create such hierarchy.

  • testHeader.cpp includes N header files
  • m-th header file includes N-1 other header files (so we have "cross" include)
  • each header file has its proper include guard
Note: For 199 GCC will return an error "#include nested too deeply". You can read more about it in gamesfromwithin.com. In general GCC thinks that 199 include level is unreal and will block it.

Test N headers

A header can look like this in this tests:
#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."
generator.exe N 100 includeTestOutput/
N = 100
GCC3,95s
GCC Compilation2,99s
VC11.02,90s
VC11.0 Compilation2,68s
N = 132
GCC5,37s
GCC Compilation3,98s
VC11.04,31s
VC11.0 Compilation4,11s
N = 164
GCC6,49s
GCC Compilation4,92s
VC11.06,10s
VC11.0 Compilation5,91s
N = 192
GCC7,40s
GCC Compilation5,77s
VC11.07,98s
VC11.0 Compilation7,77s

Test N headers - external guards

A header can look like this in this tests:
#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #ifndef _INCLUDED_HEADER_HEADER_0_H
        #include "header_0.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_1_H
        #include "header_1.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_2_H
        #include "header_2.h"
    #endif
    #include "..."
generator.exe N 100 includeTestOutput/ ifDef
N = 100
GCC3,91s
GCC Compilation2,96s
VC11.01,44s
VC11.0 Compilation1,22s
N = 132
GCC5,35s
GCC Compilation3,91s
VC11.01,71s
VC11.0 Compilation1,51s
N = 164
GCC6,41s
GCC Compilation4,86s
VC11.01,98s
VC11.0 Compilation1,77s
N = 192
GCC7,31s
GCC Compilation5,69s
VC11.02,16s
VC11.0 Compilation1,96s

Test N headers - #pragma once

A header can look like this in this tests:
#pragma once
    #ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."
generator.exe N 100 includeTestOutput/ pragmaOnce
N = 100
GCC4,02s
GCC Compilation3,08s
VC11.01,48s
VC11.0 Compilation1,28s
N = 132
GCC5,42s
GCC Compilation4,06s
VC11.01,84s
VC11.0 Compilation1,65s
N = 164
GCC6,64s
GCC Compilation5,08s
VC11.02,06s
VC11.0 Compilation1,86s
N = 192
GCC7,60s
GCC Compilation5,98s
VC11.02,39s
VC11.0 Compilation2,20s

Conclusion

  • The code structure is rather theoretical and does not represent 'common' structures that may appear in projects.
  • GCC and VC build code a bit different. GCC linker phase is much longer than in VC.
  • GCC uses lots of optimization for header files. There is almost no need to do any 'tricks' with includes. Header guards are enough.
  • VC likes header files 'tricks'!
    • There is above 2X to 3X speedup using additional (external) include guards or pragma once
    • Pragma once for such code structure seems to be a bit slower than additional (external) include guards: from 5% to even 10% slower.
  • All in all VC11.0 is faster than GCC.

What I have learnt

  • VC Release mode is not the same as GCC basic compilation
  • Make sure you set up proper experiment base
  • Automate as much as possible
  • Tool for spreadsheet is a must have-software :)
  • It is valuable to figure out how to break the compiler

Links

Article is posted also on the CodeProject

Interested in new blog posts and bonus content? Sign up for my newsletter.

© 2017, Bartlomiej Filipek, Blogger platform
Any opinions expressed herein are in no way representative of those of my employers.
This site contains ads or referral links, which provide me with a commission. Thank you for your understanding.