12 April 2013

C++ Include test: full matrix

Finally I am able to present experiment results about including headers in C++! Previously I wrote about Code Generator and now I can actually run this tool and get some numbers out of it. I compared VC11.0 (Visual Studio 2012 For Desktop) and GCC 4.7.2 (using MinGW32 4.7.2 and DevCpp).

Basics of experiment

  • VC11.0 - Visual Studio 2012 For Desktop, Release Mode, 32bit, no optimizations.
    • I turned build timings on to have detailed build performance.
  • GCC 4.7.2 32bit - MinGW 4.7.2 version and run from DevCpp 5.4.1
    • g++.exe -c testHeaders.cpp -o testHeaders.o -ftime-report
  • The machine: Core i5, 4 cores, 4GB RAM, Windows 8 64bit
    • compilation will probably take place only on one core only (there will be one translation unit only)
I run each test 3 times and then compute the average. At the end of post there is a link to detailed spreadsheet.

Code structure

The overall code structure is not a real case scenario. It was my first attempt to do such experiment and it was actually easy to create such hierarchy.

  • testHeader.cpp includes N header files
  • m-th header file includes N-1 other header files (so we have "cross" include)
  • each header file has its proper include guard
Note: For 199 GCC will return an error "#include nested too deeply". You can read more about it in gamesfromwithin.com. In general GCC thinks that 199 include level is unreal and will block it.

Test N headers

A header can look like this in this tests:
#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."
generator.exe N 100 includeTestOutput/
N = 100
GCC3,95s
GCC Compilation2,99s
VC11.02,90s
VC11.0 Compilation2,68s
N = 132
GCC5,37s
GCC Compilation3,98s
VC11.04,31s
VC11.0 Compilation4,11s
N = 164
GCC6,49s
GCC Compilation4,92s
VC11.06,10s
VC11.0 Compilation5,91s
N = 192
GCC7,40s
GCC Compilation5,77s
VC11.07,98s
VC11.0 Compilation7,77s

Test N headers - external guards

A header can look like this in this tests:
#ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #ifndef _INCLUDED_HEADER_HEADER_0_H
        #include "header_0.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_1_H
        #include "header_1.h"
    #endif
    #ifndef _INCLUDED_HEADER_HEADER_2_H
        #include "header_2.h"
    #endif
    #include "..."
generator.exe N 100 includeTestOutput/ ifDef
N = 100
GCC3,91s
GCC Compilation2,96s
VC11.01,44s
VC11.0 Compilation1,22s
N = 132
GCC5,35s
GCC Compilation3,91s
VC11.01,71s
VC11.0 Compilation1,51s
N = 164
GCC6,41s
GCC Compilation4,86s
VC11.01,98s
VC11.0 Compilation1,77s
N = 192
GCC7,31s
GCC Compilation5,69s
VC11.02,16s
VC11.0 Compilation1,96s

Test N headers - #pragma once

A header can look like this in this tests:
#pragma once
    #ifndef _INCLUDED_HEADER_HEADER_5_H
    #define _INCLUDED_HEADER_HEADER_5_H

    #include "header_0.h"
    #include "header_1.h"
    #include "header_2.h"
    #include "..."
generator.exe N 100 includeTestOutput/ pragmaOnce
N = 100
GCC4,02s
GCC Compilation3,08s
VC11.01,48s
VC11.0 Compilation1,28s
N = 132
GCC5,42s
GCC Compilation4,06s
VC11.01,84s
VC11.0 Compilation1,65s
N = 164
GCC6,64s
GCC Compilation5,08s
VC11.02,06s
VC11.0 Compilation1,86s
N = 192
GCC7,60s
GCC Compilation5,98s
VC11.02,39s
VC11.0 Compilation2,20s

Conclusion

  • The code structure is rather theoretical and does not represent 'common' structures that may appear in projects.
  • GCC and VC build code a bit different. GCC linker phase is much longer than in VC.
  • GCC uses lots of optimization for header files. There is almost no need to do any 'tricks' with includes. Header guards are enough.
  • VC likes header files 'tricks'!
    • There is above 2X to 3X speedup using additional (external) include guards or pragma once
    • Pragma once for such code structure seems to be a bit slower than additional (external) include guards: from 5% to even 10% slower.
  • All in all VC11.0 is faster than GCC.

What I have learnt

  • VC Release mode is not the same as GCC basic compilation
  • Make sure you set up proper experiment base
  • Automate as much as possible
  • Tool for spreadsheet is a must have-software :)
  • It is valuable to figure out how to break the compiler

Links

Article is posted also on the CodeProject

Interested in new blog posts and occasional updates? Please sign up for my free newsletter.

Copyright Bartlomiej Filipek, 2016, Blogger platform
Any opinions expressed herein are in no way representative of those of my employers.
This site contains ads or referral links, which provide me with a commission. Thank you for your understanding.