Skip to main content

RI code build time can be reduced to half of current build time

8 replies [Last post]
ssathish
Offline
Joined: 2011-05-26
Points: 0

RI code build time can be reduced to almost half of the current build time using the following steps.
- Enable ccache in linux/cygwin
- Enable parallel building of platform makefiles

Installing ccache:
On linux this is available via yum/apt-get etc. Ccache is also available in Cygwin, it can be installed from the usual cygwin setup.exe.

Linux:
cd /usr/local/bin
sudo ln -s /usr/bin/ccache gcc
sudo ln -s /usr/bin/ccache g++
sudo ln -s /usr/bin/ccache cc
sudo ln -s /usr/bin/ccache c++

Cygwin:
cd /usr/local/bin
ln -s /usr/bin/ccache gcc
ln -s /usr/bin/ccache g++
ln -s /usr/bin/ccache cc
ln -s /usr/bin/ccache c++
ln -s /usr/bin/ccache gcc-3
ln -s /usr/bin/ccache g++-3
ln -s /usr/bin/ccache cc-3
ln -s /usr/bin/ccache c++-3

In addition to the above modifications, we need to populate the number of processors available from cpu by using the following steps:

For linux:
NUM_PROCESSORS=$(grep 'processor.*:' /proc/cpuinfo |wc -l)
export NUM_PROCESSORS=$(($NUM_PROCESSORS + 1))

For cygwin-win32:
export NUM_PROCESSORS=1

The first time build might be a little slower so as to populate the cache and further builds will get speed up.

Testing:
These modifications were tested in Win-32 and linux platforms and results are:

-------------------------------------------------------------------------------
build time in Win-32 - linux
-------------------------------------------------------------------------------
Changing 2 files in MPEOS layer - 1 min 40 secs - 58 secs
and one file in platform layer
-------------------------------------------------------------------------------
Changing 1 java file, 2 MPEOS - 8 mins - 4 mins
and 1 platform file.
-------------------------------------------------------------------------------

I have attached the patch file {parallel_build.patch} and also a setting up environment variables script file {setEnv}.

Please let me your opinion for using the aforesaid modifications to reduce RI build time and if it is useful whether shall I raise issue tracker for the same?

AttachmentSize
patch_files.zip3.02 KB

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
smaynard
Offline
Joined: 2009-01-27
Points: 0

also - can you elaborate on your NUM_PROCESSORS settings; i.e. is this a ccache requirement and if so which version of ccache are you using?

smaynard
Offline
Joined: 2009-01-27
Points: 0

There is significant improvement (platform test builds below). I'm looking into the unsupported compiler options that are logged each build. Can you provide similar result output from your experience? In addition, It would be preferable to modify the gcc and g++ macros in the defs.mk files to simplify switching back and forth (cached versus non-cached builds); however, the builds fail when attempting to run in this fashion as it appears that ccache observes it's called name and expects the command line args to be its own versus compiler args. Did you experience this?

Init test == ccache -C; ccache -z
Test build (from $PLATFORMROOT) == cd src; make clean purge; time make; ccache -s

first win32 build:

real 5m2.083s
user 5m15.626s
sys 2m0.245s

cache directory /cygdrive/c/home/smaynard/.ccache
cache hit 3
cache miss 146
called for link 12
not a C/C++ file 3
unsupported compiler option 149
files in cache 292
cache size 72.7 Mbytes

second win32 build:

real 3m53.704s
user 4m3.162s
sys 1m53.978s

cache directory /cygdrive/c/home/smaynard/.ccache
cache hit 152
cache miss 146
called for link 24
not a C/C++ file 6
unsupported compiler option 298
files in cache 292
cache size 72.7 Mbytes

first Linux build:

real 0m45.245s
user 0m32.164s
sys 0m6.731s

cache directory /home/smaynard/.ccache
cache hit (direct) 2
cache hit (preprocessed) 1
cache miss 145
called for link 14
unsupported compiler option 148
files in cache 278
cache size 9.0 Mbytes

second linux build:

real 0m14.385s
user 0m8.496s
sys 0m3.139s

cache directory /home/smaynard/.ccache
cache hit (direct) 136
cache hit (preprocessed) 14
cache miss 145
called for link 28
unsupported compiler option 296
files in cache 278
cache size 9.0 Mbytes

ssathish
Offline
Joined: 2011-05-26
Points: 0

Yes, the unsupported compiler options is getting increased for each build.

NUM_PROCESSORS is not ccache requirement, but it is to enable multiple process with available processors in cpu.
In setEnv script (already attached in patch_files.zip), the NUM_PROCESSORS variable will be assigned with no. of processors using command "$(grep 'processor.*:' /proc/cpuinfo |wc -l)". This variable is further used in platform makefile scripts with -j option.

Results from linux build:

Before build:
=============
$ ccache -s
cache directory /home/krushna/.ccache
cache hit 0
cache miss 0
files in cache 0
cache size 0 Kbytes
max cache size 976.6 Mbytes

-----------------------------------------------------------

1st build: [Fresh build]
==========
$ ccache -s
cache directory /home/krushna/.ccache
cache hit 47
cache miss 3464
called for link 447
multiple source files 1
compile failed 159
preprocessor error 112
not a C/C++ file 99
autoconf compile/link 538
unsupported compiler option 790
no input file 176
files in cache 6928
cache size 205.8 Mbytes
max cache size 976.6 Mbytes

Build time = ~25 mins

-----------------------------------------------------------

2nd Build: [modified 3 platform files]
==========
$ ccache -s
cache directory /home/krushna/.ccache
cache hit 943
cache miss 3468
called for link 660
multiple source files 1
compile failed 159
preprocessor error 112
not a C/C++ file 134
autoconf compile/link 538
unsupported compiler option 946
no input file 190
files in cache 6936
cache size 206.5 Mbytes
max cache size 976.6 Mbytes

Build time = 40 secs

-----------------------------------------------------------
3rd build: [modified 3 files]
==========
$ ccache -s
cache directory /home/krushna/.ccache
cache hit 1842
cache miss 3469
called for link 873
multiple source files 1
compile failed 159
preprocessor error 112
not a C/C++ file 169
autoconf compile/link 538
unsupported compiler option 1102
no input file 204
files in cache 6938
cache size 207.0 Mbytes
max cache size 976.6 Mbytes

Build time = 60 seconds

smaynard
Offline
Joined: 2009-01-27
Points: 0

I have a fix for the unsupported compiler options and will check that in today. The NUM_PROCESSORS seems to have no affect for me. what version of make are you using?

ssathish
Offline
Joined: 2011-05-26
Points: 0

NUM_PROCESSORS is effective in Linux, but not in Win-32.
I have used make file version: GNU Make 3.81 on both Windows and Linux platforms.

smaynard
Offline
Joined: 2009-01-27
Points: 0

After realization that your NUM_PROCESSORS environment variable was to be provided to make as a jobs option, we want to clarify. The number of jobs is not necessarily related to the number of processors that you have, but rather a function of your disk speed and memory; i.e. if you have a slow disk but sufficient memory, you can set the maximum number of make jobs to something like 16 (make -j16) and make (while respecting dependencies) will start a job, if it becomes I/O blocked it will start another job, and so on (up to the maximum number of jobs allowed).

Since this number is relative to each developer's machine, it is best set by that individual developer at the command line (supplying the number of jobs in the recursive make commands as in your patch is not recommended). We will discuss setting "MAKEFLAGS += -j8" in the host's defs.mk...

Make on cygwin is GNU-based and also respects the -j option.

ssathish
Offline
Joined: 2011-05-26
Points: 0

Shall I raise an issue tracker for these suggestions?

smaynard
Offline
Joined: 2009-01-27
Points: 0

An issue is not required. We view these settings as "environment choices"; i.e. each developer may choose to use ccache or not as it only functions correctly with the symbolic links and cannot be applied via our normal defs.mk configuration files.

The -j option to make may also be applied at the command line on an individual basis; it is not suggested to be used in recursive make calls so your patch will not be applied. Our testing revealed that some dependency issues between libraries can arise on complete rebuilds and go unnoticed as the make output can show an error but subsequent make output can make that error scroll off. We did see that normal rebuilds do not reflect these same dependency issues as the libraries are already built and are much faster as you mention.