Skip to main content

Unstable performance for Volano benchmark

3 replies [Last post]
Joined: 2007-03-10

there is a strange thing: in latest avaible jre 1.6 i have unstable results for volano benchmark (in 1.4 i haven't). What can be the cause? Volano benchmark creates 100 threads and send messages via loopback (benchmark based on chat engine). Differences in performance about 20%

Message was edited by: vasluk

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Joined: 2005-07-11

What OS are you running on? What command line arguments are you using?
How many CPUs and how much memory does your machine have?

Joined: 2007-03-10

windows 2003 sp1 32 bit Enterprise edition,
java -Xms8m -Xmx64m COM.volano.Mark -count 5000 -rooms 10 -users 10
2 Dual Core Xeon 2,66 Ghz (HT was disabled) memory - 4 Gb

Joined: 2005-07-11

Since Volano is a server benchmark, I would highly recommend adding -server
to the command line. The JDK on Windows doesn't have the ergonomics support
that's available with the JDK on Solaris and Linux and it therefore defaults to a
-client runtime. This generally results in sub-optimal performance for server type
applications (and even for some client type applications, like large IDE apps).

In JDK 6, the compilation policy for -client, which is what you are running,
was changed from -Xbatch (-XX:-BackgroundCompilation) to -XX:+BackgroundCompilation.
This can have various performance effects that manifest as variability in results
from run to run particularly for short running benchmarks. This is usually a result
of timing differences introduced by scheduling variability and is more visiable
on multi-processor systems. Over time, these artifacts will generally wash out
for real applications. You could try adding -Xbatch to the command line to see
if this makes a difference.

I also think your heap sizes are rather small and if you insist on using such a
small heap, I would also add -XX:+UseSerialGC on the command line when
using -server (though I think +UseSerialGC would be the default anyway) to
make sure that you don't get multiple GC threads competing for such a small
amount of memory.

Finally, one big change that went into HotSpot JVM in JDK 6 is the Biased
Locking feature, which is on by default. This feature allows objects with uncontended
locks to be biased to a thread, thus giving that thread 'lock free' access to the
object. Sometimes, though, if the objects do get shared between multiple
threads the bias revocation costs can outweigh the advantage, particularly
for short running benchmarks. This revocation cost eventually is overcome in
real applications. You could try adding -XX:-UseBiasedLocking to the command
line to see if that helps.