# Is this a bug?

I took linuxhippy's code to play around - and found out this pecularity:

Original Code

-------------------

<br /> public static final bx = 0x4e35, cx = 0x015a;<br /> /* ... */<br /> /* within some method */<br /> /* ... */<br /> int ax, dx;<br /> ax *= bx;<br /> dx += cx;<br /> ax++;<br /> /* ... */<br /> /* ... */<br />

mean speed: (2.9578 Â± 0.0015) MB/s

Modified Code

-------------------

<br /> public static final bx = 0x4e35, cx = 0x015a;<br /> /* ... */<br /> /* within some method */<br /> /* ... */<br /> int ax, dx;<br /> ax *= bx;<br /> ax++; /* These 2 lines */<br /> dx += cx; /* have swapped */<br /> /* ... */<br /> /* ... */<br />

mean speed: (2.818 Â± 0.003) MB/s

The results can be consistently produced (with this tens of s.d. gap) so there might be some (trivial) optimisation case missing somewhere in HotSpot....

Or am I making sense here at all?

if I've done this correctly, using the formula provided:

t-stats = 0.1398 / 0.00106 = 131.8

(nu = 18)

which is even stronger.... :-/

(for tu-test, t-stats = 0.1398 / 0.001006 = 138.9, nu = 13.2)

Hmmm.. one set of algorithms for getting

to the T-Test (actually the tutest) is here:

(tutest = unequal variance t-test)

http://www.library.cornell.edu/nr/bookcpdf.html

Chapter 14.2, p. 617

HTH,

--Tom

Well the simplest T-Test is the one that assumes both samples are from the same population...

But ok a slightly more complicated calculation always gives a better sense of confidence :D

alexlamsl & linuxhippy:

I really appreciate the energy you have put

into this... We are kind of reaching the limitation

of the forum for facilitating this kind of collaboration.

I was going to suggest adding Wiki pages to:

http://wiki.java.net/bin/view/Projects/PerformanceCommunity

But it appears that wiki does not allow attaching files :-/

So maybe we need to have a subproject which would

allow CVS access for files and docs (although

docs *may* be easier on the wiki).

To take this to the next level we need to publish

the complete code samples in a way that they can

run in a harness. I'm thinking we can leverage

the Japex microbenchmark harness:

I haven't determined to what degree Japex (or another

harness) can help calculate basic stats

(std dev. is a must, T-Test would be great).

What does everyone think about this approach?

Regards,

--Tom

P.S. I'm swamped with about four fires at the moment

but this is *exactly* the kind of direction

we need to take the performance community.

I for one really appreciate your attention in turn; I can understand the potential frustration of being fed with (unintentional yet) misleading information plus perhaps unnecessary flames burning ;)

Japex looks good, although I'm not quite sure why it needs so many average measures but does not include any measures of spread (e.g. variance). May be it's just me overlooking things ^^"

I wouldn't mind writing a few lines of code post-processing the output information to generate these stats + perform t-tests though; being a physics undergraduate + done 3 maths GCE I should know my stuff....

Ok t-test....

total N = 20

pooled s.d. / sqrt(N) = 0.016

t-stats = 0.1398 / 0.016 = 8.9

so it's unlikely enough that we should reject null hypothesis, don't you think? :)