Skip to main content

VM speed emulation in WTK

7 replies [Last post]
davjoh
Offline
Joined: 2007-07-18
Points: 0

I've been looking at the WTK's Performance preferences, and I see that it is possible to vary the VM speed emulation between 100 up to 1000 bytecodes/millisecond.

Question is, how does this relate to real-world performance on a phone?

Is there a way of benchmarking a phone for VM speed? If not, then what's the purpose of varying the VM speed emulation in the WTK? And if so, it would be nice to know how so varying the settings in the emulation evironment could have some meaning.

-David

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
vedanuzal
Offline
Joined: 2006-11-14
Points: 0

Check out Jbenchmark http://www.jbenchmark.com/index.jsp
This has performance stats for phones and you can download and test phones you self. Or just write a simple program that does it yourself.
I haven't tried it but i'm guessing that by varying the number of bytecodes/millisecond you should be able to get a speed approximating that of the phone.

davjoh
Offline
Joined: 2007-07-18
Points: 0

Thanks for the link - jbenchmark seems to have a database of a lot of phones that they have benchmarked (pity it's a pay service for the info though!), but looking at the sample reports, I'm still stuck with what I want to do.

Here's the problem. The jbenchmark reports on performance of the JVM do not seem to have any descriptions of what the stats actually are - there's lots of rank comparison with other phones, and unlabeled graphs... I'm sure I could decipher a report eventually, but at first glance it doesn't look like it has the info I want.

So for what I want to do, lets say I want to set up an emulator that might roughly perform like a N95, I want to know what kind of bytecodes/millisecond to set up the emulator to. Likewise, if I want to emulate a slower phone, I want to figure out the degree of which to slow it down in the emulator.

davjoh
Offline
Joined: 2007-07-18
Points: 0

Correction on my last email - I see that one can actually download the benchmarks for free to test themselves... but once again, getting some meaning out of them is still a challenge.

I was thinking about this, and I guess I could benchmark the WTK emulator on different senttings, then benchmark 'real' devices to compare the two.

I'm guessing jbenchmark isn't open-source, so exactly how they run their tests is unknown to me as a developer.

I'm wondering, would implementing something like Whetstone/Dhrystone and even something like Linpack would be useful for Java ME? Imagine that: a Top500 of list of mobile phones...

vedanuzal
Offline
Joined: 2006-11-14
Points: 0

After thinking about what you want I don't think you will find an easy way of getting the information you want unless you can get it from the manufacturer of the phone.
JBenchmark test the speed at which apps run. What you want is how fast the jvm does JIT compilation which is not exactly what jbenchmark tests.

So your last suggestion is probably the best one.

davjoh
Offline
Joined: 2007-07-18
Points: 0

So...

I ported Linpack and Sieve benchmarks to see how the WTK (with VM speed restricted) performs against my N95 and here's a couple of results...

N95:
Linpack = 1.9 mflops, Sieve = 466 iterations/second

WTK set @ 1000 bytecodes/s (max speed in restricted mode):
Linpack = 0.097 mflops, Sieve = 2.84 iterations/second

So basically I can't set up the WTK to perform like a modern Smartphone in a controlled way. I'm guessing that the bytecodes/s setting was devised quite a while ago when phones were slower, hence the rather slow maxima.

Does anybody have any idea how to work around this in the WTK? Or is the WTK open-sourced at all?

vedanuzal
Offline
Joined: 2006-11-14
Points: 0

You should only need to test the application that you have developed in a speed restricted mode for these lower speeds. For the newer phones like the N95 any application you develop should not have any significant dependence on the performance of the phone except difference's in the applications responsiveness.
It will only be on the slower phones that you will need to analyze the performance so you shouldn't need a faster max speed in restricted mode. Just test that whatever application you develop runs well at both speeds.

davjoh
Offline
Joined: 2007-07-18
Points: 0

Thanks - I get what you mean, and in fact I ran the benchmarks in the WTK in an unrestricted mode, and curiously enough it wouldn't perform as well as the N95 anyway!

My main purpose of this wasn't just to test a single standalone application, but rather to create a testbed consisting of multiple WTK emulators running some P2P software. I don't have any other testbed at my disposal, and only a handful of real phones to create some test. So I wanted to use the WTK to control some of the environment's factors, hence controlling the VM restriction as a way to control the execution speeds across the test environment.

Thank again for your help!

-David