Skip to main content

Phoneme Advanced Questions

16 replies [Last post]
fivecat
Offline
Joined: 2007-02-07
Points: 0

Questions:
We found that the performance of java virtual machine(JVM) in phoneme(Debug vision) was slower fourty times than the the corresponding C language programme when they was run on the same Linux Operation System. And we found that the performance of Phoneme JVM was slower twenty times than the JVM which was provided by Linux too. Please ask me whether this situation is general.
if it is not general, what method can we use to enhance the performance of Phoneme JVM? Can we use the JIT technology? And please ask me how to use the JIT on Phoneme JVM.
I wish your answers! Thanks a lot!

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
fivecat
Offline
Joined: 2007-02-07
Points: 0

Hi, Mark:
3 Would you tell me that how times the difference of the performance between with CVM_JIT=true and with CVM_JIT=false in usual situations.

thanks a lot.

mlam
Offline
Joined: 2006-10-13
Points: 0

fivecat,

> Would you tell me that how times the difference of
> the performance between with CVM_JIT=true and with
> CVM_JIT=false in usual situations.

The difference between the JIT and interpreter performance depends on many factors:
- the type of work your application is doing
- the characteristics of the hardware you are running on e.g. CPU architecture, memory bandwidth, capacity, etc.

In my experience, on a reasonable piece of hardware, the CVM JIT can produce speed ups anywhere between 2x to 24x depending on what type of work is being done in your application. In most real world applications, the realistic speed up is somewhere between 5x to 10x. All these numbers are based on my educated guess. Your situation may vary.

Mark

Message was edited by: mlam

fivecat
Offline
Joined: 2007-02-07
Points: 0

Hi, Mark:
2 Further more, when we compile the phoneme_advanced-mr2-dev-src-b21-04_may_2007.zip with CVM_JIT=true, but the wrong was appearred. the wrong was following:

root@lwubuntu:/home/oem/middleware/phoneme_advanced_mr2/cdc/build/linux-x86-suse# make CVM_JIT=true
touch ../../build/linux-x86-suse/./generated/empty.mk
MAKEFLAGS = CVM_JIT=true
CVM_HOST = i686-Ubuntu-linux
CVM_TARGET = linux-x86-suse
SHELL = sh -e
HOST_CC = /usr/bin/cc
HOST_CCC = /usr/bin/g++
ZIP = /usr/bin/zip
FLEX =
BISON =
CVM_JAVA = /usr/bin/java
CVM_JAVAC = /usr/bin/javac
CVM_JAVAH = /usr/bin/javah
CVM_JAR = /usr/bin/jar
TARGET_CC = /usr/bin/cc
TARGET_CCC = /usr/bin/g++
TARGET_AS = /usr/bin/cc
TARGET_LD = /usr/bin/cc
TARGET_AR = /usr/bin/ar
TARGET_RANLIB = /usr/bin/ranlib
LINKFLAGS = -g -Wl,-export-dynamic
LINKLIBS = -lpthread -ldl -lm
ASM_FLAGS = -c -fno-common -march=i686 -traditional
CCCFLAGS = -fno-rtti
CCFLAGS_SPEED = -c -fno-common -Wall -fno-strict-aliasing -march=i686 -O4
CCFLAGS_SPACE = -c -fno-common -Wall -fno-strict-aliasing -march=i686 -O2
CCFLAGS_LOOP = -c -fno-common -Wall -fno-strict-aliasing -march=i686 -O4 -fno-inline
CCFLAGS_FDLIB = -c -fno-common -Wall -fno-strict-aliasing -march=i686 -O4 -ffloat-store
JAVAC_OPTIONS = -g:none -J-Xms32m -J-Xmx128m -encoding iso8859-1 -source 1.4 -target 1.4
CVM_DEFINES = -DCVM_OPTIMIZED -DCVM_DEBUG_STACKTRACES -DNDEBUG -DCVM_CLASSLOADING -DCVM_SERIALIZATION -DCVM_REFLECT -DCVM_DYNAMIC_LINKING -DCVM_JIT -DCVM_TIMESTAMPING -DJ2ME_CLASSLIB=cdc -DTARGET_CPU_FAMILY=x86 -D_GNU_SOURCE
host uname = Linux lwubuntu 2.6.15-26-686 #1 SMP PREEMPT Mon Jul 17 20:14:14 UTC 2006 i686 GNU/Linux
TARGET_CC version = 4.0.3 i486-linux-gnu
HOST_CC version = 4.0.3 i486-linux-gnu
CVM_JAVA version = java version 1.4.2_12
TOOLS_DIR = /home/oem/middleware/phoneme_advanced_mr2/tools
updating ../../build/linux-x86-suse/./generated/build_defs.mk ...
Checking for build-time classes to compile ...
Checking for phoneME Advanced classes to compile ...
Checking for test classes to compile ...
Checking for demo classes to compile ...
bison ../../build/linux-x86-suse/./jcs/tbl.cc
sh: bison: command not found
make: *** [../../build/linux-x86-suse/./jcs/tbl.cc.h] Error 127
root@lwubuntu:/home/oem/middleware/phoneme_advanced_mr2/cdc/build/linux-x86-suse#

Please help me analyse the reason. thanks a lot.

mlam
Offline
Joined: 2006-10-13
Points: 0

Fivecat,

> bison ../../build/linux-x86-suse/./jcs/tbl.cc
> sh: bison: command not found
> make: *** [../../build/linux-x86-suse/./jcs/tbl.cc.h]
> Error 127
> root@lwubuntu:/home/oem/middleware/phoneme_advanced_mr
> 2/cdc/build/linux-x86-suse#

The failure says that the bison tool isn't available on your command line. You'll need to install the bison tool. It is usually part of a linux distribution in the developer tools.

Mark

fivecat
Offline
Joined: 2007-02-07
Points: 0

Hi,Mark:
Thank you for your help.
1 We only wish test the performance of PhoneME Advanced VM because we find that the performance of phoneME JVM which we have transplanted on STB is very poor, and we didn't know what is wrong. For example, we wish display a picture(size:140kB, type:gif, 400*600pixels) to CRT. We loaded the picture to STB first. Then the picture was displayed on CRT used by the phoneME JVM which we have transplanted on STB. We fonud the time of displaying is 120 seconds, while that of the other STB products is very little(tens of seconds). We have testted the runtime of codes too, and we have found that the following codes consumed the most time.
"
Image img = Toolkit.getDefaultToolkit().createImage(buf);
MediaTracker tracker = new MediaTracker(this);
tracker.addImage(img, 0);
long begin,end;
begin = System.currentTimeMillis();
try {
tracker.waitForID(0); //note: this method consumed the most time.
tracker.removeImage(img);
} catch (InterruptedException e) {
e.printStackTrace();
tracker.removeImage(img);
img = null;
return false;
}
end = System.currentTimeMillis();
"
Please help me analyse the reason.

mlam
Offline
Joined: 2006-10-13
Points: 0

fivecat,

I'm sorry that you are not seeing the type of performance you would like from one STB. The cause of this could be anything from simply that the hardware is not as powerful, to performance of the software (the OS, the media libraries, the Java VM, middleware stacks, etc).

> Please help me analyse the reason.

Unfortunately, I cannot personally help you with the debugging / profiling work that you are asking for here because it is a very long and complex job, and is beyond the scope of what a forum like this is supposed to do. If you are interested in enlisting the help of some professional engineers from Sun through a contract with Sun's Engineering Services to help you with this, then I can get some contact information for you. Otherwise, you will need to do the analysis yourself. In general, the best we, as community members on the forum, can do is help provide some hints and advice to help you on your way. Of course, if there are other community members on the forum who would be willing to go the extra mile and take this on with you, they are welcomed to.

Regards,
Mark

Hinkmond Wong

Hi fivecat,

phonemeadvanced@mobileandembedded.org wrote:
>> Please help me analyse the reason.
>>
>
> Unfortunately, I cannot personally help you with the debugging / profiling work that you are asking for here because it is a very long and complex job, and is beyond the scope of what a forum like this is supposed to do. If you are interested in enlisting the help of some professional engineers from Sun through a contract with Sun's Engineering Services to help you with this, then I can get some contact information for you. Otherwise, you will need to do the analysis yourself. In general, the best we, as community members on the forum, can do is help provide some hints and advice to help you on your way. Of course, if there are other community members on the forum who would be willing to go the extra mile and take this on with you, they are welcomed to.
>

For more info on the Sun's Engineering Services that Mark talks about
above see:

http://www.sun.com/software/jpe/es/index.xml

And, for contact information see:

http://www.sun.com/contact/services_solutions.jsp

Thanks,
Hinkmond

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

fivecat
Offline
Joined: 2007-02-07
Points: 0

Note: For test easy, we have increased the loop times in C language.

5 When we ran the benchmark, the conditions is:
- it was on a quiet machine.
- there is not networking involved.
- the memory is 384MB to be allowed to use Linux OS .
- the all codes are coded by us.

Actually, we wish transplant the phoneME Advanced to OS20 Operation System in STB(Set Top BOX). the CPU on STB is STx5105 whose frequency is 200MHz and whose memory is 64MB DDR SDRAM(frequency:133MHz). But we found too that the performance of phoneME Advanced JVM on STB is slower 150 to 200 times than in my PC. We want to know whether this situation is general.
whether is this version phoneME Advanced JVM be fit for the platform?

fivecat
Offline
Joined: 2007-02-07
Points: 0

printf("plus_time=%d ",end-begin);
printf("done\n");
}

void main(void)
{
int a=1000000000;
computer_plus(a);
}

The java test codes are:

package java_test;
public class Test
{

public static void computer_plus(int plus)
{
int i;
int j=0;
System.out.println("start");
long begin,end;
begin = System.currentTimeMillis();
for(i=0;i {
j=i+j;
}
end = System.currentTimeMillis();
System.out.println("java:plus time="+(end-begin)+" done");
}

public static void main(String args[])
{
int a=1000000000;
computer_plus(a);
}
}

fivecat
Offline
Joined: 2007-02-07
Points: 0

Hi:
thank you very much!

We found that the performance of JVM in phoneme(Debug version) was slower one time than that of Release Version on linux Operation System.
For your questions, my answers are following:
1 the version of phoneME Advanced is MR2-B02;
2 We bulid phoneME Advanced on PC(Intel(r) Pentium(r)4 CPU 3.2GHz, 512MB memory) and my PC's Operation System is Windows XP. We ran a linux virtual machine named VMware on Windows XP, and we ran phoneME on the linux VM.
3 The build options are J2ME_CLASSLIB=basis, CVM_DEBUG=false, CVM_PRELOAD_LIB=true and the others is default.
4 The C test codes are:

#include
#include
void computer_plus(int plus)
{
int i;
int j=0;
time_t begin,end;
printf("start\n");
time(&begin);
for(i=0;i {
j=i+j;
}
j=0;
for(i=0;i {
j=i+j;
}
j=0;
for(i=0;i {
j=i+j;
}
j=0;
for(i=0;i {
j=i+j;
}
time(&end);
printf("plus_time=%d ",end-begin);
printf("done\n");
}
void computer_sub(int sub)
{
int i;
int j=0;
time_t begin,end;
printf("start\n");
time(&begin);
for(i=0;i {
j=i-j;
}
j=0;
for(i=0;i {
j=i-j;
}
j=0;
for(i=0;i {
j=i-j;
}
j=0;
for(i=0;i {
j=i-j;
}
time(&end);
printf("sub_time=%d ",end-begin);
printf("done\n");
}
void computer_mul(int mul)
{
int i;
int j=0;
time_t begin,end;
printf("start\n");
time(&begin);
for(i=0;i {
j=i*5;
}
j=0;
for(i=0;i {
j=i*5;
}
j=0;
for(i=0;i {
j=i*5;
}
j=0;
for(i=0;i {
j=i*5;
}
time(&end);
printf("mul_time=%d ",end-begin);
printf("done\n");
}
void computer_div(int div)
{
int i;
int j=17;
time_t begin,end;
printf("start\n");
time(&begin);
for(i=0;i {
j=i/17;
}
j=17;
for(i=0;i {
j=i/17;
}
j=17;
for(i=0;i {
j=i/17;
}
j=17;
for(i=0;i {
j=i/17;
}
time(&end);
printf("div_time=%d ",end-begin);
printf("done\n");
}
void main(void)
{
int a=1000000000;
int b=1000000000;
int c=234500000;
int d=1000000000;

computer_plus(a);
computer_sub(b);
computer_mul(c);
computer_div(d);
}

The java test codes are:

package java_test;
public class Test
{

public static void computer_plus(int plus)
{
int i;
int j=0;
System.out.println("start");
long begin,end;
begin = System.currentTimeMillis();
for(i=0;i {
j=i+j;
}
end = System.currentTimeMillis();
System.out.println("java:plus time="+(end-begin)+" done");
}
public static void computer_sub(int sub)
{
int i;
int j=0;
System.out.println("start");
long begin,end;
begin = System.currentTimeMillis();
for(i=0;i {
j=i-j;
}
end = System.currentTimeMillis();
System.out.println("java:sub time="+(end-begin)+" done");
}
public static void computer_mul(int mul)
{
int i;
int j=0;
System.out.println("start");
long begin,end;
begin = System.currentTimeMillis();
for(i=0;i {
j=i*5;
}
end = System.currentTimeMillis();
System.out.println("java:mul time="+(end-begin)+" done");
}
public static void computer_div(int div)
{
int i;
int j=17;
System.out.println("start");
long begin,end;
begin = System.currentTimeMillis();
for(i=0;i {
j=i/17;
}
end = System.currentTimeMillis();
System.out.println("java:div time="+(end-begin)+" done");
}
public static void main(String args[])
{
int a=1000000000;
int b=1000000000;
int c=234500000;
int d=1000000000;
computer_plus(a);
computer_sub(b);
computer_mul(c);
computer_div(d);
}
}

Note: For test easy, we have increased the loop times in C language.

5 When we ran the benchmark, the conditions is:
- it was on a quiet machine.
- there is not networking involved.
- the memory is 384MB to be allowed to use Linux OS .
- the all codes are coded by us.

Actually, we wish transplant the phoneME Advanced to OS20 Operation System in STB(Set Top BOX). the CPU on STB is STx5105 whose frequency is 200MHz and whose memory is 64MB DDR SDRAM(frequency:133MHz). But we found too that the performance of phoneME Advanced JVM on STB is slower 150 to 200 times than in my PC. We want to know whether this situation is general.
whether is this version phoneME Advanced JVM be fit for the platform?

mlam
Offline
Joined: 2006-10-13
Points: 0

fivecat,

I have a few responses for you. FYI, below, I call the phoneME Advanced VM by its other name, CVM (just so that I do less typing):

1. Which did you do: a debug build or a non-debug build? Here you say that you are comparing a debug build of CVM against presumably the the optimized version of JavaSE on linux x86:

> We found that the performance of JVM in phoneme(Debug
> version) was slower one time than that of Release
> Version on linux Operation System.

Here, you say that you are building a non-debug build of CVM:

> The build options are J2ME_CLASSLIB=basis,
> CVM_DEBUG=false, CVM_PRELOAD_LIB=true and the others
> is default.

So, which is it? Are you doing your benchmark with a debug build or not?

2. CVM's x86 port is not fully tuned and optimized yet. Hence, of course, you should expect the optimized JavaSE version to be faster here. CVM is normally targetted towards embedded devices with different characteristics than the x86 PC that you are running on. The ARM port for example is one that is tuned and optimized (for some configuration of the ARM).

3. You didn't build CVM with its JIT enabled. Hence, you are comparing an interpreter run on CVM (possibly in debug mode) against the optimized JavaSE VM with its JIT. Of course, you would expect JavaSE to be faster here too. Turn on the JIT with CVM_JIT=true and you will see that it will run faster, though JavaSE will still be faster.

4. Your benchmark code is a classic example of a bad benchmark for various reasons:

a. the gcc C compiler will detect that all the arithmetic you are doing in those loops are dead code, and just eliminate them. The net effect is that you have functions that measure the time of doing nothing.

b. real world applications (such as the set top box ones you are hoping to target) will not sit around in loops like these and do arithmetic that will result in dead code that gets eliminated. Hence, for real world applications, a C compiler will not be able to generate code that runs as fast as code that does nothing like in your benchmark example.

c. the JavaSE JIT will also optimize away some of this code, but will not do as complete a thorough job as the C compiler. The reason is not because we cannot make it optimize this kind of code. It is because there is no point in doing so. As mentioned above, real world applications don't do things like this. Hence, it would be a waste of footprint and code complexity to make the JIT optimize away code like this when they don't exists in real world applications.

d. CVM's JIT will do even less optimizations of this kind than JavaSE because CVM is targetted torwards embedded devices (like the set top box you are hoping to port to), and embedded devices are even more sensitive to footprint and CPU resources. Hence, it will not waste resources to optimize this kind of code that normally doesn't appear in applications.

For all the above reasons, your benchmark comparison is not meaningful.

5. You said:
> Actually, we wish transplant the phoneME Advanced to
> OS20 Operation System in STB(Set Top BOX). the CPU on
> STB is STx5105 whose frequency is 200MHz and whose
> memory is 64MB DDR SDRAM(frequency:133MHz). But we
> found too that the performance of phoneME Advanced
> JVM on STB is slower 150 to 200 times than in my PC.

> We bulid phoneME Advanced on PC(Intel(r)
> Pentium(r)4 CPU 3.2GHz, 512MB memory) and my PC's
> Operation System is Windows XP. We ran a linux
> virtual machine named VMware on Windows XP, and we
> ran phoneME on the linux VM.
> - the memory is 384MB to be allowed to use Linux OS

Let's compare:
a. STB runs a CPU at 200MHz. PC runs at 3.2GHz. Comparing CPU clock speeds, that's a difference of 16x.

b. PC has a front side bus @ 800MHz. STB bus speed is 133MHz. That's a difference of 6x.

c. ST20 has 2K I-cache, 2K D-cache, 2K SRAM. PC has an 8K L1 cache, 512K L2 cache and maybe 2M of L3 cache. Let's assume the L3 is not present. That's a rough difference of about 512K / 6K = ~85x.

NOTE: As an estimate, I'm using data about the Pentium 4 3.2GHz from this article: http://www.pcstats.com/articleview.cfm?articleID=808. I got the ST20 cache data by googling for "ST20 cache" and made an educated guess.

Without taking all other factors into consideration (e.g. IO access speed to load the Java VM code into memory, CPU architecture, instruction set, RAM capacity, RAM speed, etc), the above alone gives us a hardware performance difference of roughly 16 x 6 x 85 = 8160x to the advantage of the PC. So, why is it not realistic that you would expect CVM to run slower on the STB than on the PC by around 150 to 200 times?

Of course, my calculation is very simplistic, but it certainly illustrates the point. The real performance difference is more complex than that and based on too may variable to calculate in a straightforward formula like this. That's why you are seeing a performance difference of ~200x instead of 8160x.

6. You asked:
> We want to know whether this situation is general.
> whether is this version phoneME Advanced JVM be fit
> for the platform?

Whether CVM is fit or not for your platform depends on your needs. If you want to use it like a desktop to do desktop PC type work, I think you will be disappointed for obvious reasons. But if you intend to use it as an STB to do STB type processing, then I think it will probably be a good fit.

Mark

Johannes Eickhold

Nice benchmark bashing :-).

Ok - we learned (if we didn't knew before anyway): proper benchmarking
is a difficult task.

What kind of (synthetic) benchmarks do you suggest to test the CVM?

Jonek

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Hinkmond Wong

Johannes Eickhold wrote:
> Nice benchmark bashing :-).
>
> Ok - we learned (if we didn't knew before anyway): proper benchmarking
> is a difficult task.
>
> What kind of (synthetic) benchmarks do you suggest to test the CVM?
>
>

EEMBC is a good one.

Hinkmond

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Terrence Barr - Evangelist, Java Mobile & Embedded

Yes, EEMBC is what we have been using for years now. You can get the
MIDP version at:

www.grinderbench.com

You can also use the same jar file to run GrinderBench from the command
line in CDC. I need to dig up the command line parameters.

-- Terrence

Hinkmond Wong wrote:
> Johannes Eickhold wrote:
>> Nice benchmark bashing :-).
>>
>> Ok - we learned (if we didn't knew before anyway): proper benchmarking
>> is a difficult task.
>>
>> What kind of (synthetic) benchmarks do you suggest to test the CVM?
>>
>>
>
> EEMBC is a good one.
>
>
> Hinkmond
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
> For additional commands, e-mail: advanced-help@phoneme.dev.java.net
>
>
[terrence.barr.vcf]
---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Hinkmond Wong

Terrence Barr - Evangelist, Java Mobile & Embedded wrote:
> Yes, EEMBC is what we have been using for years now. You can get the
> MIDP version at:
>
> www.grinderbench.com

Thanks for the link, Terrence! :-)

Hinkmond

> You can also use the same jar file to run GrinderBench from the command
> line in CDC. I need to dig up the command line parameters.
>
> -- Terrence
>
> Hinkmond Wong wrote:
>> Johannes Eickhold wrote:
>>> Nice benchmark bashing :-).
>>>
>>> Ok - we learned (if we didn't knew before anyway): proper benchmarking
>>> is a difficult task.
>>>
>>> What kind of (synthetic) benchmarks do you suggest to test the CVM?
>>>
>>>
>>
>> EEMBC is a good one.
>>
>>
>> Hinkmond
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
>> For additional commands, e-mail: advanced-help@phoneme.dev.java.net
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
> For additional commands, e-mail: advanced-help@phoneme.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

mlam
Offline
Joined: 2006-10-13
Points: 0

fivecat,

For starters, it is not appropriate to run benchmarks using a debug build. Debug builds do a lot of extra stuff to facilitate debugging. This will cause the performance to degrade significantly. Performance should be measured on an optimized production build (i.e. non-debug) because that is what will be deployed in the real world. Debug builds are only use for development.

Apart from that, please answer a few questions to clarify things:
1. What version of phoneME Advanced are you using?
2. What target platform did you build phoneME Advanced for?
3. What build options did you use?
4. What benchmark code did you use as the basis of your comparison?
5. Please describe the conditions of how you ran the benchmark. For example:
- was it on a quiet machine?
- is there networking involved?
- how long does the benchmark runs for?
- what does the benchmark measure?
- if relevant, how much memory is the VM and your C equivalent allowed to use?
- what is the amount of free memory availability on your system?
- how did you get the C equivalent benchmark?

I may have more questions depending on your answers. I will need to know more info on what you actually did before I can tell whether the performance difference you saw is real, or due to a mistake.

Regards,
Mark