Skip to main content

FLOATS bug, duplicated code

15 replies [Last post]
manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Hello guys,

As some might know we're trying to get squawk running into a bluetooth chip (CSR brand, based on the XAP2 ASIC from Cambridge Consultants).

Thing is that this processor doesn't support Floating Point numbers, so the compiler rejects any compilation related to this kind of numbers, so I modified my build.properties to set FLOATS=false (among a lot of other stuff) and I started getting compilation errors.

I noticed that there were some missing: /*if[FLOATS]*/.... /*end[FLOATS]*/ on the code, so I started patching, thing is that I find that there are files duplicated everywhere. For example PrintStream is both in phoneme/cldc/src/javaapi/.... and in cldc/src/.... which one is used? Why there's so many duplicated code?

I'm going to commit a patch regarding this [FLOATS] thing today, but would like to make it as clean as possible, so I would like to patch only the needed files.

Thanks a lot,
Manuel Naranjo

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Hello guys,

Well patching was really easy, just added some if[FLOATS] checks, and now I can compile the java code.

I uploaded the patch to the tracker section, Issue ID is 1365, link is here: https://squawk.dev.java.net/issues/show_bug.cgi?id=1365

Thanks,
Naranjo Manuel Francisco
manuel at aircable dot net
www.aircable.net

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

Ok, so this was my bad when I was working on getting the code ready for open source. The issue came from the fact that I removed all of our own core code for CLDC and used the phoneME source. I did not test all of our optional compile flags.

What is on my todo list is to work with the phoneME guys to get this integrated into their source tree so we can leverage it. So I am not sure what I can do about the patch you submitted.

The issue here is that what we are trying as much as possible NOT to duplicate code where we can. It is much easier for us for dealing with TCK and Java certification if we can use the same code as the phoneME guys. I've put this on the backburner on my end, but it seems that a number of you are running into this issue, so maybe I will move it to the forefront.

Here is one way you could integrate this into your build system. There is an "install" step where you need to do a copyphoneme. At the moment, this takes code from the phoneme module and copies it into cldc and imp modules. The phoneme module is intended to disappear and rather use the code directly from the phoneme source tree. I would suggest that as part of your build process, do the copyphoneme, then apply your patch.

I am thinking as I write this, maybe I could make the copyphoneme task apply a patch if it finds a file called phoneme.patch. This way we could all use this technique until I get a chance to convince the phoneme project owners to take some of our changes.

This would also REALLY help the guys doing the port to Lego NXT, they could go back to using the Squawk source repo instead of cloning the whole tree into their repo.

What do you guys think ? Did I explain the problem well enough :)

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

> Ok, so this was my bad when I was working on getting
> the code ready for open source. The issue came from
> the fact that I removed all of our own core code for
> CLDC and used the phoneME source. I did not test all
> of our optional compile flags

Ouu, damn why me!!!! hehe.

> What is on my todo list is to work with the phoneME
> guys to get this integrated into their source tree so
> we can leverage it. So I am not sure what I can do
> about the patch you submitted.
I can get in touch with them, but I'm not sure if the guys from phoneME have an scenario of devices without FLOATS.

> The issue here is that what we are trying as much as
> possible NOT to duplicate code where we can. It is
> much easier for us for dealing with TCK and Java
> certification if we can use the same code as the
> phoneME guys. I've put this on the backburner on my
> end, but it seems that a number of you are running
> into this issue, so maybe I will move it to the
> forefront.
Indeed duplicating code makes no sense at all :D, let's not reinvent the wheel, it works well as it is all ready.

> I am thinking as I write this, maybe I could make the
> copyphoneme task apply a patch if it finds a file
> called phoneme.patch. This way we could all use this
> technique until I get a chance to convince the
> phoneme project owners to take some of our changes.
Ok it makes sense.

> What do you guys think ? Did I explain the problem
> well enough :
Makes sense. You explained the problem pretty well.....

BUT I'm still with my FP problems :S. I'm patching the code as I write this mails, and bugs are not only from phoneME, guess it's related to the code been open sourced. So far I had patched about 34 files, and still can't get the build process to end :S.

Once I got d to end, rom cldc failed telling we aren't using FP which makes sense. So then I made changes to lots of classes including CID, I marked CID to not define FLOAT and DOUBLE when we aren't in FP mode, but this seems to make things ever worst :S

derek_white
Offline
Joined: 2006-09-08
Points: 0

Manuel, thanks for finding this. We should be able to compile without floating point support.

The main reason for doing this is to reduce the size of the system.

Some systems (I don't kow about yours), may want to use the same software floating point that we use on the ARM 9 - the Sun SPOT doesn't support floating point in hardware either. We use the software libraries in newlib plus some Java wrappers that provide strict fp semantics.

34 files is a lot to patch. We could also think about getting the translator to skip methods that use floats or doubles are parameters or return values. I can look monday and see if there is any pattern to this FP code....

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

> Manuel, thanks for finding this. We should be able to
> compile without floating point support.
Cool

> The main reason for doing this is to reduce the size
> of the system.
>
> Some systems (I don't kow about yours), may want to
> use the same software floating point that we use on
> the ARM 9 - the Sun SPOT doesn't support floating
> point in hardware either. We use the software
> libraries in newlib plus some Java wrappers that
> provide strict fp semantics.
We don't have FP in hardware, and we don't want to waste processor power to make fp calculations, I have to take a look at the newlib code, but I don't think it makes much sense for us to use FP on the processor, we will just waste flash memory for this.

I will take a look at the SPOT port as soon as I can.

> 34 files is a lot to patch. We could also think about
> getting the translator to skip methods that use
> floats or doubles are parameters or return values. I
> can look monday and see if there is any pattern to
> this FP code....
I started by modifying String (valueOf for double and float makes no sense). Then things started, and started patching files. And then I reached the big number of 34 :S.

One of the things I did, and broken a lot of stuff was removing FLOAT, DOUBLE, FLOAT_ARRAY and DOUBLE_ARRAY from both CID and Klass. But this broke the hole stuff, including the translator.

It could help a lot if some of you can take a look, I'm modifying the code in blind mode as I don't know how this works at all. I'm learning the process, but I'm still missing stuff.

Thanks,
Manuel

derek_white
Offline
Joined: 2006-09-08
Points: 0

OK, I'm still not sure zbout the best way to fix the problem, but I was able to push through a build by patching the following phoneme files (using the obvious conditional compilation):

FloatingDecimal
Random
Long
Integer
DataOutput
DataInput
PrintStream

Plus several Squawk cldc files:
VM
RawMemoryFloatAccess
MathUtils
Math

In debugger (optional):
TestApp

So I don't think you need to mess with CID or Klass. This will waste a tiny bit of space for classes that won;t be used, but it shouldn't hurt the build.

After making the changes, be sure to do a clean before a build. I got burned a few times that way...

I'll try to get the squawk/cldc changes checked in soon...

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Thanks that really helps, now I can pass the hole squawk build process, even rom cldc works.

I will see if the C compiler likes the code now.

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Oops, sorry it didn't worked, I forgot to edit the build.properties after getting the latest svn trunk.

Maybe I'm missing something I'm doing:
JAVA_HOME=/usr/java/jdk1.5.0_15_i386/
export JAVA_HOME
./d clean
cd builder
./bld.sh $JAVA_HOME
cd ..
./d copyphoneme
./d -verbose

The last lines fails to build cldc, I get errors in:
cldc/java/io/PrintStream.java
cldc/java/io/DataOutputStream.java
cldc/java/lang/FloatingDecimal.java
cldc/com/sun/squawk/io/j2me/channel/ChannelOutputStream.java
cldc/com/sun/squawk/io/j2me/channel/ChannelInputStream.java

I all ready have a patch for this files that fixes the compilation issues with them, I'm going to apply and see if I can get the rest of the process to continue.

Thanks,
Manuel

derek_white
Offline
Joined: 2006-09-08
Points: 0

Yes, you still need your patch to the phoneme srcs.

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Oops, now it worked, thanks.

I'm going to test the compiler now then.

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Yippie the compiler liked the code.

We will now implement the os.c and all the native parts needed, and keep you up to date.

BTW there's still a missing bug related to FP, I'll check this later, even though FLOATS=false rom cldc tried compiling squawk/vmcore/src/vm/fp

I now can't wait until I get Java running on our bluetooth chip

Cheers,
Manuel

derek_white
Offline
Joined: 2006-09-08
Points: 0

[From Manuel Naranjo, in another thead]

The guide is perfect, we aren't porting experts either, so we are going to learn together :D.

Ok so let's keep it in this thread then.

This is the stack trace I'm getting:
java.lang.RuntimeException: Klass [B was not serialied
at com.sun.squawk.NativeUnsafe.resolveClasses(NativeUnsafe.java:717)
at com.sun.squawk.ObjectGraphSerializer.serialize(ObjectGraphSerializer.java:113)
at com.sun.squawk.VM.copyObjectGraph(VM.java:686)
at com.sun.squawk.Suite.save(Suite.java:776)
at com.sun.squawk.Romizer.createImage(Romizer.java:774)
at com.sun.squawk.Romizer.access$100(Romizer.java:44)
at com.sun.squawk.Romizer$2.run(Romizer.java:393)
at com.sun.squawk.util.ComputationTimer.execute(ComputationTimer.java:127)
at com.sun.squawk.util.ComputationTimer.time(ComputationTimer.java:177)
at com.sun.squawk.Romizer.run(Romizer.java:391)
... 20 more

Unfortunately it doesn't make much sense to me, as I don't know how the java -> C code thing works.

This are my settings:
SQUAWK=true
EXCLUDE=false
KERNEL_SQUAWK=false
SQUAWK_64=false
REVERSE_PARAMETERS=true
TRACING_ENABLED=false
ASSERTIONS_ENABLED=false
DEBUG_CODE_ENABLED=false
SUITE_VERIFIER=false
J2ME.STATS=true
MACROIZE=false
GC=com.sun.squawk.Lisp2GenerationalCollector
NATIVE_GC_ONLY=true
SMARTMONITORS=true
INCLUDE_EXECUTECIO_PARMS=false
FLOATS=false
NATIVE_VERIFICATION=true
NATIVE_VERIFICATION_ONLY=true
CLDC1.1=true
JDK1.0=false
JAVA_SE=false
FULL_SLOT_CLEARING_ANALYSIS=false
TRUST_SLOT_CLEARING=true
CHECK_SLOT_CLEARING=false
BUFFERCHANNELINPUT=false
J2ME.HEAP_TRACE=false
REUSEABLE_MESSAGES=false
TYPEMAP=false
FLASH_MEMORY=true
ENABLE_DYNAMIC_CLASSLOADING=false
TRUSTED=false
VM2C=false
FINALIZATION=true
OLD_IIC_MESSAGES=false
FAST_INVOKEINTERFACE=true
RESOURCE.CONNECTION=false
VERBOSE_EXCEPTIONS=false
ENABLE_SDA_DEBUGGER=false
REAL_TIME=false
REAL_TIME_MINI=true
REAL_TIME_PROTO=true
RTSJ1.0=false

Can you check if you can get cldc rom to finish with this settings?

Cheers,
Manuel

derek_white
Offline
Joined: 2006-09-08
Points: 0

OK, I tried a compile using your settings, and patches to the phoneme sources, and it seems to compile fine.

One strange point is that this is still dying in the suite creation process, before the C compiler is called.

Can you post all the output past "[running romize...]"? Thanks!

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

I had fixed all this, there were some missing files that needed to patch. I'm going to update the patch soon.

Is there any way someone (and admin obviously) can delete my post to the other thread? I think I shouldn't have replied into that post sorry.

For the record until I send the new patch, this are my modified files, I have to review the patch and commit (there was a bug when CLCD1.1=false too, and I want to have the patches splited):
M translator/src/com/sun/squawk/translator/ir/verifier/VerifierBase.java
D vmcore/src/vm/cio.c
A vmcore/src/vm/cio.c.spp
M vmcore/src/vm/squawk.c.spp
M vmcore/src/vm/suite.c
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/StringBuffer.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/String.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/FloatingDecimal.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/Integer.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/Long.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/Float.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/Double.java
M phoneme/cldc/src/javaapi/cldc1.1/java/lang/Math.java
M phoneme/cldc/src/javaapi/cldc1.1/java/io/PrintStream.java
M phoneme/cldc/src/javaapi/cldc1.1/java/io/DataInput.java
M phoneme/cldc/src/javaapi/cldc1.1/java/io/DataOutput.java
M phoneme/cldc/src/javaapi/cldc1.1/java/util/Random.java
M cldc/src/com/sun/squawk/vm/ChannelConstants.java
M cldc/src/com/sun/squawk/VM.java
M cldc/src/com/sun/squawk/Suite.java

manuelnaranjo
Offline
Joined: 2006-06-26
Points: 0

Here's the patch in case anyone is interested:
https://squawk.dev.java.net/nonav/issues/showattachment.cgi/8/floats-2.p...

Before commiting the patch I forgot to make svn update, and thus patch -p0 < floats-2.patch fails to patch VM.java. But it doesn't make any difference as the change I did with my patch is all ready there.

Some might asking why wouldn't I see this patch before, quite easy (well not really, took me some time to figure out, and I did by mistake :S ). The translator found the bug, but then instead of launching an exception and make the compilation process end, it continued working.

Here's what it showed up:
[translating suite squawk [closed: false, parent: null] ...]
VerifyError: java.lang.Integer
Loaded suite stripping settings from squawk.library.properties
excluding: com.sun.squawk.compiler.*
excluding: com.sun.squawk.os.*

VerifyError was the clue. java.lang.Integer has some functions that gives floats values out of an integer, and off course that shouldn't be included in a non fp compilation. Thus I fixed the integer class by adding the /*if[FLOATS]*/.../*end[FLOATS]*/ thing, and started the compilation again, until I found all the classes with problems.

Now it compiles, and even gcc likes the new code :D. I got a 70KB of VM, and about 250KB of java classes, question: how can I see how much memory is used on each part?.

I'm starting to get the way the toolchain works, I really like, I'm adding the classes needed to get our provider gcc to work (they don't provided us with linux binaries, only win binaries, and thus we need to use wine, I'm going to extend GccCompiler.java.

Another question, and this one is related to legal stuff, in the headers it says I can't remove or modify the license header, I know that I can't remove it because it's GPL which is perfect fine, but can't I add my name to the classes I create as I'm one of the persons that wrote it. I'm not a lawyer and I hate all this legal crap, that's why I'm asking. I can leave with sun been the owner of my files, but would like to see my name (and I'm not talking from the company point of view, but from my personal point of view) reflected some where....

Cheers,
Manuel