Skip to main content

Building squawk

17 replies [Last post]
yroussel
Offline
Joined: 2008-02-06
Points: 0

Hi,

I checked out Squawk yesterday and spent a little time trying to build the project... for now I failed.
Is there some kind of readme that could help me or at least a description of the preferred environment ? Is this forum a place where I can post some precise questions about the build and the mechanism of the project ? If not, does such space exists ?

Thanks

PS I'm sorry to post here if this kind of message is not expected on this forum

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
rasped
Offline
Joined: 2008-01-31
Points: 0

[b]Squawk Build on Windows[/b]
[b]Installing[/b]
a. Install TortoiseSVN from http://tortoisesvn.tigris.org/
b. Remove old JDK and JREs. Install a Java JDK either from your SunSpot CD (D:\.jdks\jdk-1_5_0_07-windows-i586-p.exe) or something like JDK 5.0 Update 14 (http://java.sun.com/javase/downloads/index_jdk5.jsp). Do not let the JDK install in the default location (C:\Program Files\Java\jdk1.5.0_07), but instead install it in [i]C:\Java\jdk1.5.0_07[/i]. This has to do with spaces in files names. Now you have to skip the next step in the JDK installer, which is the JRE installation, because it sets JAVAHOME (in the registry) to point to the JRE, but we want it to point to the JDK. This is because we later need [i]tools.jar[/i] on the classpath. So press [i]Cancel[/i] and go on to press [i]Finish[/i].
c. Install Visual C++ 2008 Express (http://www.microsoft.com/express/vc/). It includes the needed SDKs, so only one download is needed. Eric's (see previous post) downloads works as well. If you open a CMD prompt after the install and write [i]set[/i] then this environment variable must appear: [i]VS90COMNTOOLS=C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools\[/i]. If not, re-install...

[b]Preparing[/b]
a. Get squawk by entering [i]https://squawk.dev.java.net/svn/squawk/trunk[/i] in the URL box in the SVN window that appears when right-clicking on a folder that you want to be squawk SVN root.
b. Append to your Path environment variable (My Computer->View System Information->Advanced->Environment Variables) the path [i]C:\Java\jdk1.5.0_07\bin[/i] (pointing to java.exe and javac.exe)
c. Create (or edit) the environment variable JAVA_HOME to to be [i]C:\Java\jdk1.5.0_07[/i]. It is needed when squawk.exe runs.
d. Replace line 750 (see previous posts) in com.sun.squawk.builder.Builder with [i]toolsJarPath = toolsJarURL.getPath();[/i] (see post above)
e. Edit the two lines (line 36 and 40) in bld.bat that starts with [i]javac[/i] to start with [i]javac -cp %JAVA_HOME%\lib\tools.jar -target 1.5 -source 1.5
[/i] and just leave the rest of the lines unchanged.
f. Paste these lines right after line 88 (String vsCommonToolsDirectory = System.getenv("VS80COMNTOOLS")) in /builder/src/com/sun/squawk/builder/ccompiler/MscCompiler.java line :
[i] // Check for newer version
if(vsCommonToolsDirectory == null) {
vsCommonToolsDirectory = System.getenv("VS90COMNTOOLS");
}[/i]
This way the Microsoft c compiler [i]cl.exe[/i] for the new Visual C++ 9 edition is found by the squawk build system.
g. Edit the file [i]d.bat[/i] in the squawk root directory. You need to replace line 50 with [i]goto gatherArgs[/i]. Otherwise it will not take more than one argument.

[b]Building/Running[/b]
Open a CMD prompt and change directory to the root of your squawk SVN repository. The file d.bat should be in this directory. Now you need to run a series of build commands to get to the point where squawk.exe runs on the host. Enter each of the following commands (the text in parenthesis are comments):
a. [i]d.bat clean[/i] (to clean up previous code)
b. [i]d.bat copyphoneme[/i] (to copy the cldc java files to the cldc folder)
c. [i]d.bat builder[/i] (to build the builder)
d. [i]d.bat[/i] (to build the rest)
e. [i]d.bat rom cldc[/i] (to build the VM squawk.exe)
f. [i]squawk.exe com.sun.squawk.Test[/i] (it takes about 60-90 seconds to complete. Do not get scared about the nullpointer exception: it is part of the test).

Best, Rasmus

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

This is great all. I will work on getting the instructions and integrating some of these comments. I am currently on vacation snow-boarding in Colorado so a little slow to respond :)

The build process has been a lot more solid in the past, but with this open source release I had to make a number of changes that seems to have caused some issues.

Keep the comments coming, we will do our best to react.

rasped
Offline
Joined: 2008-01-31
Points: 0

Hi Eric,
It was nice to talk to you in Menlo Park. I played with the options when creating the suite and if we replace the command in step (e) in the "Building/Running" (see above) with
[i]d -nocomp rom -strip:a cldc[/i]
then the suite goes down by about 1/3 in size and ends around 210 KB.
If the source is modified so CLDC and FLOAT can be set to false then I think it may fit "out of the box" in around 256 KB including the firmware (vmcore).
Best, Rasmus
PS: The options (like -strip:a) are described in /Squawk-translator/src/com/sun/squawk/SuiteCreator.java
PSS: A wiki will/would be helpful at some point (as we discussed:-))

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

Working on the wiki angle, but it will be a little time. Got some deliverables to get out first :)

Indeed it was really cool to hook up and spend time to go through a high level discussion on this stuff. We should have recorded to share with the others :)

As for the flags, you are correct, but there are some others ones we will be able to use that will get you even smaller. But I am glad you found some now that are helping :)

One thought I had was, if our cldc 1.0 vm was 80K, why is yours 55K ? You using thumb ? The size difference makes me worry that something key is missing from the VM ?

rasped
Offline
Joined: 2008-01-31
Points: 0

It is most likely that something is missing:-|:-)

I am working on trans scripting the handwritten notes from some of things you said. Including your useful Eclipse multiproject URL. I will put it up in less than a week or so. There is another deliverable with a very fixed deadline coming up:-)

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

I forgot to mention, I have integrated some of the changes that were suggested here with some further enhancements. There are still a couple of issues but I a working them out.

dennisgg
Offline
Joined: 2005-02-11
Points: 0

I'm building the project on a Mac OSX, as a pointer for others:

After I did a svn checkout, running 'd', 'd rom cldc' got me into all kinds of
NoClassDefFoundErrors. I got my checkout working by executing:

'./d copyphoneme' first.

rasped
Offline
Joined: 2008-01-31
Points: 0

hi,
Nice. That is correct you need to do copyphoneme. It relates to step (b) in "Building/Running" in this post:
http://forums.java.net/jive/message.jspa?messageID=258473#258473
If you find other things that can be documented in the forum, pls. post. I think is working on changing the copyphoneme thing.
Best, Rasmus

rogerd_dev
Offline
Joined: 2007-03-21
Points: 0

Running on Suse I get this build error after update to rev 59 (rev 52, rev 56 also...)
Is there anything in my environment or Java setup that is incorrect?

At this point I have NO .class files anywhere in the tree, certainly not Build.class.
(Which jar file is Build in?)

Thanks!

Seems to hit this in Launcher.java:
try {
Class buildClass = loader.loadClass("com.sun.squawk.builder.Build");
Method mainMethod = buildClass.getMethod("main", String[].class);
mainMethod.invoke(null, (Object) args);

rogerd@cat:~/jdev/squawk> ./d clean
Launcher: Found tools.jar in /usr/lib/jvm/java-1.5.0_11/lib/tools.jar, by popping up a level from jre.
Exception in thread "main" java.lang.RuntimeException: Problems finding builder
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:60)
Caused by: java.lang.ClassNotFoundException: com.sun.squawk.builder.Build
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:56)

rogerd@cat:~/jdev/squawk> ./d copyphoneme
Launcher: Found tools.jar in /usr/lib/jvm/java-1.5.0_11/lib/tools.jar, by popping up a level from jre.
Exception in thread "main" java.lang.RuntimeException: Problems finding builder
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:60)
Caused by: java.lang.ClassNotFoundException: com.sun.squawk.builder.Build
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:56)

rogerd@cat:~/jdev/squawk> ./d
Launcher: Found tools.jar in /usr/lib/jvm/java-1.5.0_11/lib/tools.jar, by popping up a level from jre.
Exception in thread "main" java.lang.RuntimeException: Problems finding builder
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:60)
Caused by: java.lang.ClassNotFoundException: com.sun.squawk.builder.Build
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:56)

rogerd_dev
Offline
Joined: 2007-03-21
Points: 0

Did an zip test on build.jar files and they were both empty! I remembered a conflict during svn update to rev 56, and resolved them, badly I guess.

Removed the SVN tree and did a full checkout of rev 64. Got much further!

./d copyphoneme
./d
Launcher: Found tools.jar in /usr/lib/jvm/java-1.5.0_11/lib/tools.jar, by popping up a level from jre.
Builder.JDK: Looking for JDK in /usr/lib/jvm/java-1.5.0_11, popped up a level from jre
Launcher: Found tools.jar in /usr/lib/jvm/java-1.5.0_11/lib/tools.jar, by popping up a level from jre.
For vm2c tools.jar=/usr/lib/jvm/java-1.5.0_11/lib/tools.jar
[building cldc...]
[building debugger...]
[building translator...]
[building hosted-support...]
Note: hosted-support/preprocessed/com/sun/squawk/vm/GUIOutputChannel.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[running squawk.jar...]
@hosted-support/squawk.jar.filelist: No such file or directory
Error adding @hosted-support/squawk.jar.filelist to jar archive!
build failed: Process.exec(jar cf squawk.jar @hosted-support/squawk.jar.filelist) returned 1

(Is the @ really part of the command?)

dennisgg
Offline
Joined: 2005-02-11
Points: 0

hi roger,
i noticed the following in your output.

-->build failed: Process.exec(jar cf squawk.jar @hosted-support/squawk.jar.filelist) returned 1
could it be that the 'jar' executable is not in your path ?

rogerd_dev
Offline
Joined: 2007-03-21
Points: 0

Yes, jar pointed to way old version of jar.

Point of reference, my trusty Suse 9.3 machine is not suitable for Squawk. JDK is OK, gcc is downlevel.

Everything works with my OpenSuse 10.2 machine.

paolosalvi
Offline
Joined: 2004-06-09
Points: 0

My goal is a squawk running on ARM9 and I should have many questions about (ie : is u-boot suitable to load squawk, etc.).
Right now I would really glad to have a working environment to develop with on my PC.
Using rev. 59 everything works with -nocomp, but trying to get a working squawk I'm not able to get over this error :

E:\Squawk\trunk>d rom cldc
Launcher: Found tools.jar in D:\jdk1.5.0_14\lib\tools.jar, by popping up a level
from jre.
Builder.JDK: Looking for JDK in D:\jdk1.5.0_14, popped up a level from jre
Launcher: Found tools.jar in D:\jdk1.5.0_14\lib\tools.jar, by popping up a level
from jre.
For vm2c tools.jar=D:\jdk1.5.0_14\lib\tools.jar
[running rom...]
[running romize...]
Loaded suite stripping settings from squawk.library.properties
excluding: com.sun.squawk.compiler.*
excluding: com.sun.squawk.os.*
excluding: com.sun.squawk.BytecodeTracer
excluding: com.sun.squawk.CheneyCollector*
excluding: com.sun.squawk.Lisp2Collector*
excluding: com.sun.cldc.io.connections.HttpConnection
excluding: com.sun.squawk.ClassTunnel
[translating suite squawk [closed: false, parent: null] ...]
E:\Squawk\trunk\vmcore\src\vm\rom.h is already up to date
Romizer processed 374 classes and generated these files:
E:\Squawk\trunk\squawk.sym
E:\Squawk\trunk\squawk.suite
E:\Squawk\trunk\squawk_classes.jar
E:\Squawk\trunk\squawk.suite.api
[compiling floating point sources in vmcore\src\vm\fp ...]
build failed: IO error during Process.exec(cl /c /nologo /wd4996 /DTRACE /DASSUM
E /DNATIVE_VERIFICATION=true /ZI /DIOPORT /DPLATFORM_BIG_ENDIAN=false /DPLATFORM
_UNALIGNED_LOADS=true -DWRITE_BARRIER -DLISP2_BITMAP /ID:\jdk1.5.0_14\include
/ID:\jdk1.5.0_14\include\win32 /Ivmcore\src\vm\fp /Ivmcore\src\rts\msc /Fovmcore
\build\e_rem_pio2.obj vmcore\src\vm\fp\e_rem_pio2.c)
caused by: java.io.IOException: CreateProcess: cl /c /nologo /wd4996 /DTRACE /DA
SSUME /DNATIVE_VERIFICATION=true /ZI /DIOPORT /DPLATFORM_BIG_ENDIAN=false /DPLAT
FORM_UNALIGNED_LOADS=true -DWRITE_BARRIER -DLISP2_BITMAP /ID:\jdk1.5.0_14\includ
e /ID:\jdk1.5.0_14\include\win32 /Ivmcore\src\vm\fp /Ivmcore\src\rts\msc /Fovmco
re\build\e_rem_pio2.obj vmcore\src\vm\fp\e_rem_pio2.c error=2
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.(ProcessImpl.java:81)
at java.lang.ProcessImpl.start(ProcessImpl.java:30)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
at java.lang.Runtime.exec(Runtime.java:591)
at java.lang.Runtime.exec(Runtime.java:429)
at com.sun.squawk.builder.Build.exec(Build.java:2315)
at com.sun.squawk.builder.Build.exec(Build.java:2297)
at com.sun.squawk.builder.ccompiler.MscCompiler.compile(MscCompiler.java
:104)
at com.sun.squawk.builder.commands.RomCommand.run(RomCommand.java:395)
at com.sun.squawk.builder.Build.run(Build.java:954)
at com.sun.squawk.builder.Build.runCommand(Build.java:919)
at com.sun.squawk.builder.Build.mainProgrammatic(Build.java:1440)
at com.sun.squawk.builder.Build.main(Build.java:1404)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.sun.squawk.builder.launcher.Launcher.main(Launcher.java:58)
E:\Squawk\trunk>

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

We really need to put instructions up and have just run out of time these past couple of weeks. We are currently on a big on-site with meetings every day.

If you are on a Linux machine, it should work fairly straightforwardly, but I have not had a chance to test it extensively.

For Mac OS you will need to install the Developer tools.

Windows is the most painful, you need to install
- Visual C++ 2005 Express beta and Visual Studio 2005 beta from
http://msdn2.microsoft.com/en-us/vstudio/Aa700736.aspx and install it.
- Windows platform SDK. The XP version works on Win2000. You can do a custom
install and skip everything apart from the core SDK, and within that, leave out the 64-bit
stuff). Go to http://www.microsoft.com/msdownload/platformsdk/sdkupdate/

Now to build
cd
d
d rom cldc
squawk com.sun.squawk.Test

For *nix machine, if . is not on your PATH then you may need to do a ./d. For windows it should work.

Here is a short description of what each step does
d
- build all modules
d rom cldc
- create Squawk executable and bootstrap suite with the classes found in cldc module
squawk com.sun.squawk.Test
- Execute the Squawk exe we have just generated and launch the main of com.sun.squawk.Test
- This class is one that we include for testing

We will provide documentation as soon as we can, but you can do a d -help until we get more.

yroussel
Offline
Joined: 2008-02-06
Points: 0

Looks like I'm lucky since I run my tests on linux.

Here is what I've done today to have something running on my ubuntu with a JDK6 as default.

on first try, running ./d failed with lots of java compilation error due to missing classes. First it looks like tools.jar was missing in the class path whatever I tryed to configure.
So I modified
com.sun.squawk.builder.Builder at line 750 by replacing
[i] toolsJarPath = toolsJarURL.getPath().substring(1);
[/i]by
[i] toolsJarPath = toolsJarURL.getPath();
[/i]which allow for me, that the generated file vm2c/preprocessed/javac.input contains a valid JDK path :
[i] -classpath vm2c/classes::/usr/lib/jvm/java-1.5.0-sun-1.5.0.13/lib/tools.jar -d vm2c/classes -target 1.5 -source 1.5 -g ...
[/i]instead of
[i]-classpath vm2c/classes::usr/lib/jvm/java-1.5.0-sun-1.5.0.13/lib/tools.jar -d vm2c/classes -target 1.5 -source 1.5 -g ...
[/i]

to rebuild builder.jar and builder-command.jar I previously had to modify builder/bld.sh so that it include tools.jar in the classpath
[i]
...
#----------------------------------------------------------#
# Go ahead and build build.ja #
#----------------------------------------------------------#
rm -rf classes
mkdir classes
$JAVAC -[b]cp $JAVA_HOME/lib/tools.jar[/b] -target 1.5 -source 1.5 -d classes -g src/com/sun/squawk/builder/launcher/*.java
$JAR cfm ../build.jar build-manifest.mf -C classes .
rm -fr classes
mkdir classes
$JAVAC[b] -cp $JAVA_HOME/lib/tools.jar[/b] -target 1.5 -source 1.5 -d classes -g `find src -name '*.java'`
$JAR cfm ../build-commands.jar build-commands-manifest.mf -C classes .
rm -fr classes
[/i]

then it builds using the following command in builder dir :
[i]./bld.sh /usr/lib/jvm/java-1.5.0-sun-1.5.0.13/[/i]

* it is mandatory to specify a JDK 1.5 path, it does not compile using JDK6

aflter this, running the following command in completes normally
[i]JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun-1.5.0.13 ./d[/i]
so as
[i]JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun-1.5.0.13 ./d rom cldc[/i]

running the test with the following command :
[i]LD_LIBRARY_PATH=/usr/lib/jvm/java-1.5.0-sun-1.5.0.13/jre/lib/i386:/usr/lib/jvm/java-1.5.0-sun-1.5.0.13/jre/lib/i386/client ./squawk com.sun.squawk.Test[/i]

Is a success : Great !!

Thanks a lot for your quick answer.

So now it's time for me to try understanding if I can have usage of this :D

Message was edited by: yroussel

eric_arseneau
Offline
Joined: 2004-07-15
Points: 0

Good catch. That's a "fix" I put in at the last minute in order to enable a fresh SPOT build a couple of weeks ago, and was unable to test it properly.

I will be looking into why the substring(1) seems to be needed on some Windows machines and put a more robust patch in.

The good news is that we are working on a regression harness to catch these types of issues when we make changes to the build system. This way we will be able to ensure that what we do on one platform will work on all.

CONGRATS btw, you are the first one to build Squawk outside of the group here :)

rasped
Offline
Joined: 2008-01-31
Points: 0

Hi Eric,

Thanks for the Windows hints above.
Here is a first attempt for building on Windows XP.
a) Change line 50 in d.bat to [i]goto gatherArgs[/i]. Otherwise the bat file will not take more than one argument.
b) start with [i]d.bat copyphoneme[/i]

squawk.exe completes fine with its tests in com.sun.squawk.Test.

I'll get back with more details later:-)
Best regards, Rasmus