Skip to main content

Create an open source javac for experimenting with new features

21 replies [Last post]
jsando
Offline
Joined: 2003-06-22

There are some terrific ideas being brought up, more than a few I wish I could try on for size and see how they work out.

Towards this end how about if Sun release an open source version of javac, to allow developers to implement their ideas without having to build a compiler from scratch? It seemed like they did something like this for generics.

We could setup a project page for it, to support the experimenting with new language features, to help developers who think they've got a great idea on how to actually implement that within the compiler.

If there is already an open source Java compiler written in Java, point me to it. If there isn't ... why not?

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
jsando
Offline
Joined: 2003-06-22

Forget the hand-coded parser comment, I just didn't see it in the source drop. On their website they explain how to build the parser and what tool to use.

See http://eclipse.org/jdt/index.html and click on "core" for the home page.

tomia
Offline
Joined: 2003-06-13

try the eclipse-compiler ;)

jsando
Offline
Joined: 2003-06-22

Duh! I can't believe I didn't look there, I just assumed they were running javac. It was right in front of me!

I took a quick peek at it. It is production code, which is fantastic. However that makes it more difficult to experiment with, since the scanner and parser are hand optimized rather than generated.

But wow, being able to experiment with a production-quality compiler, along with a complete IDE with intellisense and everything, is an incredible opportunity. I think Eclipse is where its at. I'll just have to jump in and try something.

jsando
Offline
Joined: 2003-06-22

Wow, they really put a lot of work into it. Lots of documentation. I was unable to spot a LICENSE or COPYRIGHT notice anywhere, and it also has not been updated apparently since 1998. So the question is could it be updated under an open source license.

I like the use of JavaCC - using a parser generator makes it easier to experiment with. They manually build the syntax tree, and utilize the node classes to organize code generation, as opposed to using sort of "dumb" nodes and just visiting them from a single code generation method. I'm kind of mixed on that, since if you want to make a major change you'll have to modify the 80 syntax tree node classes rather than just visiting them differently.

arya_18
Offline
Joined: 2003-06-12

I agree with you..can we try to do something to bring this poject back to life...I am no expert on compilers but would use this oppurtunity to understand the internals...u sound like one though:)!!!...can we take it further??

jsando
Offline
Joined: 2003-06-22

Not an expert, just played around quite a bit over the years. There's a lot I would like to learn, such as code optimization, and how compilers deal with object oriented languages.

I find compilers particularly fascinating. I think they must be some of the most complex, highly integrated programs that exist. And you can always see the most of a language in its compiler, its just so complex it tends to utilize the complete features of the language.

jsando
Offline
Joined: 2003-06-22

Although I feel like a BSD style licensed compiler would be a better platform to experiment with, I have gone ahead and set up the Javac Research Project on java.net, to see if there is really much interest in this concept.

The project is at: http://javacresearch.dev.java.net

See 'Documents and Files' for the current downloads.

I've created a bundle that includes JUST the javac source code, along with an Ant build file and win32 .bat files to run it. Folks wanting to experiment can download the 'base' bundle, which is the pristine javac source.

I've also implemented the "allow objects in switch" idea, and posted that under the folder 'obj-switch'. It contains a docs folder that has more explanation.

Feedback / suggestions welcomed!

jsando
Offline
Joined: 2003-06-22

Okay, I have a modified Javac 1.5 with support for Objects in switch() statements. Its not very good on error checking.

JavaC has a pretty nice internal structure, and its only about 70 source files compared to eclipse's 470 (or so).

It was somewhat easy to figure out, but when it came to code generation I got really bogged down. It took me probably an hour just to figure out how to stick "Object.equals(Object)" into the constant pool.

I will see if I can configure a java.net project with the JRL and post my modified version.

patrikbeno
Offline
Joined: 2004-10-11

No sure about the license. Can you do that?

jsando
Offline
Joined: 2003-06-22

The restrictions are that it be used only for research or educational purposes and not for commercial "productive" use; also that each user must do a click-through of the JRL for source downloads.

I would prefer to use something that's under a more free license, but these are Sun's terms.

The benefit to using JavaC is that it IS the reference implementation. Its already 1.5 compatible. The drawback is the JRL. Perhaps, for experimentation, it might be better to NOT use the reference implementation, so that we can all feel comfortable doing major refactoring.

The Eclipse compiler is heavily integrated into the platform. It has big dependencies and a massive amount of source files. And it absolutely screams. They're working on 1.5 features right now. With Eclipse, you could add a new language feature AND have it integrated into an IDE.

Another option is to start from scratch, but why, when there are at least two production quality compilers already available in source code? If we build something, how good a quality is it likely to be? Good enough for daily use? No way! How much experimentation would happen with a tool that nobody trusts?

And now I've argued myself into a corner ;) Because the JRL prohibits "productive use", so javac can never be used for anything besides a toy. And that's a MAJOR problem. If you can't try an idea out in a real situation, then you haven't tried it out at all.

Maybe another option could be to study what's available, take what's free (Eclipse), and refactor it to be what we need.

monika_krug
Offline
Joined: 2004-10-14

What do you mean by "it screams"?

Monika.

jsando
Offline
Joined: 2003-06-22

It seems to run very fast.

kcpeppe
Offline
Joined: 2003-06-15

This is a great idea.. in fact, IMHO, it is one that is really needed. As for language of implementation, I agree that it should be in Java. I've used javacc in the past to generate some very specialized languages. It is a little bit of a bear to use but that said.. I do believe there is a Java2 grammer avaliable that one could use as a starting point.

seligman
Offline
Joined: 2003-06-10

Javac, along with all of the JDK 5.0 source, is now
available for download under the Java Research License.
http://www.java.net/jrl.csp

jsando
Offline
Joined: 2003-06-22

A lot of developers will have issues with the JRL.

However, it is to great advantage to experiment with the actual javac compiler, since if Sun/JCP agrees to incorporate a feature it will already be in the proper source tree (as opposed to eclipse or some homegrown compiler). There's a better chance of it being accepted.

I'll take a look at javac.

vhi
Offline
Joined: 2004-10-11

I guess Jikes is exactly what you are looking for:

http://www-124.ibm.com/developerworks/oss/jikesrvm/

jsando
Offline
Joined: 2003-06-22

No doubt the VM could be a part of it, but I'm much more interested in language enhancements that would run on existing VM's.

The Jikes compiler exists, but is written in C++ (same for gcj). I'd like to see an open source Java language compiler that's written in Java.

zander
Offline
Joined: 2003-06-13

> I'd like to see an open source Java
> language compiler that's written in Java.

Why?

jsando
Offline
Joined: 2003-06-22

So that the [i]Java[/i] community can experiment with it. Jikes and gcj are in C++, so they are not nearly as accessible for Java developers.

I've done lots of C in years past, including parsers, but from a few hours spent looking at the sources for jikes and gcc I can say they require more effort than I'm willing to commit, mainly because Java is now my primary platform.

javac is written in Java. If Sun was willing they could make it available, but it would need to be under a very free license (such as, "do whatever you want, just don't call the resulting language 'Java'").

arya_18
Offline
Joined: 2003-06-12

+1 vote for ur idea. I have been trying to find java compilers written in java for quite some time now. One academic work I came across was Espresso. So I downloaded the code but only to my utter disappointment that the source bundle did not include the src for the most critical class in the application:(

monika_krug
Offline
Joined: 2004-10-14

Did you try to decompile it?

Monika.