Skip to main content

Official Abstract Syntax Tree (AST) for Java

19 replies [Last post]
angben
Offline
Joined: 2004-10-07
Points: 0

J2SE should specify a standard abstract syntax tree implementation for Java. This form should be annotable for flexible extension. The AST should be easily serializable to and from XML. I would like to be able to write code that reasons about other Java code without having to depend on a particular Java parser's specific AST implementation. Some of this may overlap with JSR 198.
Thanks for the feedback.
Ben Youngdahl

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
sagarshinde
Offline
Joined: 2011-07-24
Points: 0

How can use the JDT to create an AST from java source code and then what are the methods to extract/ parse the methods/ variables etc..from source code and then change AST to reflect the changes in new code. ?

angben
Offline
Joined: 2004-10-07
Points: 0

> I am one of the designers and implementers or apt.
> For Mustang or Dolphin, we plan to run a JSR to
> o standardize a second generation version of the apt
> API. One of the additions being considering is
> providing a full AST of the source program structure.

This is great to hear. I would say that this feature cannot wait for Dolphin. It is critical to competing with .NET as mentioned in other posts in this thread. Without the AST standardized, folks can't write cross-compiler AST-driven tools; that hurts the evolution of the platform.

cancin
Offline
Joined: 2007-09-11
Points: 0

Are there any news concerning this topic. I know it's rather old but nevertheless very interesting. Is there a standard ast implementation für java now?

tsinger
Offline
Joined: 2003-06-10
Points: 0

I believe, that the future of programming languages [i]is[/i] changing the language to match your problem perfectly, to create a domain-specific language. For more information, please take a look at http://www.codegeneration.net/tiki-read_article.php?articleId=60 (search for "MPS").

Tom

patrikbeno
Offline
Joined: 2004-10-11
Points: 0

That's what I believe, too. But domain-specific language is something to be built on a rock solid base (general purpose language).

By allowing everyone making arbitrary changes to this rock solid base, we get 80% problems and 20% potential benefits.

What you describe is something different.

nevertheless, back to the subject - I agree official AST would be a positive thing.

rreyelts
Offline
Joined: 2003-06-10
Points: 0

No, I think patrick is on the mark here. Sun (and most developers) don't want macros in Java due to the arbitary syntax changes.

Annotations and a writable AST give us a way to perform transformations that a lot of commonly used tools already do - just much more easily and in a standard and portable fashion.

God bless,
-Toby Reyelts

jddarcy
Offline
Joined: 2004-11-02
Points: 0

I am one of the designers and implementers or apt. For Mustang or Dolphin, we plan to run a JSR to standardize a second generation version of the apt API. One of the additions being considering is providing a full AST of the source program structure.

mortench
Offline
Joined: 2004-11-03
Points: 0

That would really great!!! Better sooner than later!Microsoft.NET 1.1 already has something like this called CodeDOM which has proven to be very popular (and a great enabler for modern MDA approaches). You should look at CodeDOM for inspiration.

When you do this for Java, try to make it extensible (i.e. permit custom ast nodes) so that java language extensions like Aspects can use this as well (i.e. again like CodeDOM which allows for multiple languages even though it is closely related to C#).

jodeleit
Offline
Joined: 2004-10-26
Points: 0

> Some of this may overlap with JSR 198.

Do you mean "JSR 199: Java Compiler API"? This was scheduled for Tiger - didn't made it.
And Neal Gafter, the mantainer left SUN and works now for Google.

Quoted from the JSR Page:
"Future versions of this API might expose more of the structure of the program, for example the declaration structure of the program (ala the javadoc API), program annotations (JSR 175) or even the code itself (ASTs: Abstract Sytntax Trees). These are not goals of the initial version of this specifications."

This does not sound promising for Mustang :(

jodeleit
Offline
Joined: 2004-10-26
Points: 0

and 'til this dream comes true - use Eclipse..

dgriffit
Offline
Joined: 2004-10-13
Points: 0

As someone who has written over a hundred thousand lines of syntax tree analysis and manipulation code against the IntelliJ IDEA OpenAPI (InspectionGadgets, MetricsReloaded, IntentionPowerPack), let me say that this would be an extremely valuable tool in entrenching Java as the programming language of choice for enterprise development. An official AST for Java could drive the cost of a lot of valuable development tools down to nothing. I'm thinking of tools like refactoring engines, semantic search, static analyzers, metrics, layout, duplicate finders, test coverage analyzers, test case generators, and program structure and execution visualizers.

odd
Offline
Joined: 2003-08-20
Points: 0

This would be a great addition to the Java platform.

I am aware that it could (and probably sometimes would) be used in ways that would make code less understandable, but the argument that a certain feature could be misused is not automatically a strong enough reason not to add a feature.

In this case I think the usefulness of an official abstract syntax tree for Java far outweigh the potential problems of any misuses.

tsinger
Offline
Joined: 2003-06-10
Points: 0

Interesting idea. Especially if one could extend the recognized language in some way (e.g. some guys want to introduce their own === operator, other want to introduce const).

Tom

patrikbeno
Offline
Joined: 2004-10-11
Points: 0

Well, and we all end up with thousands of different Java versions, minor "enhancements" and "modifications" and one will no longer have a chance of reading and maintaining other's code.

Isn't this why operator overloading and macros were rejected?

vhi
Offline
Joined: 2004-10-11
Points: 0

Give a developer gun to shoot,
he will definitely aim it on the foot. :-)

prunge
Offline
Joined: 2004-05-06
Points: 0

This would be absolutely brilliant for apt. Having the ability to have annotations add/remove/change methods, fields, etc. to classes would be very useful.

rreyelts
Offline
Joined: 2003-06-10
Points: 0

I agree. Let's take it one step further and rewrite apt so that it exposes a writable version of this standard AST. This would make life so much easier for those of us currently writing annotation processors, bytecode enhancers, or other related tools.

God bless,
-Toby Reyelts

cowwoc
Offline
Joined: 2003-08-24
Points: 0

+1

Moving bytecode rewriting into APT (Toby don't shoot me!) would be great! :) :)

Gili

henry_az
Offline
Joined: 2010-12-27
Points: 0

there are some open-source AST Generators, such as javacc, antlr etc.

Hope you could refer to them and provide developers a simple JAVA AST interfaces. it can be very useful to code check.

best appreciates