sanjay_dasgupta's Blog

Vulcan-ized Rhino: Telepathic Power for your Code

In this article we coax the JVM's Rhino (an elusive, misunderstood, and ignored member of the ecosystem) into a mind meld, giving it access to the JVM's thoughts, experiences, memories, and knowledge; and take it where no Rhino has gone before !

Let me set the context with some quick code:

ScriptEngineManager sem = new ScriptEngineManager(); 
ScriptEngine jsEngine = sem.getEngineByName("javascript"); 
    ... 
String message = "Hello rhino!"; 
    ... 
jsEngine.eval("println(message)");

Everyone knows that this code does not work (it produces a "ReferenceError: "message" is not defined"). To make it work the variable message must be put into the script engine's bindings, as described in these articles. That's easily done. But the overhead and distraction of the extra boilerplate makes the body of code much less intuitive. (The 4-line example above already has 2 lines of distracting boilerplate!)

A Quick Example

What can we do to make something as simple as "println(message)" in a script just work? In fact, let's raise the bar some more. Take a look at Sqrt.java. Let's say you were explaining that code to a novice, and wanted to provide a probe into the while loop of the running program, by adding the line in red:

    ...
while (Math.abs(t - c/t) > epsilon*t) {
    t = (c/t + t) / 2.0;
    if (args.length == 2) 
        VulcanRhino.eval(args[1]);
}
    ...

Think of class VulcanRhino as your friendly telepathic pachyderm, and eval() its static, void JavaScript evaluator. The idea is that a JavaScript snippet could be passed into the program as an optional second command-line argument. That snippet (specified at run time) could contain logic with references to any of the in-scope Java variables. The code above is a simple example. But this approach allows you to include any number of VulcanRhino.eval()s, located wherever the invocation of a static void function would be legal, each executing a different script. Each invocation of VulcanRhino.eval() has access to all in-scope variables at its location.

Our modified Sqrt.java would run normally (doing nothing unusual) if run with just one command-line argument, but giving it a second argument awakens the slumbering telepath. Here are a few sample runs (the different colors separate the command line from the program's output) ...

> java Sqrt 49 "println(t)"
49
25
13.48
8.557507418397627
7.141736912383411
7.001406475243939
7.000000141269659
7.000000000000002

> java Sqrt 49 "println(c/t)"
1
1.96
3.635014836795252
5.725966406369197
6.861076038104466
6.9985938072953795
6.999999858730344
7.000000000000002

The last line of output (struck out) is not from the script, but is the program's normal 1-line output. The examples above use scripts to track the values of "t" and "c/t" respectively. But you are free to pass in any expression that makes sense at the location of VulcanRhino.eval(). You may even use it for something completely unforeseen ...

> java Sqrt 49 
  "println(java.lang.System.
  currentTimeMillis())"
1327399502966
1327399502970
1327399502973
1327399502974
1327399502975
1327399502976
1327399502977
7.000000000000002

> java Sqrt 49 
  "println(java.lang.Runtime.
  getRuntime().freeMemory())"
30472936
30472936
30472936
30472936
30308384
30308384
30308384
7.000000000000002

The one thing you can not do with a script in this way is to assign a value to a variable.

The Vulcan-Rhino User Guide

To use this approach, you must pre-process your source-code using a tool described below. This step is the key to the magic -- it augments each VulcanRhino.eval() in your code with something that gives it access to all the in-scope variables. So, proceed as follows:

• edit your program (say Sqrt.java), adding VulcanRhino.eval()s as required, and save it with a different name (say SqrtVR.java)
• pre-process SqrtVR.java following instructions below. Save the output as Sqrt.java. Note: this overwrites any other Sqrt.java
• run as usual, making sure that class VulcanRhino is on the classpath. The VulcanRhino.java source should be compiled and deployed as required.

To pre-process a file use the following command:

The files used are described below:

• VLL4J.jar is the distribution JAR from the VisualLangLab project (a visual parser-generator)
• VulcanRhino.vll is the transformation grammar described further under Pre-Processor Internals below
• SqrtVR.java is actually Sqrt.java saved with a different name

If you have trouble with the above steps, check the following:

• does a SqrtVR.java file exist?
• have you edited SqrtVR.java to add VulcanRhino.eval(args[1])
• copy and paste the command line above directly
• ensure VulcanRhino has been compiled and exists on the classpath

How Does it Work?

Let's first get VulcanRhino out of the way. Observe that eval() does nothing special, but there is another function defVars() that enables the caller to inject information about variables into the JavaScript engine.

import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;

public class VulcanRhino {
    public static void eval(String script) {
        try {
            engine.eval(script);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    public static void defVars(Object... args) {
        engine.getBindings(ScriptContext.ENGINE_SCOPE).clear();
        for (int i = 0; i < args.length; i += 2) {
            String name = (String)args[i];
            Object value = args[i + 1];
            engine.put(name, value);
        }
    }
    static ScriptEngine engine = new ScriptEngineManager().getEngineByName("javascript");
}

Next take a look at the pre-processed version of Sqrt.java.

    ...
while (Math.abs(t - c/t) > epsilon*t) {
    t = (c/t + t) / 2.0;
    if (args.length == 2)
        {VulcanRhino.defVars("epsilon", epsilon, "c", c, "t", t, "args", args); VulcanRhino.eval(args[1]);}
}
    ...

The part you added is still in red. But the pre-processor has spliced in the blue text. The pre-processor makes this change at each occurrence of VulcanRhino.eval(...), injecting information about the locally visible variables into the JavaScript engine.

Pre-Processor Internals

I won't go into all the details here, presuming that not everyone is interested. So the remaining part of the article is a short summary of the technique together with links to all the other material you will need to understand the details.

The pre-processor uses a parser for the Java language to analyze your program and obtain information about which variables are visible at each VulcanRhino.eval(...) location. It then modifies the source code by wrapping each VulcanRhino.eval(...) in a block ({ ... }) preceded by a VulcanRhino.defVars(...) call that injects the information required into the JavaScript engine.

The parser-generator used is the easily learned, completely visual tool VisualLangLab. For an introductory tutorial look at A Quick Tour. Scala programmers will find Rapid Prototyping for Scala useful too.

The last piece of the puzzle is in the grammar file VulcanRhino.vll. This file contains a Java grammar modified with action functions that perform the pre-processing. To examine the grammar, its rules, and the code in the action functions, proceed as follows:

• double-click VLL4J.jar (the same file used in the pre-processing step described above). this will start up the VisualLangLab GUI as shown in Figure-1 below
• select "File" -> "Open" from the main menu, choose the grammar file VulcanRhino.vll, then click the Open button
• in the rule-tree (the JTree at the left of the GUI) select (click on) the node just below the root node (see red arrow). This will cause the action-code associated with this parser-rule to be displayed under Action Code (right side of the GUI). This is the code (in JavaScript) that pre-processes your code

alt="Using VisualLangLab">

Figure-1 VisualLangLab GUI with VulcanRhino grammar loaded

The information used by the action-code above is in several global variables (VLL members). That information is gathered by other action-code in other rules. To examine all the remaining code proceed as follows:

• select the rule named block (use the combobox in the toolbar), and click on the reference node labeled blockStatement
• select the rule variableDeclaratorId, and click on the sequence node just below the root node
• select statement, click on the node just below the token node for FOR

If you do want to pursue this further, a thorough reading of A Quick Tour is strongly recommended. You will also need AST and Action Code and Editing the Grammar Tree.

VisualLangLab 7: New Features, Expanded Tutorials

A new tutorial that exercises VisualLangLab using all the examples and techniques in Chapter-3, A Quick Tour for the Impatient, of the book The Definitive ANTLR Reference can be found at this link.

Various other improvements have been made in version 7:

• A new WildCard pseudo-token that matches any defined token has been added to facilitate recovery from errors in the grammar.
• Each part of the AST description is now tagged with the contributing sub-rule's description field to clarify the sub-rule to AST-segment association.
• Intuitive icons have been added to the rule-tree's context menus
• use of the token-creation dialogs have been simplified 

These changes make VisualLangLab even more powerful and user friendly.