Making Fluid 3D Creatures with JOGL

What has your GPU done for you today? Most modern computers are equipped with a ridiculously fast chip that's dedicated to graphics processing, but Java programs rarely get the chance to make it sing. With the JOGL [7] API, we now have a way to let the GPU take over the lion's share of the math, which suddenly makes Java the language of choice for a lot of 3D applications. If we put all of those design patterns and sophisticated coding techniques we've been learning and refining together with bare-metal access to the graphics processor chip, we should be able to create surprisingly robust 3D apps that are also as fast as can be. This article describes my JOGL adventures while building a virtual universe that I call "Fluidiom" (fluid + idiom), based on push and pull forces.

Figure 1 shows Fluidiom in action.

[8]
Figure 1. Fluidiom main window

If you have a recent JVM, installed properly, and a 3D-enabled computer, you should be able to run my Web Start application with one click on the screen shot above. You will have to approve of my self-made certificate for signing .jars. The creatures you'll see there have learned to run by survival-of-the-fittest evolution!

Push and Pull?

Things look alive when they move around sort of autonomously, and even more so if their structures are not rigid like crystal. Most of the 3D graphics you see these days, with the exception of the pre-rendered Pixar miracles, seems to be made of solid blocks, and if you're lucky, they appear to articulate, but mostly on hinges or ball-joints like Transformer toys. 3D animators masterfully encode object movements to look as natural as possible, but that takes the keen perception of an artist to fool you.

At the beginning of 1996, I had an unstoppable urge to find out what this new Java language was all about, and at the same time I had become fascinated with a piece of outdoor art [9] that I had seen in the east of Holland. I wanted to recreate this tower in the computer. Soon, I became addicted to the wobbly tension-based structures that were appearing. I built the model on the basis of springs (things that push or pull), which I later called elastic intervals.

Why elastic intervals? Simplicity. There just aren't many ideas simpler than two dots connected with a line that has a preferred length. The fun starts when you connect the dots; when you connect up four dots with six lines that all prefer the same length, you're already launched into 3D! Connect a bunch more together, and they start to show all sorts of sometimes-surprising behavior, such as when a structure's elastic intervals are all under stress, but the structure as a whole is completely stable.

I started to think about what other things could be easily done. It seemed that the intervals could just as well be like muscles if their preferred lengths varied over time. Then, with structures writhing about methodically, you get to wonder if they could learn to walk. That's what I'm working on these days, but there's a mailing list [10] for this kind of banter. Time to get into JOGL.

The Need for Speed!

My first renderings of the structures used java.awt.Graphics from the Java 1.0 Virtual Machine to make wireframe images. I was sold on Java immediately when I saw how easy it was to put together a GUI with lots of controls and still see a fairly reasonable frame rate on the wireframe graphics. There were a lot of calculations going on, but the JVM seemed to perform well enough, and that improved vastly when JIT compilers arrived.

Then Java 1.2 arrived, with its graphics layer completely rewritten. The new java.awt.Graphics2D was able to do all of the wild and wonderful things you can do with PostScript using Java2D, such as making partially transparent lines that are 1.432133 pixels thick, and other stuff. That was cool, but my animation frame rate collapsed, and my heart sank. Java was no longer good at animation, but had chosen to be good at desktop publishing instead. I'm sure there were a million justifications, and one of them was perhaps that there was a 3D API coming out. Major changes would have to be made, so that put the Fluidiom project on ice.

Eventually I got back to the project (during train rides) and started to program using the maturing Java3D API. It was definitely interesting and educational to understand how scene graphs [11] in Java3D worked, and it made sense to use Java to attach behaviors to 3D objects. The 3D acceleration helped a lot, and bigger springy structures appeared with shading and all, but larger structures were still bringing the frame rate back to one frame per second.

A friend of mine looked over my shoulder one day and shook his head at how slow it was, and that got me mad. He had worked in OpenGL using C, and so I went hunting and found JOGL. The following day turned into a coding frenzy; by the end of the day I had rewritten the entire program to use JOGL instead of Java3D. The previously one-frame-per-second structures suddenly flowed smoothly enough on my screen that there was no need to talk about frame rate at all! Suffice it to say that I celebrated that night.

Nobody should have the illusion that any Java3D program can be rewritten in one day, as I had done, because I had some very specific advantages. First of all, my structures made of springy things connected together was already a kind of scene graph, so I had actually been mapping it to the Java3D scene graph. My project also doesn't need terribly sophisticated interaction with the mouse or other devices. More importantly, the quantum performance leap was because Fluidiom required that each and every object in the scene move all the time, which is fairly rare in 3D and precludes a whole bunch of optimizations that can be applied to static things. Java3D is set up to optimize groups of graphics objects that together form a big object, but it doesn't help when every little object in the whole scene is jiggling around.

What's on Display?

I had it easy. There are only two kinds of things that need to appear in Fluidiom graphics: intervals and joints. Each interval is a springy thing connecting two joints, and a joint brings together one end of a bunch of intervals. The intervals and joints are gathered together into something called a fabric, and that's all there is. A joint contains the 3D coordinates and a vector describing where it's moving. An interval holds its preferred length (span), and its two joints. The fabric is just a collection of joints and intervals.

The essence of these classes is this:

// a moving dot in space that brings together
// a bunch of interval ends
public class Joint {
  public Point3f locus = new Point3f();
  public Interval [] interval;
  public Vector3f moment = new Vector3f();
}

// a connection between two joints, with
// a preferred length
public class Interval {
  public Joint alpha;
  public Joint omega;
  public float span;
  public float stress;
}

// collect joints and intervals together
public class Fabric {
  public Joint [] joint;
  public Interval [] interval;
}

In the real code (available in the Resources section below) there are some other things involved. The reason the fields are public, for example, is to make it easy to store the objects in XML using reflection (but that's another story). It also allows the code to be quite a bit more concise, avoiding the getXxx() and setXxxx() methods for the most commonly accessed things. I chose to make the fabric of joints and intervals publicly accessible and deal with them through a special singleton called FabricShell that lets you change the fabric, and at the same time broadcasts FabricEvent messages to any observers. Ha, two design patterns already!

Note that I'm using the Point3f and Vector3f classes, which was one nice thing inherited from Java3D: vecmath.jar. Originally, I had created some classes for doing the vector calculations, but when I started with Java3D, I discovered that it had all been done quite well [12], and there were matrix operations in there that I didn't really want to write.

Probably the most important observer of the Fabric is the JoglFabric, since it is responsible for maintaining a parallel data structure that deals with all things JOGL. Every change in the Fabric immediately changes what you see when the JoglFabric reacts to the change event. More about that in a minute.

Add the queen of all design patterns, the visitor [13], and suddenly the code becomes loosely coupled and you can freely and easily add new chunks of functionality that can be plugged in or out whenever you want. It also greatly simplifies the FabricShell, since you can do all sorts of different things by simply calling one overloaded method:

public class FabricShell {
  private Fabric fabric;
  public void admitVisitor(JointVisitor theVisitor) {
    // have the visitor visit all the joints
  }
  public void admitVisitor(IntervalVisitor theVisitor) {
    // have the visitor visit all the intervals
  }
}

public interface JointVisitor {
  void visit(Joint theJoint);
}

public interface IntervalVisitor {
  void visit(Interval theInterval);
}

There are visitors for all sorts of things in Fluidiom, but there are two main visitors responsible for breathing life into the fabric by activating the pushes and pulls. The physics of pushing and pulling is nothing more than basic vector math, with its little arrows and dots. The fabric is given life by a straightforward brute-force iteration process. First, the Exerter, an IntervalVisitor, looks at the difference between preferred and actual span and exerts vector forces on the moment (a vector for where the joint is moving) of its two joints, either pushing or pulling them. Then, the Mover, a JointVisitor, is sent in to change the locations of all of the joints based on the accumulated moment vectors. Just keep repeating these steps, and the whole thing wobbles like jelly.

Synchronization

When I teach Java to C/C++ programmers, I always tell them that threads and synchronization are good for nostalgia. With those other languages, we created single-threaded code with tiny bugs that produce unpredictable results, whereas in Java, you get a slap on the wrist in the form of an exception. Multithreading can give you that feeling of unpredictability again in Java, so savor it. Ask a good Swing programmer about threading, and they'll tell you about the importance of keeping things simple.

The best way to synchronize is to not need synchronization at all, if possible. When you write using JOGL, there's a part of your program that has a mind (or better, a thread) of its own, because when you create a GLCanvas you want it to take care of its own updates in its own time with an Animator. What you get in return are callbacks to any GLEventListener that you attach to the canvas. Your code should be modifying the visible objects in between frames only, so it's in the callbacks that all the work has to be done.

I need the iterations to happen at a certain speed, regardless of the frame rate available for a particular machine, so in the display callback I check if it's time to do the number crunching with the FabricShell.get().iterate() method:

public void display(GLDrawable theDrawable) {
  // .... do all the displaying first ...
  // then decide if the fabric needs
  // an iteration yet
  if (System.currentTimeMillis()-lastIteration
            > ITERATION_DELAY) {
    FabricShell.get().iterate();
    lastIteration = System.currentTimeMillis();
  }
}

JOGL Basics

In the beginning, there was a GLCanvas, with an Animator and a GLEventListener:

public class JoglFabricView extends JFrame
implements FabricListener, GLEventListener
{
  private GLCanvas canvas;
  private JoglFabric joglFabric = new JoglFabric();
  private Animator animator;
  // construct the parts in the frame
  public JoglFabricView() {
    super("Fluidiom Fabric");
    canvas = GLDrawableFactory.getFactory()
      .createGLCanvas(new GLCapabilities());
    canvas.addGLEventListener(this);
    getContentPane().add(canvas,BorderLayout.CENTER);
    animator = new Animator(canvas);
  }
  // get things ready for display
  public void init(GLDrawable theDrawable) {
  }
  // draw the scene
  public void display(GLDrawable theDrawable) {
  }
  // not sure what this is for so left it empty
  public void displayChanged(
    GLDrawable theDrawable,
    boolean theModeChangedFlag,
    boolean theDeviceChangedFlag
  ) {
  }
  // set up the matrices
  public void reshape(
    GLDrawable theDrawable,
    int theX, int theY,
    int theWidth, int theHeight
  ) {
  }
}

The first callback you get is to initialize things. I didn't dig too deeply here, but rather just snagged some hints from code in the JOGL demos and from the various OpenGL tutorials you can find on the net. That's the thing with JOGL: most of the information you need can be found in the form of C or C++ code examples, since the API that JOGL presents to you is almost the same as the one that all OpenGL programmers get. Code snippets are surprisingly easy to translate into Java.

The code that I found helped me set up the init() method, which sets up lighting and sets some basic rendering attributes, and start the display() method, which clears the screen and then draws everything. Both of these methods actually stayed quite small, because I had them delegate work to similar methods in the JoglFabric class, which was responsible for setting up and showing the fabric rendering. The display() method and the cascade of calls inside of it do most of their work in matrix operations, which I'll get to shortly.

The Frustum

There's one more thing to work out before it can all get going, and that's the frustum, which is just a funny-shaped box in space that represents what the viewer can see. The frustum has six flat planes: front, back, top, bottom, left, and right. The front and back planes are parallel, but the back plane is bigger than the front one. Objects inside of the frustum appear on screen, and the rest do not. It makes no sense for the graphics card to calculate all of the shading and such for objects that are not going to be painted on the screen; the frustum gives the GPU the ability to ignore all irrelevant polygons. Too close, too far away, or beyond the sides, and the object vertexes calculations are enough. This is cool, because this allows you to fly inside of complicated objects with your "camera"!

The frustum is something you set up inside of the reshape() method, because its actual shape depends on the aspect ratio of the canvas you're using. There is a call to reshape() before anything is shown, so you get a chance to set up the frustum first.

The GL_PROJECTION matrix is the one that embodies the frustum, while most of the work is done with the GL_MODEL_VIEW matrix. There's a lot of documentation on the Web to explain, in excruciating detail [14], how these matrices are used.

Picking

Click your mouse on the display window to select a 3D object. It sounds easy, but imagine trying to make it work. Your mouse click happens at some integer (x,y) coordinate on the screen, but the object is being projected through a bunch of floating-point matrix operations to its spot on the screen. You have to think of your mouse click shooting an arrow into the "scene" to see which object it hits. Down deep inside, somebody has to figure out which polygon in the scene intersects with the arrow you shot, and then determine which "object" owns that polygon. Java3D has a solution for this, but JOGL leaves it up to you.

I must confess that laziness overwhelmed me here, so I took advantage of the simplicity of my model. I sidestepped the idea of finding a polygon intersection point, because I had no idea how to do it. Rather, I just shoot the mouse-click arrow into the scene looking for the closest midpoints of intervals (just 3D points, not polygons!) and then favoring the closer ones over the further ones. Even my lazy approach wasn't trivial, however! In my code, the picking trick is in JoglFabricView.display(), using special matrix calculations.

The Matrix

Just like Neo in the movie [15], you have to master the matrix if you want to make it work for you. A matrix is like a lens that transforms one universe of coordinates to another one in very specific ways. It can translate things from one position to another, rotate things around the origin, and scale things wider or narrower in different directions. OpenGL lets you perform these operations individually, with each one actually affecting the current matrix in the graphics card.

In Fluidiom, I display hundreds of intervals, all moving at the same time from frame to frame, and this is all done with hundreds of different matrix operations performed on what is effectively a solitary graphics object. One graphical object is displayed in many different ways, or viewed through different matrix lenses, to make up one image. It's a very economical way to work, since single objects are reused, and this kind of activity is behind most 3D stuff you see.

Here's the code that displays an interval, for example:

// preserve the current matrix for later
theGL.glPushMatrix();
// throw the object out into space
theGL.glTranslatef(
  intervalLocation.x,
  intervalLocation.y,
  intervalLocation.z
);
// twirl it around to the right angle
theGL.glRotatef(
  RADIANS_TO_DEGREES*
    (float)Math.acos(intervalDirection.z),
  -intervalDirection.y, intervalDirection.x, 0
);
// stretch it out along the z-axis
theGL.glScalef(
  theIntervalRadius,
  theIntervalRadius,
  span/2
);
// paint the object
theGL.glCallList(shape);
// restore the preserved matrix
theGL.glPopMatrix();

Pay close attention to the ordering, because it might trick you. You translate first, then rotate, then scale. What you're actually doing is setting up a single new matrix, and the operations are actually happening in the opposite order. First, we scale the object (here, it gets oblong, because it's supposed to represent a springy interval) then rotate it (trigonometry anyone?), and finally, flick it out into space at the desired location.

You can see that here, too, you are responsible for telling the graphics card to save its current matrix before you mangle it, and then telling it to restore the previous one when you're finished. The only non-matrix call here is the glCallList(), which brings us to the next topic.

OpenGL Lists and OOP

OpenGL is not an object-oriented universe (it's really just a flat static C API), and JOGL doesn't do a lot more than represent it as some colossal Java classes with kerjillions of static (native) method calls. (The classes are so large, by the way, that they hang my IDE if I ask for code completion!) That is, of course, not to say that you can't write tidy object-oriented code against this API. You just have to imagine a bridge between the CPU and the GPU, and imagine your classes as having part of their state living "over there."

In OpenGL, you can create what is called a list, which effectively represents a long series of OpenGL commands, or rather, the result of these commands: a graphical object. All you get back to identify the graphical object is an integer, so you use that integer to reach across the CPU-GPU bridge and paint the object with glCallList(shape). In your tidy object-oriented code, the Java-side proxy for the graphical object contains the shape integer as part of its state.

For one Fluidiom rendering I wanted blimp-like oblong ellipsoids, so I created a list that contains a sphere. The matrix operations in the above code snippet stretch it, twirl it, and toss it out into space before anybody ever sees the sphere.

In true object-oriented tradition, I needed to be able to plug in different shape objects to render intervals on the fly, so I abstracted the interface needed for this:

public interface IntervalShape
{
  float RADIANS_TO_DEGREES = 180f/(float)Math.PI;
  void init(GL theGL, float theRadius);
  void display(
    GL theGL,
    Interval theInterval,
    float theIntervalRadius,
    FabricOccupant theOccupant
  );
}

The init() method gives the implementation a chance to create its OpenGL lists in the GPU and hang on to the integers that represent them, and the display() method does the matrix magic and then calls the list using these shape integers. An IntervalShape is one of these bridging objects that has part of its state (the list) living in the GPU.

There are some strange extra things in the display method that help in setting up the matrix, and the FabricOccupant (which represents you, the viewer) is included for a very special reason. When you want an object to appear smooth in OpenGL, it should have lots of polygons, but too many polygons is inefficient. Long ago, graphics coders figured out the idea of level of detail, which lets you display things in more detail if the viewer is closer, and use simpler versions of the objects if the viewer is far away. The distance from an interval to the FabricOccupant is used for this.

Fluidiom intervals are also supposed to show the viewer how much stress they are under, so there is a list stored for each of the different colors to represent stress (from red=push to blue=pull). The plot thickens.

Putting It All Together

So far, you have an idea as to what is to be displayed (the intervals) and the JOGL code needed for creating the efficient GPU-side lists that are eventually painted (the various IntervalShape implementations). The only important thing missing is the JoglFabric.

The job of the JoglFabric is to observe the fabric and represent it to JOGL through delegated calls to the various IntervalShape implementations. Each visible thing in the fabric is mirrored by an inner class bridge object in the JoglFabric. For example, there is an inner class called IntervalBridge:

private class IntervalBridge {
  protected Interval interval;

  public IntervalBridge(Interval theInterval) {
    interval = theInterval;
  }

  public Interval getInterval() {
    return(interval);
  }

  public void display(GL theGL) {
    intervalShape.display(
      theGL,
      interval,
      intervalRadius,
      fabricOccupant
    );
  }
}

This is an inner class, because that makes it easy to deal with the IntervalShape, which happens to be global, as well as making other global variables available: the intervalRadius and the fabricOccupant. At the same time, the call to display is given the bridge's own interval reference.

At the JoglFabric level, callbacks from observing the fabric are received, and they result in additions and removals to the mirrored inventory of IntervalBridge objects, which are stored in a java.util.Map. When a change happens at interval level, we can quickly look up its proxy here using the map and reflect the change.

Conclusions

Obviously, things get a little dirty when you have to dig in a program against a procedural API from within an object-oriented language, but there's nothing like heating up the transistors on your graphics card a bit with Java code. JOGL lays OpenGL out as flat as can be, and leaves it up to you to make your code as tidy and object-oriented as you please. All you have to do is imagine that part of your object's state is over there on the graphics card instead of in your VM.

It's a little annoying that the JOGL classes have enough methods to actually hang your development environment if they try code completion, but you get used to avoiding that problem. It also takes a little while to learn how to interpret the vast quantity of OpenGL tutorials and code snippets out there on the Web and put them into JOGL form, but you get better at it.

Java3D was doing far too much calculation before talking to the graphics card. The jump to JOGL is all that I needed to get the kind of everything-in-motion performance that I needed to display the evolved running Fluidiom [16] creatures, and Java Web Start was there to make deployment a breeze.

Resources

• Fluidiom Project [16] at SourceForge.
• Download [17] the code or get it from CVS [18].
• Fludiom Code mailing list [10] where you can meet the people involved.
• JOGL [7] at java.net.
• OpenGL.org [19]: the OpenGL community.
• NEHE Productions [20] has some popular tutorials.
• Game Tutorials [21] has some more on OpenGL.

Gerald de Jong [1] is a senior Java coach/architect/programmer in the Netherlands with his own consulting practise, Beautiful Code BV.

Related Topics >> GUI [22]   |