Published on Java.net (http://www.java.net)

Interaction Happens: Thinking Graphically

In this installment of "Interaction Happens," Jonathan Simon shows how to "think holistically" about your GUI.

Think visual!

July 12, 2005

Jonathan Simon [1]

Some applications make perfect sense to the developer, but not to the end user. Are you thinking of your GUIs in terms of what the developer needs, what the application needs, or what the user needs? In this installment of "Interaction Happens," Jonathan Simon shows how to "think holistically" about your GUI.

Contents

Example 1: Address Detail Panel

Implementation

Example 2: Frame Title Abuse! [3]

Implementation [4]

Example 3: Information Messages [5]

Writing Your own Message Bubble [6]

Wrap-Up [7]

Props

Resources

Making a polished user interface is hard work. Interaction design takes experience and time, which are usually inconceivable luxuries. And that's before we mention the technical limitations in Swing, which we are all too familiar with. On top of that, programming and interaction design are often at direct odds with each other. Making a polished, functioning interface can take much more code (and much more complicated code) than a less-polished alternative. Similarly, developing with graphics toolkits and widgets often forces the implementation to use a less than perfect solution from an interaction perspective. The combined effect of these forces is a difficult playing field where we try to solve these problems and develop a product.

This article takes a holistic view at the problem space. We'll take a look at three different user interface design issues from an interaction perspective. Following that, we'll look at an interaction design solutions to these problems that I've used and seen others use on countless projects. Finally, we'll implement them in Swing, exploring the technical details of implementing these designs. The goal is to understand, through practical examples, that there is a single problem space from interaction design to code. Along the way, you'll also see some code examples of Swing techniques you may not have used before, but might find interesting (think transparent painting and rounded borders).

Example 1: Address Detail Panel

A common way to display detailed information is to create a panel with one label per field, and the field displayed alongside it. Figure 1 shows an example of one of these panels for address details of someone named "What A. Guy."

Figure 1. Standard details panel displaying an address

Although it's easy to create a one-to-one mapping between the address fields and this panel, there are a number of interaction issues with this design.

Difficult alignment: In this example, the field names are all right-aligned, but their lengths vary. Some field names have more space between the end of the field name and the beginning of the value, making it difficult to correlate the field name and the value. Some interaction designers prefer to right align the field name and left align the value, but then your detail panels start to look an awful lot like ASCII Christmas trees. You can improve the alignment significantly by standardizing the field name length (say, between five and ten characters) but it's still going to be awkward.
Waste of space: As explained above, the field names are different lengths, and you have to make the field labels wide enough to accommodate the widest field name. Some of the fields will undoubtedly be shorter. So in addition to making the alignment difficult, it's a waste of screen real estate. Save the pixels!
Non-dynamic: These displays don't usually adjust themselves for the information they are displaying. For example, if there is no middle initial, the value will simply be blank. But like looking through a database table, the blank fields are always there--you'll always have the field label there, and the space is still taken up. And if you try and make it dynamic, the screen is very noticeably changing all the time.

On the upside, these panels are pretty easy to build. You can even generate them automatically from a data object using reflection or a field mapping. This is a great example of where interaction design goals differ from coding simplicity. Even though they might be difficult from an interaction perspective, they do have some redeeming features. But we're here to focus on interaction and work towards the code, so let's see if we can do a little better.

Figure 2 shows a more graphically oriented representation of the same address.

Figure 2. Graphical address

The formatting in this panel is very close to how you would write or print the same address. This allows you to remove the labels, saving screen real estate, simplifying alignment, removing clutter, and more. It's easy for users to tell the information from the context. For example, addresses in the United States are always street address, then city <comma> state <space> zip code on the following line. Quickly glancing for the comma and looking after it is a quick way to find the zip code. So you don't need a label marked "Zip Code" as in Figure 1. Other good examples of things that don't need labels would be email addresses and websites. As soon as someone sees "http," they know it refers to a website. The same applies to the "@" in an email address. In those cases, the label is simply redundant and potentially distracting.

But sometimes you do need labels. As an example, the last line shows two phone numbers. Cell phones and home phones have identical formatting, so there's no way to tell the difference by context alone. Yet this doesn't mean that you need to have full labels that say "Home Phone" and "Cell Phone." Its already noticeable that they are phone numbers due to the formatting, so you can eliminate the "Phone" from the label. Also, there are a limited number of types of common phone numbers--home, cell, office, etc. Notice that the phone labels in Figure 2 only have a single letter, "C" or "O," for home or office.

I can't stress enough how careful you have to be with this type of logic, however. It's important to analyze the data you're working with before doing a graphical display panel. You need to ask yourself (and your users, if possible) whether they can figure out the data from the context. For example, I built a trading system and used these types of panels to display bond information. Bonds are often identified by a combination of three data elements: the issuer, the coupon rate, and the maturity date. Since these traders deal with this information all day long, you can display the ticker, coupon, and maturity in order--"F 2.7 2010," for example--and expect these users to immediately understand it.

Implementation

There are a couple of different ways to implement this. You could use a series of JLabels and other text widgets, or you could use a JTextPanes with styled text. We'll implement a later example with a JTextPane and styled text, so let's implement this example with a series of JLabels for variety. Here is the code to set up the JLabel-based GraphicalAddressPanel, starting with the titled border and label instantiation.


public GraphicalAddressPanel() {
    setBorder(BorderFactory.createTitledBorder("Address Details"));
    nameLabel = new JLabel();
    addressOneLabel = new JLabel();
    addressTwoLabel = new JLabel();
    phoneLabel = new JLabel();
        
        ...
}

Next, we'll continue the constructor implementation with the layout. Since the top label is going to be larger (because of the font size) we can't use a simple GridLayout, which would equally space all of the components. Instead, we'll use a few panels and nested layouts. Start by setting the panel layout to BorderLayout and adding the name label to BorderLayout.NORTH.


setLayout(new BorderLayout());
add(nameLabel, BorderLayout.NORTH);

Now, make a subpanel with a GridLayout, named detailsPanel, for the remaining detail labels. Add the remaining labels.


JPanel detailsPanel = new JPanel(new GridLayout(0,1));
detailsPanel.add(addressOneLabel);
detailsPanel.add(addressTwoLabel);
detailsPanel.add(phoneLabel);

Next, create a panel named bottomPanel with a BorderLayout. Add detailsPanel to BorderLayout.NORTH and add it to the main panel. This is a quick and dirty method of using nested panels and layouts to keep the labels oriented to the top.


JPanel bottomPanel = new JPanel(new BorderLayout());
bottomPanel.add(detailsPanel, BorderLayout.NORTH);

add(bottomPanel, BorderLayout.CENTER);

Finally, set the fonts of the labels. This uses the deriveFont() helper method of Font. It creates a new Font with the same style as the old Font with a few changed parameters--very useful.


nameLabel.setFont(
    nameLabel.getFont().deriveFont(Font.BOLD, 18));
addressOneLabel.setFont(
    nameLabel.getFont().deriveFont(Font.PLAIN, 12));
addressTwoLabel.setFont(
    nameLabel.getFont().deriveFont(Font.PLAIN, 12));
phoneLabel.setFont(
    nameLabel.getFont().deriveFont(Font.PLAIN, 12));

Now to display the address. Create a method called displayAddress(). This method is pretty lightweight. We'll just use it to forward off the request for each line to a helper method. This method takes in an address data object called AddressDO. This is a simple data object with getters and setters for each field.

 
public void displayAddress(AddressDO addressDO) { 
    nameLabel.setText(getName(addressDO)); 
    addressOneLabel.setText(getAddressOne(addressDO)); 
    addressTwoLabel.setText(getAddressTwo(addressDO));
    phoneLabel.setText(getPhone(addressDO)); 
}

Let's take a look at the getPhone() method as an example of the type of dynamic formatting you'll do with these types of display panels. The logic is straightforward--if the home phone number exists (i.e., it is non-empty and non-null), add the "(H)" indicator and the home phone number. Do the same for the cell phone number. If we didn't do this, we would display labels for phone numbers that don't exist.


private String getPhone(AddressDO addressDO) {
    StringBuffer buffer = new StringBuffer();

    if (addressDO.getHomePhone() != null
            && addressDO.getHomePhone() != ""){
        buffer.append("(H)");
        buffer.append(addressDO.getHomePhone());
        buffer.append("  ");
    }

    if (addressDO.getHomePhone() != null
        && addressDO.getHomePhone() != ""){
        buffer.append("(C)");
        buffer.append(addressDO.getCellPhone());
        buffer.append("");
    }

    return buffer.toString();
}

It's pretty easy to take it to the next level from here. If you have your application deployed to multiple regions, you can implement a display decorator to format names and addresses as appropriate for current locale. You can also add and remove fields at will. The end result is that it's easier to change the formatting and display without a rigid label/value display like Figure 1. At the same time, you conserve a lot of screen real estate and can make the display more intuitive to your users at the same time.

Example 2: Frame Title Abuse!

Frame titles are meant to label your application as a whole. It's the single reference to your application as an entity. Its also used by Windows, for example, in the task bar to identify your application. In other words, it's a bad idea to cloud the frame title bar with useless information.

That being said, it also happens to be a very visible place to put essential information. But I've seen a number of applications that commit something I'll call frame title abuse. This is where an excessive amount of information is crammed into the title bar to make the information visible. This can include server connection information, build numbers, environment information, and more. It gets pretty ugly. Figure 3 shows an unfortunate example.

Figure 3. Long frame title

There are a couple of fundamental problems here. First and foremost is that the unnecessary information is confusing the important information--the application name--in the title bar. This is not to say that you shouldn't put other information in the title bar. Web browsers, for example, usually put the web address your browser is viewing in the title bar. This makes a lot of sense. But environment and other system-specific information does not belong there!

The other problem with adding information to the title bar is that it's too easy to leave it in when your application is used by real live users. It becomes second nature to testers and developers to see this extra information--so you simply forget to take it out until it's too late. I heard a story about internal names for software: it was suggested that internal names for systems be restricted to bodily fluids (i.e. "bile") so there would be no hope of something so hideously named getting released.

In that spirit, Figure 4 shows an environment name watermark transparently displayed across the application. Watermarks are used in print all the time and work very well for electronic documents and software as well.

Figure 4. Transparent watermark

The theory behind this is that it's immediately obvious that you are in a test environment (because otherwise you wouldn't see the watermark) and it's clear which environment you're in, for example, if you have several. Also, notice that it doesn't interfere with any functionality for your system--user-relevant information on the title bar, for example, would function identically with or without the watermark. Finally, it's jarring and obvious enough that you would never release it for real! Precisely what we're looking for.

Implementation

We'll implement the watermark by creating a custom component that paints the transparent text across a window. We'll make this component generic enough so that it can be easily added to any Window. Start by creating a class called JWatermark that extends JComponent.

You'll need variables for the font and the watermark display string for reference. You'll also need a couple of constants for the rotation and opacity (how much you can see through the text, which we'll get to in a bit).


private static final float OPACITY = 0.15f;
private static final double ROTATION = -(Math.PI / 4);

private Font font = UIManager.getFont("Label.font");    
private String text = "";

Here's the constructor that caches the display string.


    public JWatermark(String text) {
        this.text = text;                         
    }

The heavy lifting for the JWatermark component is in paintComponent(). There we'll call drawString() with the opacity and rotation properly set, as well as proportional size and location. The paint method starts with creating the virtual drawing space--a rectangle half the width and half the height of the window, centered.


Window window = SwingUtilities.getWindowAncestor(this);
Rectangle viewRect = window.getBounds();
int halfWidth = viewRect.width / 2;
int halfHeight = viewRect.height / 2;

Next, set the AlphaComposite on the Graphics2D instance. The AlphaComposite is used to set the opacity--in other words, to make the text semi-transparent. Notice the opacity is set to 0.15, or 15 percent. You can change this if you like, but I find that to be a good balance between visibility and distraction when running applications with watermarks.


Graphics2D graphics2D = (Graphics2D) g;
AlphaComposite ac = AlphaComposite.getInstance(AlphaComposite.SRC_OVER, OPACITY);
graphics2D.setComposite(ac);

Next, set the font proportionally to the screen size. To do this, find the smaller of the width and height and derive a new font whose height is 1/7th the height or width of the screen--whichever is smaller. As with the opacity, you can change the magic number of 1/7th, but I've found this works well on my watermarked applications.


final int minSide = Math.min(viewRect.width, viewRect.height);
font = font.deriveFont((float)( minSide / 7));
graphics2D.setFont(font);

This next bit is a little bit of magic code that calls SwingUtilities.layoutComponentLabel to determine the painting rectangle of the text. Note that the Rectangle variable textRect is not returned from the method. Rather, it is mutated inside the method itself, since Java can only return a single object.

Rectangle textRect = new Rectangle();

graphics2D.setFont(font);
graphics2D.rotate(ROTATION, halfWidth, halfHeight);
SwingUtilities.layoutCompoundLabel(this,
      graphics2D.getFontMetrics(), text, null,
      SwingConstants.CENTER, SwingConstants.CENTER,
      SwingConstants.CENTER, SwingConstants.CENTER,
      viewRect, new Rectangle(), textRect,
      0);

Now you can paint the String. Again, there are some magic numbers in here that work nicely in my applications, but feel free to change them. They are used to roughly center the watermark in the window.

graphics2D.setColor(getForeground());
int string_x = halfWidth - (int) (textRect.width / 2);
int y = halfHeight + textRect.height / 7;
graphics2D.drawString(text, string_x, y);

Finally, you need to install the watermark. Rather than make all of your windows add the watermark separately, let's just make a static helper method to add it. In order for it to keep from interfering with window functionality, we'll put the watermark layer one layer above the content layer. This means that the watermark will show up in front of your components in the window (so you can see it) but behind higher-level layers, so it does not affect things like pop-up menus or drag-and-drop. Layers are added by an index indicating priority, with constants for standard layers. You can add a new layer by giving your layer a new index. To make this layer just higher than the content frame layer, get the constant for that layer and add one.

public static Integer WATER_MARK_LAYER 
    = new Integer(JLayeredPane.FRAME_CONTENT_LAYER.intValue()
                  + 1);

With the correct layer in hand, here is the rest of the helper method code to set the bounds, color, and opacity of the panel. Notice that the watermark is added to the layered pane with the WATERMARK_LAYER_INDEX just above the content layer.


public static void createWatermark(JFrame frame, String text){
    JWatermark watermark = new JWatermark(text);
    watermark.setOpaque(false);
    Dimension screenSize = 
        Toolkit.getDefaultToolkit().getScreenSize();
    watermark.setBounds(0, 0,
            (int)screenSize.getWidth(),
            (int)screenSize.getHeight());
    watermark.setVisible(true);
    watermark.setForeground( Color.BLUE );

    final JLayeredPane jLayeredPane = frame.getLayeredPane();
    jLayeredPane.add(watermark, WATER_MARK_LAYER, 0);
}

We'll wrap up this example with the code for the WatermarkSimulator showing the usage of JWatermark. Using the watermark is a one-line addition to the creation of the frame: just call the static method createWatermark().


JFrame frame = new JFrame(...);
JWatermark.createWatermark(frame, "Development");

Here is the complete code for the WatermarkSimulator.


public class WatermarkSimulator {

    public static void main(String[] args) {
        JFrame frame = 
            new JFrame("Java Application Cache Viewer");
        JWatermark.createWatermark(frame, "Development");

        frame.setLocation(200,200);
        
        frame.getContentPane().setLayout(new BorderLayout());
        ImageIcon icon =
            new ImageIcon(
              WatermarkSimulator.class.getResource("javaws.gif"));
        JLabel label = new JLabel(icon);
        frame.getContentPane().add(label, BorderLayout.CENTER);

        frame.setSize(new Dimension(510, 450));

        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);
    }
}

Example 3: Information Messages

The third and final example we'll look at deals with informational messages. Figure 5 shows WinCVS with a command-line-esque information log.

Figure 5. Command-line-esque display

This is a pretty familiar paradigm for programmers and old-school command line users. It tells you what it's doing and keeps a log of your history. This is great for programming tools--very informational--but I wouldn't put something like this in front of a non-technical user. To those users, it looks like a lot of technobabble and is potentially confusing, due to the additional information of the log as well as the display style itself. I can only imagine what my father would think if Microsoft Word had a command-line log of all of its actions at the bottom of the screen!

That said, there are definitely some redeeming features. The situation shown here is a failure to connect to a server. Many applications would show an error dialog on a connection failure. Alan Cooper, Jeff Johnson, and others point out the problems with dialogs, ranging from irritation from interruptions to difficulties with resolution instructions. Figure 6 shows an interesting middle ground--a message bubble panel.

Figure 6. A message bubble

The idea behind the message bubble as a status indicator is simple. First, it's easy to see what's going on. There is no history to cloud the user's view like there is with the command-line display. Likewise, the status won't keep the user from interacting with the rest of the system like a dialog box would. That also means that you can put instructions in the status bubble that you can't in a dialog. On this point, Jeff Johnson makes reference to instructions on a helicopter door for ejecting. The first instruction is to release the door, thereby removing the rest of your instructions. Dialogs with instructions are the same thing. As soon as you hit OK, the instructions are gone.

Writing Your own Message Bubble

The strategy here is that we'll make our own panel that has a bubble in it. This will require a little custom painting to create the rounded border effect and a little tweaking of the panel's insets. Once that's done, you can set layouts and add components just like any other panel. We'll implement this in a class called JBubblePanel.

Start off by declaring a couple of constants: colors for the yellow bubble background and gray bubble border, margins for the top and bottom, and the height and width dimensions for the arc (rounded corner) in the rounded border.


private static final Color YELLOW_SNOW =
    new Color(255, 255, 204);
private static final Color BUBBLE_BORDER = Color.GRAY;

protected static final int X_MARGIN = 4; 
protected static final int Y_MARGIN = 2;

private final int ARC_WIDTH = 8;
private final int ARC_HEIGHT = 8;

Next, override all of the constructors and add call an init() method. The init method adds the empty border around the panel. This empty border makes sure there is space around the bubble panel.


public JBubblePanel() {
    super();
    init();
}

... other constructors ...

protected void init() {
    this.setBorder(BorderFactory.createEmptyBorder(3, 3, 3, 3));
}

Since we added a border around the bubble panel, we need to tweak the insets so that components added to the panel's edge are still inside of the bubble inside of the panel. We'll make a helper method to get the insets from the superclass. Then we'll override the getInsets() method to adjust the insets within the panel. Here is the code to tweak the insets.


protected Insets getRealInsets() {
    return super.getInsets();
}

public Insets getInsets() {
    Insets realInsets = getRealInsets();
    Insets fakeInsets =
        new Insets(realInsets.top + Y_MARGIN,
                   realInsets.left + X_MARGIN,
                   realInsets.bottom + Y_MARGIN,
                   realInsets.right + X_MARGIN);

    return fakeInsets;
}

And as usual, you'll need to override the paintComponent() method. The important code here is in the fillRoundRect() and drawRoundRect() calls. The round rect methods are used to paint rectangles with rounded corners. The fillRoundRect() method draws the yellow bubble itself, while the drawRoundRect() method draws the border.


protected void paintComponent(Graphics g) {
    Graphics2D g2d = (Graphics2D) g;
    Insets insets = getRealInsets();
    Color savedColor = g2d.getColor();

    int rectX = insets.left;
    int rectY = insets.top;
    int rectWidth = getWidth() - insets.left - insets.right;
    int rectHeight = getHeight() - insets.top - insets.bottom;
    
    // Paint the yellow interior
    g2d.setColor(YELLOW_SNOW);
    g2d.fillRoundRect(rectX, rectY,
        rectWidth, rectHeight,
        ARC_WIDTH, ARC_HEIGHT);

    // Draw the gray border
    g2d.setColor(BUBBLE_BORDER);
    g2d.drawRoundRect(rectX, rectY, 
        rectWidth, rectHeight,
        ARC_WIDTH, ARC_HEIGHT);

    g2d.setColor(savedColor);
}

Usage of the JBubblePanel is very straightforward: you just treat it like any other panel. Here is a code sample from the BubblePanelSimulator, which adds a JTextPane to the JBubblePanel. It's exactly the same as any other panel. All of the code to add insets and spacing is encapsulated in the JBubblePanel itself.


JBubblePanel bubblePanel = new JBubblePanel();
JTextPane textPane = new JTextPane();
bubblePanel.setLayout(new BorderLayout());
bubblePanel.add(textPane, BorderLayout.CENTER);

Let's wrap things up with the complete code for the simulator. This is also a good example of a JTextPane. Note the use of the AttributeSet along with the string to format text. Also, note that text is added to the JTextPane by retrieving the Document and adding the text to the Document, not to the JTextPane itself.


public class BubblePanelSimulator {

    public static void main(String[] args) {

        JFrame frame = new JFrame();
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        JBubblePanel bubblePanel = new JBubblePanel();
        JTextPane textPane = new JTextPane();
        bubblePanel.setLayout(new BorderLayout());
        bubblePanel.add(textPane, BorderLayout.CENTER);

        SimpleAttributeSet normal = new SimpleAttributeSet();
        SimpleAttributeSet bold = new SimpleAttributeSet();
        StyleConstants.setBold(bold, true);



        try {
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                "Your connection to ",
                normal);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                "cvs.dev.java.net ",
                bold);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                "failed. Here are a few possible reasons.\n\n",
                normal);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                " Your computer is may not be " +
                "connected to the network.\n" +
                "* The CVS server name may be " +
                entered incorrectly.\n\n",
                normal);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                "If you still can not connect, " +
                "please contact support at ",
                normal);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                "support@cvsclient.org",
                bold);
            textPane.getDocument().insertString(
                textPane.getDocument().getLength(),
                ".",
                normal);
        } catch (BadLocationException ex){
            ex.printStackTrace();
        }


        frame.getContentPane().setLayout(new BorderLayout());
        frame.getContentPane().add(bubblePanel, BorderLayout.CENTER);

        frame.setBounds(200,300, 400,360);
        frame.setVisible(true)
    }
}

Wrap-Up

These examples give a good first look at some of the difficulties with interaction design and implementation. The first example with the addresses is a good example of the simple code solution leading to the less-than-ideal interface (the label value display panel), leading to the graphical display panel implementation with more code, but a polished interaction. I can't overemphasize how important decisions like this are for your application. These decisions make all the difference between what looks like a thrown-together interface and a polished one. Even if the interaction is very similar, your users will consciously or subconsciously notice the difference between the frame title abuse and the watermark or status bubble from the third example. Small, well-thought-out changes like these can make all the difference for your application.

And as a final thought, these are just ideas. Take the spirit of these thoughts and apply it to your applications. Maybe some will work, maybe not. Possibly, a variation of one of these solutions might be great for your specific product. Or maybe some of these examples just give you inspiration for something entirely new. However you do it, do whatever it takes to build excellent, polished applications.

Props

Shouts out to Brett Kotch for showing me his thought process on some of these issues, and to Kevin Hein and Rob Linwood for contributing some of the example code.

Resources

Sample code [8] for this article

Jonathan Simon [1] is a developer and author specializing in user interaction.