Chet Haase investigates one of Mustang's most interesting desktop features: subpixel rendering, also known as LCD anti-aliasing.
Mustang text smoothing up close
Anti-aliasing helps with the appearance of text, but on LCD monitors you can do even better: you can use the spatial arrangement of the red, green, and blue parts of each pixel to achieve an even better anti-aliasing effect. This feature is coming to Java in Mustang, and in this article, Chet Haase explains how it works.
This article represents a slight divergence from typical java.net
articles, or even my articles: it covers generic technology that is
not specific to Java 2D, or even the Java platform. But because we
just integrated this capability into Mustang, and since it is such
a visible change, and maybe because it is such a cool graphics
trick, I thought it would be worth going into a little more detail
on the subject. For this article, I will confine myself to talking
about the fundamental technology that allows it to work.
Now Available in Mustang: LCD Text Support!
LCD text is anti-aliased text that takes advantage of the
properties of LCD displays to render better-looking, smoother text.
Also known as "subpixel rendered text," LCD text is an algorithm
for performing anti-aliased text rendering on LCD displays. LCD
displays are, of course, the flat panels used in laptops and in the
increasingly prevalent flat-panel monitors.
LCD text support was recently checked into
"http://mustang.dev.java.net">the Mustang release . I posted a
blog entry about it titled "
Phil's Font Fixes ," with discussion from 2D's own font engineer
Phil Race. That blog entry talked about the new rendering hints for
enabling LCD text, about how the new capability is used in Swing's
native and cross-platform looks and feels, and about Swing's new
tracking of desktop preferences to make Swing application text look
more like native application text. There are comparison images of
Swing versus native application text (for both Windows and Gnome),
as well as a Swing versus Swing comparison, with and without LCD
All very interesting stuff, but but but ... how does it
Excellent question; thanks for asking! Stay with me, and
hopefully this article will answer your questions. I won't repeat
the details about the Mustang feature here. If you want more
information on how you can use LCD text in the Java platform, check
out the blog entry.
What is Anti-Aliasing?
First, let's take a small step back and talk about
anti-aliasing (AA). Anti-aliasing is the term used for smoothing
the "jaggies" that result from rendering text or images onto screens with discrete
pixel values. For example, when we draw a diagonal line on a
monitor, we turn pixels on and off (it's not just a continuous
stream of color). If you look closely, you can see the stair-step
effect as we set the color of pixels along the slope of the line,
kind of like Figure 1:
"Pixels of a diagonal line" />
Figure 1. Pixels of a diagonal line
Note that the line above does not flow continuously down the
screen, but just turns on its pixels by coloring a pixel in the
right area according to the slope of the line. It doesn't look much
like a line here, but if you back up a bit and squint, you can
start to see it.
Lines on the screen are exactly the same as the image above,
only smaller. They look better than the image above because the
density of the pixels is such that you do not have to back up or
squint to see the line; the combination of the small pixel size and
your viewing distance from the screen makes it work. If you scoot
forward close enough, you will see the individual pixels of
onscreen lines similar to the example above.
Although lines and other graphics primitives (such as text, just
to relate this back to the current article) are usually "good
enough," there are jaggy effects that can be improved upon. We
can improve things by using anti-aliasing, which smoothes the hard
edges of rendering primitives to reduce the on-or-off
characteristic of the jaggies. In anti-aliasing, we basically ramp
up to the full color value of the primitive, and then ramp back
down. That is, we blur the edges a bit with the background color to
remove the hard-edge effect of non-anti-aliased primitives.
For the rest of this discussion, I will focus on just one
individual pixel; pretend that it is part of a larger line or
character that we are rendering. The principles involved are the
same regardless of the number of pixels, but it's a bit easier to
see what's going on if we narrow this down a bit.
First, consider a simple jaggy (aliased) version of one of
the pixels from the line above. Figure 2 shows what that looks
Figure 2. A single aliased pixel
Figure 3 shows a simple anti-aliased version of that same
"A single anti-aliased pixel" />
Figure 3. A single anti-aliased pixel
Note the effect above; we are still drawing the main pixel in
the same location as before, but now we have a wider area that is
grayed out at the edges (using a gray value that is half the
intensity of the black color, in this example). The result that the
user will see is that the line is smoother and blends in better
with the background colors, because each of the individual pixels
has a smoother transition from the background color to the line
I used a line example above because it is easier to demonstrate
anti-aliasing with that simple primitive. But you can do the same
thing with text: just imagine every single pixel value in each
character as being smoothed out by having bordering pixels blurred
between the character color and the background color.
This is all basic anti-aliasing technology. We've had that
capability in Java 2D since version 1.2; you just have to set the
RenderingHint to enable it. So what's the deal with LCD
The Deal with LCD Text
When you add extra bordering pixels to text, even when you blend
those colors with the background color, you run the risk of having
an overall "blurry text" effect; the characters get too wide and
indistinct, and the smoothness of the text is not a good tradeoff
for the accompanying blurriness. In fact, you can see this tradeoff
at work in various platforms by seeing how desktops manage
anti-aliasing. On some platforms, when you tell the desktop that
you want anti-aliasing, it will go ahead and do AA rendering on
many characters, but some fonts at some sizes will still be
rendered non-AA. In particular, small font sizes are usually
rendered non-AA because the area of color ramping causes the stems
of the glyphs to run into each other, often resulting in illegible
text. Think about it: the smaller each character is, the larger
each pixel is in proportion to the character, and the larger the
area of color ramp-up is comparatively.
How do you get the great smoothing effects of anti-aliasing while
avoiding the "my text is blurry!" effect that AA sometimes
Enter LCD text. (And you wondered whether I'd ever get back to
the article subject, didn't you?) LCD text is basically a way to
use the physics of the LCD screen to get three subpixels out of
each physical pixel (in the horizontal or vertical direction).
What? How can you get more pixels than are actually
LCD text uses the color striping layout of the monitor to do
this. Every LCD screen uses red/green/blue stripes, in RGB or BGR
order and in the vertical or horizontal direction. It's true; check
it out. Get realllly close to your LCD monitor (my lawyer
advises me to tell you to check with your doctor before doing this
for an extended period of time) and you can see the stripes. Look
at a white pixel; it illuminates all of these stripes
simultaneously. You should see something like Figure 4:
"Color stripes of an LCD pixel" />
Figure 4. Color stripes of an LCD pixel
Each "pixel" on the display is actually composed of a square
containing red, green, and blue stripes. For example, the RGB
stripes above represent three pixels. To make this a little
clearer, Figure 5 the same image with white lines dividing the
"Color stripes of an LCD pixel with spaces between pixels" />
Figure 5. Color stripes of an LCD pixel with spaces between
Note that this is only the case for LCD monitors; on a CRT
display, the RGB values are usually arranged in a triangle for each
pixel; similar visual effect, but very different physical
Now, if you ignore the colors in each of these stripes, you can
think of each stripe as a subpixel. So instead of each pixel being
one physical pixel, it's actually three subpixels (wide or high,
depending on the striping direction on your monitor).
Given more pixels we could have a smoother ramp, right? First,
let's visualize these subpixels in terms of the original jaggy
pixel (Figure 6):
Figure 6. The original jaggy pixel
Note how each physical pixel now takes up nine subpixels in this
LCD pixel space.
With our previous anti-aliasing algorithm, this subpixel-space
would look like Figure 7:
Figure 7. Initial anti-aliasing
Let's suppose that we have vertical striping on our display.
How can we use that information to get better AA? How about having
a more gradual ramp up to the full pixel color values? Figure 8
shows this approach.
"Gradual anti-aliasing with vertical stripes" />
Figure 8. Gradual anti-aliasing using subpixels
Now, you can see (if you squint) that in the same space as
before (one pixel), we've effectively done a smoother ramp up from
no-line to full-line color. Whereas before we had a simple
one-color ramp up and down around the true line color, now we have
a more gradual ramp up through the subpixels of the stripes.
Hopefully you can see where it's a Good Thing to have these
subpixels and that we get better smoothing out of using them.
But the observant reader may, at this point, balk and say
"Hey, they ain't no subpixels. They're colors! This is going
to look ridiculous on the display!"
How Do RGB Stripes Become White Pixels?
This is where we get into how the eye sees, and how the LCD text
rendering algorithm takes advantage of that system to be able to
get the smoothing effects we want above while actually using
different colors for each of these pixels.
I think the best way to explain this would be to take the high
road and use Art, with a Capital A, as an example. There was a
movement in the Art community in the late 1800s called
Pointillism, which exploited the same effect of our visual
system. A great example of this is George Seurat's
"http://www.artic.edu/aic/collections/eurptg/28pc_seurat.html">A Sunday on La Grande Jatte-1884 . The Pointillists
worked by using distinct dots of primary colors instead of blending
the colors on their palettes. The effect up close is odd; you see
all of these individual colors on the canvas, but have a hard time
making them work together to create the true colors of the overall
picture. But step back from the painting and it all works; the
individual colors blend together in your eyes to create new colors
that are combinations of the individual colors used in the dots.
The amazing thing to me about these paintings is: how did the
artists create them when they necessarily had to be close to the
canvas to apply the paint? How can you paint a picture if you
cannot actually see the picture as you go? Maybe it's like
Beethoven composing his Ninth Symphony while deaf. Or Fonzie
putting that motorcycle back together when he was temporarily
blind. True artists operate beyond the senses.
One of the reasons that the painters took this approach (apart
from the usual "I'm an artist, I have to do something different"
mentality that also pervades engineering) was because they could
get a much brighter result from this use of color than they could
with the traditional approach. This is because they were using
additive mixing of colors to achieve the new color blends of the
individual dots. Traditional color mixing, actually stirring the
paints together, tends to dim the contributed colors and make the
result darker overall than the individual components (this type of
color mixing is referred to as subtractive mixing). However, if
you can keep the intensity of the original colors and yet still get
a color blend, then you will have an overall brighter effect.
In this additive mixing approach, painters were relying on the
capability of our eyes to blend distinct colors into totally new
ones. For example, a red, a green, and a blue dot all next to each
other will create the net effect of a larger white dot when seen
from an appropriate distance. And that white dot will have all of
the intensity/brightness of the component colors.
Note my use of red, green, and blue for the example in the
previous paragraph. Do those colors ring a bell? Of course they do;
those are the component colors on computer displays (both
traditional CRT monitors and the more recent LCD displays).
Displays use exactly the same additive mixing principle to be
able to display different colors by depending on our eyes to blend
the red, green, and blue component colors into other colors, when
viewed at an appropriate distance from the screen. When they are
all on at full intensity, we see a white pixel. When only one of
them is on and the others are off, we see just that component
color. And when there are other combinations of these colors, we
see other colors appropriately.
Putting it Together: LCD Text's Use of Additive Color
How does all of this relate back to LCD text? Well, recall
that we glossed over the fact that our smooth ramp up to the line
color went through individual and different-colored stripes, and
pretended instead that we were just dealing with black, white, and
shades of gray. If we go back to the subpixel AA example above and
look at the values of those shades of gray cast in terms of each
color stripe, we will get a more accurate picture of what's going
on at the pixel level.
First off, let's redo the image above using red/green/blue
stripes, just like an LCD monitor would (here, we assume that the
display has vertical red/green/blue striping). See Figure 9:
"Anti-aliasing with red/green/blue stripes" />
Figure 9. Anti-aliasing with red/green/blue stripes
The floating point values here are the intensities of each
stripe, the same as the gray stripes in the example above.
R G B R G B R G B
.75 .5 .25 0 0 0 .25 .5 .75
This makes sense in the smooth ramp up sense, as the intensities
move from full-on (the background color) to full-off (the
foreground color of black). But now, let's reconsider the above
pixels in terms of the RGB triples involved. We have the pixel on
the left with the RGB color (.75, .5, .25), the black pixel in the
middle with RGB values (0, 0, 0), and the pixel on the right with
the RGB color (.25, .5, .75). If we display this as actual colors,
we would get something like Figure 10:
"Actual colors of anti-alias pattern" />
Figure 10. Actual colors of anti-alias pattern
Hmmm. Is this actually going to work?
Well, yes--and no:
- Yes: We have already seen above how the eye is able to
blend distinct color components into other colors through additive
mixing. After all, a white pixel, consisting of red, green, and
blue components, does not look like a green pixel flanked by red and
blue pixels, right? So in theory, it is possible to have the right
effect while having obvious color edges to our primitive.
And in fact, if you look closely at any LCD text rendered on
your display, you will see this exact colored edges effect. For
example, Figure 11 is a screenshot of an "F" in both normal and
close-up views on my current system:
"Close-up view of anti-aliasing" />
Figure 11. Normal and close-up views of anti-aliasing
Notice the funny color modulation taking place, with the colors
moving from yellow to red to blue and then cyan as we go from left
to right through the pixels of the character. And even more
fascinating, from a "How do my eyes do that?" standpoint, is
the fact that the only true black color in this character is on
long horizontal runs; our eyes combine the other colors and somehow
come up with black where there is none.
And No: This simple approach to the solution is not
sufficient to give us the right effect; it is too simplistic and
produces color differences that are too stark to be compensated for
by our visual system. Instead, the actual calculation of the color
ramp up and down is more involved (and beyond the scope of this
article) and results in more gradual shifts in color that work
better with our additive-mixing vision. In the "F" character above,
we can see this in the way that there are actually four
pixels/colors for the vertical bar of the F; this allows a more
gradual shift from the white background up to the true foreground
color and back to the background color.
Check out the LCD text capabilities in
"http://mustang.dev.java.net">Mustang . I think you'll like the
results. But even better, I think you'll like how your eyes can
somehow magically blur the odd color effects used in LCD text to
make smoother uniform colors and smoother characters. You gotta
love physiology. Oh, and graphics hacks.
width="1" height="1" border="0" alt=" " />