Skip to main content

Drawing to an off screen buffer

14 replies [Last post]
cbare
Offline
Joined: 2004-02-02

Hi,

I'm trying to build a little program that draws complicated time-consuming plots. I'd like to avoid having my UI freeze up while I'm rendering, so I don't want to do the rendering on the event dispatch thread. So, what's the modern (well, Java 5 level of modern) way of doing this?

First, should I be drawing on a Canvas or a JPanel? I'm using Swing components in the rest of my app and I've read that you're not supposed to mix Swing and AWT, but I can switch if there's good reason.

Next, do I want to use VolatileImage, BufferStrategy, or something else? Can you even get a BufferStrategy for a JPanel? I'd like to get hardware double buffering, if possible.

Last, I assume it's OK to draw on the offscreen image on another thread and then put a runnable on the event dispatch thread to do the blt? Or is that the wrong way to go about it?

Any pointers to the current (JDK1.5) wisdom would be appreciated! Thanks,

-chris

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Peter B. West

java2d@JAVADESKTOP.ORG wrote:
> I'm starting think my problem is knowing what solution applies to my situation. There are plenty of solutions for handling long-running tasks in Swing. (SwingWorker, etc.) There are solutions for double buffering for animations (BufferStrategy). But, I don't think either of those apply to long-running rendering tasks.
>
> Maybe a solution to that would be to drip-feed little subtasks onto the event queue such that there is only one rendering task on the queue at a time and user events will be interleaved in the queue between rendering tasks.
>
> Any other ideas or examples for maintaining a responsive UI in the presence of long-running rendering tasks?
>
> Thanks,
>
> -chris

For complex paragraph layout tasks, the problem I had was to redo the
layout in response to changes in the size of the display panel: to
layout with wider of narrower line widths. Layout takes some time, so I
am currently trying the following.

In the panel (and in the EDT) capture the resize information, storing
the latest size, and trying to start a worker. If a worker is already
active, do nothing.

Otherwise, start a size monitoring worker.

The size monitoring worker in turn invokes another worker, whose job is
to perform the layout. It creates the layout worker, triggers it with
execute(), then blocks on a get() from the same worker.

The doInBackground method returns an object representing the laid out
paragraph. The done() method calls repaint, which collects the layout
data and paints it, according to the size of the panel when the task
started.

Meanwhile, the size monitoring worker has returned from the get(), and
can now check to see whether another size change has occurred.

If so, it repeats the cycle of creating a layout worker.

If not, it terminates, and the system waits on another size change event.

It may seem that the size monitoring worker is redundant, but I found
that it neatly separated the phases. One of the problems, I think, was
that the actual repaint() takes some time itself. Performing the repaint
in the done() method of layout worker seems to work, but ideally, I
would like the size monitoring worker to wait until the repaint was
finished before starting another layout worker, but I can't see an easy
way to do that, while preserving the nice blocking characteristics of
the get() method.

It's a messy explanation, I know, and it was developed by trial and
error. If you need anything clarified, let me know.

--
Peter B. West
Folio

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

rosmord
Offline
Joined: 2006-05-05

If I understand well, what takes time here is the computation of the plot.

Now, have you considered building some kind of representation of the plot (say, in terms of shapes) ? In this case, you would have :

* A thread building an abstract representation of the data
* the graphical thread whose work would only be to draw the said representation in a JPanel or JComponent.

If your plot is really a bitmap, you could consider a thread which would render small parts of it in small bitmap areas which would then be available to the event loop to render ?

ewin
Offline
Joined: 2005-07-15

Well, lets clarify/assume a few things.

You have some data source, constantly producing data, right? While that data source happily produces data the user is also looking at a representation of that data, right? And the user can interactively select (scrolling, zooming) which portion of the data he/she gets a representation of?

Ok, first, I am not a big fan of rendering and rasterizing (drawing to an image) the data in the backgrond in such a case. For at least two reasons: Zooming on already rasterized data doesn't work well, so for zooming it is required to freshly rasterize that data. Second, if the plot is indeed very large you can not fully predict to which area the user might scroll next. So you can't reasonably well guess which area should be preferentially rendered in the background. Because of this two issues, a lot of CPU power used for background rendering is just wasted, since you render something you can't use.

So, instead of rendering in the background, organize/store the incomming data in a way that it is easy to render in the background (in a thread). This, for example, might include to already decompose the data into primitive graphic elements like lines. You don't actually draw such lines, you just store easy to process information that you would draw a line. And provide a locking mechanism on the data, so that the maintenance of the data and the rendering (see below) are properly synchronized.

Further, if the plot is really so large, spatially organize that graphic element data in the background in a way that you can quickly find all data within a particular rectangular 2D region (quadtree). If you have many overlapping graphic elements you might even want to organize the data in three dimensions (octree), so that you can easily cull details when the user has zoomed out very fare (because in this case many details would anyhow be lost when rendering the data).

Now to the user interaction and actual rendering. You create an own Swing component based on a JComponent or JPanel. Creating an own Swing component has often been document, I won't go into this. Just make sure you get the basics and the nastier details right :-) Also implement the Scrollable interface. I assume you put that component in a JScrollPane.

The paintComponent() method of your component will be called when Swing thinks your component or a part of your component needs to be redrawn. The key here is "part of your component". Swing indicates what part needs to be redrawn with the clipping region (don't forget to take the Graphics2D origin into account, it might not always be 0,0). So in your paintComponent() you use that information (and the current zoom factor) to identify the part of your graphic element data that needs rendering. That part should be much less then your whole data. Then, right in paintComponent() you render it to the provided Graphics2D.

I assume you have some extra buttons for zooming in or out (or some mouse events). Anyhow, whenever the zoom factor changes you remember the new zoom factor, then call repaint(). At some point in time repaint() will result in a call to paintComponent(). And since your paintComponent() takes the zoom factor into account (well, it should), it will render the plot with the new zoom factor.

Also, whenever the incomming data has changed significantly, you call repaint, too. Don't do it to often, and don't do it for every tiny change. Accumulate a bunch of changes, then call repaint(). However, if you really think you need 15fps and every tiny change in the data needs to be immediately visible, than forget everything I wrote and change to active rendering.

That's the basic architecture. If that is still not responsive enough, consider caching the renderings you did in paintComponent() in BufferedImages. And before you render some area you check your cache if you have already all or parts of that area in your cache. If you have, you just copy that part from the cache to the screen. You need proper cache management, like remembering which zoom factor was used for a particular rendering, or if the incoming data would invalidate a cached rendering.

Careful planning, a well designed data structure, knowing Swing's paint mechanism (with all its barely documented idiosyncrasies), a clear understanding of world, component and screen coordinate systems, and probably a thousand odds and ends are needed to make this fly. Or you drop that Java2D junk, where you are never sure if you get hardware acceleration or not, and use some commercial data visualization system or write directly for a particular OS and native graphics system, so you are not at the mercy of a VM with rubbish documentation.

cbare
Offline
Joined: 2004-02-02

Looking at some BufferStrategy examples, it seems that the pattern is to do rendering in a timed loop, something like this:

[code]
// Render loop
while (!done) {
Graphics g = strategy.getDrawGraphics();
render(g);
g.dispose();
strategy.show();

try {
Thread.sleep(100);
}
catch (InterruptedException e) {
done=true;
}
}
[/code]

I'm drawing in response to UI events (scrolling, zooming, mouse clicks for selecting objects). Does that mean BufferStrategy is not for my situation? Pointers to examples would be great.

Thanks,

-chris

Ken Warner

Do you get reasonable animation with this method?
How many frames a second do you get using invokeLater()?

java2d@JAVADESKTOP.ORG wrote:
> You don't need a BufferStrategy or VolatileImage unless you are doing live animation. It was hard to tell from your message if that was the case.
>
> I usually use a BufferedImage that you get from createCompatibleImage and draw using Java2D to the graphics context you get from it. When the plot is ready hand it over to the event dispatch thread and tell some component to paint it with invokeLater().
>
> Otherwise look for examples that use BufferStrategy...
> [Message sent by forum member 'swpalmer' (swpalmer)]
>
> http://forums.java.net/jive/thread.jspa?messageID=301594
>
> ===========================================================================
> To unsubscribe, send email to listserv@java.sun.com and include in the body
> of the message "signoff JAVA2D-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".
>

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

bcorbett8769
Offline
Joined: 2003-07-31

I recommend using something similar to the Foxtrot API. It is not recommended to perform time intensive processes in the event queue.

cbare
Offline
Joined: 2004-02-02

I'm starting think my problem is knowing what solution applies to my situation. There are plenty of solutions for handling long-running tasks in Swing. (SwingWorker, etc.) There are solutions for double buffering for animations (BufferStrategy). But, I don't think either of those apply to long-running rendering tasks.

Maybe a solution to that would be to drip-feed little subtasks onto the event queue such that there is only one rendering task on the queue at a time and user events will be interleaved in the queue between rendering tasks.

Any other ideas or examples for maintaining a responsive UI in the presence of long-running rendering tasks?

Thanks,

-chris

Harald Kuhr

Hi Chris,

You might want to take a look at the Task class from Swing Application
Framework. It is a great abstraction for long-running operations in
general. It's basically a fancy SwingWorker, with support for a nice
publish/process-protocol, which I think you could use for your "drip-
feed" mechanism.

Not really sure it fits your needs, but I think it's worth looking into.

Best regards,

--
Harald K

On 25. sep.. 2008, at 22.34, java2d@JAVADESKTOP.ORG wrote:

> I'm starting think my problem is knowing what solution applies to my
> situation. There are plenty of solutions for handling long-running
> tasks in Swing. (SwingWorker, etc.) There are solutions for double
> buffering for animations (BufferStrategy). But, I don't think either
> of those apply to long-running rendering tasks.
>
> Maybe a solution to that would be to drip-feed little subtasks onto
> the event queue such that there is only one rendering task on the
> queue at a time and user events will be interleaved in the queue
> between rendering tasks.
>
> Any other ideas or examples for maintaining a responsive UI in the
> presence of long-running rendering tasks?
>
> Thanks,
>
> -chris
> [Message sent by forum member 'cbare' (cbare)]
>
> http://forums.java.net/jive/thread.jspa?messageID=301678
>
> =
> =
> =
> =
> =
> ======================================================================
> To unsubscribe, send email to listserv@java.sun.com and include in
> the body
> of the message "signoff JAVA2D-INTEREST". For general help, send
> email to
> listserv@java.sun.com and include in the body of the message "help".

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

afishionado
Offline
Joined: 2004-05-26

Use the Canvas if you need hardware double-buffering badly enough that you're willing to deal with the Z-ordering issues that you get from mixing Swing and AWT widgets. Otherwise, use a JPanel. You cannot use a Canvas inside of a JScrollPane, for one thing.

Rendering outside of the event thread can be iffy. I wouldn't try using a VolatileImage outside the event thread, but a BufferedImage should work. One thing that we should make clear is that getting hardware acceleration (for things like blitting the image) and rendering outside the event thread will be pretty much mutually exclusive. (Unless you're doing some really deep AWT hacking.)

Are we talking about rendering an animation, or rendering a graphic infrequently (say, in response to button clicks)? What you want to do really depends on the answer to that.

cbare
Offline
Joined: 2004-02-02

Thanks for all the hints. The frame rate I'm getting using invokeLater is very slow (~5 seconds per frame). And, the event dispatch thread gets clogged with lots of rendering tasks and becomes unresponsive. So, that's no good.

My rendering is happening in response to user actions - scrolling, zooming, and mouse selections. My goal for the UI to remain responsive even if rendering takes several seconds.

Ken Warner

You can render in a different thread. I do in my viewers
and they remain responsive.

http://pancyl.com/LakeMary_1.htm

java2d@JAVADESKTOP.ORG wrote:
> Thanks for all the hints. The frame rate I'm getting using invokeLater is very slow (~5 seconds per frame). And, the event dispatch thread gets clogged with lots of rendering tasks and becomes unresponsive. So, that's no good.
>
> My rendering is happening in response to user actions - scrolling, zooming, and mouse selections. My goal for the UI to remain responsive even if rendering takes several seconds.
> [Message sent by forum member 'cbare' (cbare)]
>
> http://forums.java.net/jive/thread.jspa?messageID=301639
>
> ===========================================================================
> To unsubscribe, send email to listserv@java.sun.com and include in the body
> of the message "signoff JAVA2D-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".
>

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

swpalmer
Offline
Joined: 2003-06-10

You don't need a BufferStrategy or VolatileImage unless you are doing live animation. It was hard to tell from your message if that was the case.

I usually use a BufferedImage that you get from createCompatibleImage and draw using Java2D to the graphics context you get from it. When the plot is ready hand it over to the event dispatch thread and tell some component to paint it with invokeLater().

Otherwise look for examples that use BufferStrategy...

cbare
Offline
Joined: 2004-02-02

Thanks swp. I'm drawing a long 2D plot. The user can scroll right and left along a very long X axis, and I'd like that to be responsive. I'm less concerned about rendering speed than responsiveness, but a high frame-rate certainly won't hurt either. So, my approach is to work on responsiveness first, and rendering speed later, which is why I want to do rendering off the event dispatch thread.

Is it safe to share the BufferStrategy with another thread? If not, what's the proper way to handle a long-running rendering task?

Thanks!

afishionado
Offline
Joined: 2004-05-26

I wouldn't be comfortable using a BufferStrategy outside of the event thread. That's mainly useful for reducing flicker in animations, anyway.

If you have your heart set on rendering outside the event thread, a BufferedImage is the only option I would trust. Even then, you need to make sure that you are synchronizing correctly.

Also, we should be clear that multithreading like this will *not* make your program faster (in fact, it will probably slow it down in this case). What it will accomplish is that it will allow the program to continue accepting user input while the drawing occurs.