Skip to main content

double buffering on personal basis profile

6 replies [Last post]
Anonymous

Hi all!
I would really like to know yours ideas and experiences about the
subject, I will introduce the problem and my current development:

On MHP set top boxes my experiences/expectations are that xlets doing
paint are double buffered, this means that the paint cycle should be
something like this:
- all drawings operations go to back buffer
- at the next vertical sync the back buffer is switched with the front
buffer
- the back buffer is cleared

HScene is a Frame and it always cleans itself upon repaint, others
components are lightweight.

So what's the problem?

The problem is that this doesn't fit too good with some double buffering
pattern used in Java like this:
http://developers.sun.com/learning/javaoneonline/2007/pdf/TS-5724.pdf
and I guess quite common on applications.

Hinkmond in the slides is suggesting to create a double buffer at
xlet/application level but this, I my current idea, would actually lead
into a performance loss because the middleware is actually doing a
double buffering switching pointers to memory maps of screen buffers and
not blitting/coping the images' memory, no need to blit it twice...
probably I am missing something.

bests,
Lorenzo Pallara

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Hinkmond Wong

Lorenzo Pallara wrote:
> Hi all!
> I would really like to know yours ideas and experiences about the
> subject, I will introduce the problem and my current development:
>
> On MHP set top boxes my experiences/expectations are that xlets doing
> paint are double buffered, this means that the paint cycle should be
> something like this:
> - all drawings operations go to back buffer
> - at the next vertical sync the back buffer is switched with the front
> buffer
> - the back buffer is cleared
>
> HScene is a Frame and it always cleans itself upon repaint, others
> components are lightweight.
>
> So what's the problem?
>
> The problem is that this doesn't fit too good with some double buffering
> pattern used in Java like this:
> http://developers.sun.com/learning/javaoneonline/2007/pdf/TS-5724.pdf
> and I guess quite common on applications.
>
> Hinkmond in the slides is suggesting to create a double buffer at
> xlet/application level but this, I my current idea, would actually lead
> into a performance loss because the middleware is actually doing a
> double buffering switching pointers to memory maps of screen buffers and
> not blitting/coping the images' memory, no need to blit it twice...
> probably I am missing something.

I'm not sure exactly what your situation is without seeing your source
code, so I can just guess. My suggestion is you should implement the
change to use the double buffer and measure the performance both
visually to see effect if you can perceive any difference and make a
measurement with some code (like measure frames per second) to see if
you have a true performance loss or just a theoretical performance hit.
Sometimes what you think might be slow, actually looks fine to the
human eye.

Hope this helps.

Hinkmond

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Lorenzo Pallara

Hinkmond Wong wrote:
> Lorenzo Pallara wrote:
>> Hi all!
>> I would really like to know yours ideas and experiences about the
>> subject, I will introduce the problem and my current development:
>>
>> On MHP set top boxes my experiences/expectations are that xlets doing
>> paint are double buffered, this means that the paint cycle should be
>> something like this:
>> - all drawings operations go to back buffer
>> - at the next vertical sync the back buffer is switched with the
>> front buffer
>> - the back buffer is cleared
>>
>> HScene is a Frame and it always cleans itself upon repaint, others
>> components are lightweight.
>>
>> So what's the problem?
>>
>> The problem is that this doesn't fit too good with some double
>> buffering pattern used in Java like this:
>> http://developers.sun.com/learning/javaoneonline/2007/pdf/TS-5724.pdf
>> and I guess quite common on applications.
>>
>> Hinkmond in the slides is suggesting to create a double buffer at
>> xlet/application level but this, I my current idea, would actually
>> lead into a performance loss because the middleware is actually doing
>> a double buffering switching pointers to memory maps of screen
>> buffers and not blitting/coping the images' memory, no need to blit
>> it twice... probably I am missing something.
>
> I'm not sure exactly what your situation is without seeing your source
> code, so I can just guess. My suggestion is you should implement the
> change to use the double buffer and measure the performance both
> visually to see effect if you can perceive any difference and make a
> measurement with some code (like measure frames per second) to see if
> you have a true performance loss or just a theoretical performance
> hit. Sometimes what you think might be slow, actually looks fine to
> the human eye.
>
> Hope this helps.
>
> Hinkmond
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
> For additional commands, e-mail: advanced-help@phoneme.dev.java.net
>
>

You are suggestions are right Hinkmond, I do actually use many "tricks"
at application level to smooth things for human perception, what I would
like to talk about is how the underlying AWT stack is supposed to
behave, I will just drop a few more lines about double buffering and PBP
if anybody wants to get deeper into the matter:

- To get a better overview as starting point I can suggest
http://java.sun.com/docs/books/tutorial/extra/fullscreen/doublebuf.html

- Context: I am developing on low end set top box and generally video
output is controlled in a raster/scan way, so you can't update pixels
like on mobiles' lcd (even if you have a lcd panel connect ;-) maybe the
newer chipsets do have some hardware accelleration, but it's currently
out of our scope.

- Measurements: I often/always have fps and milliseconds prints for
debugging, that's because fps goes down easily and we need to pay attention.

- Similaritis: I had experience on MHP commercial available products on
sale here in Italy since some years, I developed many java applications
on them and what I found out on development is:
If you call repaint of the root window there are no visual artifacts
even if damn slow.
If you don't call repaint of the root window but you use getGraphics()
and you draw 1000 rects you will see the drawing happening on the screen
with artifacts like flickering and half images.
To don't call repaint is out from AWT's best practices and as expected
it doesn't work properly and I also agree it should produce side effects
if the appliction does it.
The tests just lead my guess that repaint should actually switching
front and back buffer and also clearing the back buffer as repaint
should do on heavyweight components.

I applied this solution adding a couple of native calls at
java.awt.Window.java on my repository and I am happy about them but I
think many JavaMe developers expectes the same behaviour as on mobiles
but they won't get it on Standard Definition Java set top boxes and my
fear is that there is no solution about it.

- The source code: you can can have a look at:
http://www.experimentalstuff.com/Technologies/mwpbp/download.html
and get this to build on phoeneme advanced on linux-86-generic with this
parameters:
J2ME_CLASSLIB=basis
AWT_IMPLEMENTATION=microwindows
MICROWIN=$HOME/microwin-x86-generic/src/engine
AWT_LIB_LIBS="-L$HOME/microwin-x86-generic/src/engine -lmwengine
-lfreetype -lX11"
CVM_STATICLINK_LIBS=true
Unluckly you will need the patches we were discussing on the other
threads about microwindows on PBP to make this compile.

- At this point you just need to add some font and can run demos and
applications on your linux x86 pc like:
bin/cvm -cp democlasses.jar basis.DemoFrame
It's quite easy to test that if you do draw an image into an off screen
surface and then you draw the offscreen surface, you are executing the
same code twice, once a for a small area, the image, then for a larger
area, the offscreen surface
and this is clearly all overhead. Just add a few fprintf at native level.

Here is an example code to use for testing:

import java.io.*;
import java.lang.*;
import java.awt.*;
import java.util.*;
import java.awt.event.*;

public class OffScreenTest extends Component {

private static Frame scene;
private static OffScreenTest app = null;
private static Graphics bufferGraphics = null;
private static Image offscreen = null;
private static boolean switchOffScreen = false;
private static Image lulu = null;

public OffScreenTest () {

try {
MediaTracker mediaTracker = new MediaTracker(this);
lulu = Toolkit.getDefaultToolkit().getImage("lulu.gif");
mediaTracker.addImage(lulu, 0);
mediaTracker.waitForAll();
} catch (Exception e) {
e.printStackTrace();
}

}

public static void main (String[] args) {

try {
scene = new Frame();
app = new OffScreenTest();
scene.add(app);
app.setVisible(true);
scene.setVisible(true);
scene.requestFocus();

offscreen = scene.createImage(200, 200);

bufferGraphics = offscreen.getGraphics();

int i = 0;
for (i = 0; i < 5; i++) {
app.repaint();
Thread.sleep(60);
}

} catch (Exception e) {

e.printStackTrace();
}

}

public void paint(Graphics g) {

if (switchOffScreen) {

bufferGraphics.clearRect(0, 0, 200, 200);
System.out.println("blitting lulu");
bufferGraphics.drawImage(lulu, 0, 0, null);

System.out.println("blitting offscreen");
g.drawImage(offscreen, 0, 0, this);

} else {
System.out.println("blitting lulu");
g.drawImage(lulu, 0, 0, this);
}
switchOffScreen = !switchOffScreen;

}

}

A possible prints layout will be:

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120
blitting offscreen
blitsrc 0, 0, 200, 200

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120
blitting offscreen
blitsrc 0, 0, 200, 200

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120

container is painting:OffScreenTest[,0,0,640x480]
blitting lulu
blitsrc 0, 0, 143, 120
blitting offscreen
blitsrc 0, 0, 200, 200

As you can see if lulu is blitted, the next time both lulu and offscreen
are blitted, the result on video is the same but performance changes.

bests!
Lorenzo

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Hinkmond Wong

Hi Lorenzo,

Lorenzo Pallara wrote:
> You are suggestions are right Hinkmond, I do actually use many
> "tricks" at application level to smooth things for human perception,
> what I would like to talk about is how the underlying AWT stack is
> supposed to behave, I will just drop a few more lines about double
> buffering and PBP if anybody wants to get deeper into the matter:
>
> - To get a better overview as starting point I can suggest
> http://java.sun.com/docs/books/tutorial/extra/fullscreen/doublebuf.html

> ...

> A possible prints layout will be:
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
> blitting offscreen
> blitsrc 0, 0, 200, 200
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
> blitting offscreen
> blitsrc 0, 0, 200, 200
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
>
> container is painting:OffScreenTest[,0,0,640x480]
> blitting lulu
> blitsrc 0, 0, 143, 120
> blitting offscreen
> blitsrc 0, 0, 200, 200
>
> As you can see if lulu is blitted, the next time both lulu and
> offscreen are blitted, the result on video is the same but performance
> changes.
>
> bests!
> Lorenzo

Thanks for the summary! That helps in describing what you are doing.
You might want to note that the calling of paint is not always only from
your repaint() call, but can also be a result of other UI events, such
as if/when your window becomes covered then exposed or
minimized/maximized or just a chain reaction from a another component or
sub-components paint call, so that might be something you are not
expecting. And, to complicate matters it might be getting called
multi-threaded also.

Instead, if you want to guard your paint call and make sure you only
allow it to blit when you want and control that, you might want to use a
synchronized paint method instead of a simple flag.

Ex.

http://math.hws.edu/eck/cs124/javanotes1/c6/s4.html

---

To apply synchronization to animation with an off-screen image, you can
define a synchronized paint method that copies the off-screen image to
the screen:

Image OSC = null; // the off-screen image

synchronized public void paint(Graphics g) {
if (OSC != null) // if off-screen image exists, copy it to screen
g.drawImage(OSC,0,0,this);
else { // otherwise, fill the applet with the background color
g.setColor(getBackground());
g.fillRect(0,0,size().width,size().height);
}
}

public void update(Graphics g) {
// redefine update() so that it doesn't erase the applet;
// this will avoid flickering when the applet is redrawn
paint(g);
}
---

Hinkmond

Hinkmond

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

Lorenzo Pallara

Hinkmond Wong wrote:
> Hi Lorenzo,
>
>
> Lorenzo Pallara wrote:
>> You are suggestions are right Hinkmond, I do actually use many
>> "tricks" at application level to smooth things for human perception,
>> what I would like to talk about is how the underlying AWT stack is
>> supposed to behave, I will just drop a few more lines about double
>> buffering and PBP if anybody wants to get deeper into the matter:
>>
>> - To get a better overview as starting point I can suggest
>> http://java.sun.com/docs/books/tutorial/extra/fullscreen/doublebuf.html
>
>> ...
>
>> A possible prints layout will be:
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>> blitting offscreen
>> blitsrc 0, 0, 200, 200
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>> blitting offscreen
>> blitsrc 0, 0, 200, 200
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>>
>> container is painting:OffScreenTest[,0,0,640x480]
>> blitting lulu
>> blitsrc 0, 0, 143, 120
>> blitting offscreen
>> blitsrc 0, 0, 200, 200
>>
>> As you can see if lulu is blitted, the next time both lulu and
>> offscreen are blitted, the result on video is the same but
>> performance changes.
>>
>> bests!
>> Lorenzo
>
>
> Thanks for the summary! That helps in describing what you are doing.
> You might want to note that the calling of paint is not always only
> from your repaint() call, but can also be a result of other UI events,
> such as if/when your window becomes covered then exposed or
> minimized/maximized or just a chain reaction from a another component
> or sub-components paint call, so that might be something you are not
> expecting. And, to complicate matters it might be getting called
> multi-threaded also.
>
> Instead, if you want to guard your paint call and make sure you only
> allow it to blit when you want and control that, you might want to use
> a synchronized paint method instead of a simple flag.
>
> Ex.
>
> http://math.hws.edu/eck/cs124/javanotes1/c6/s4.html
>
> ---
>
> To apply synchronization to animation with an off-screen image, you
> can define a synchronized paint method that copies the off-screen
> image to the screen:
>
> Image OSC = null; // the off-screen image
> synchronized public void paint(Graphics g) {
> if (OSC != null) // if off-screen image exists, copy it to
> screen
> g.drawImage(OSC,0,0,this);
> else { // otherwise, fill the applet with the background color
> g.setColor(getBackground());
> g.fillRect(0,0,size().width,size().height);
> }
> }
> public void update(Graphics g) {
> // redefine update() so that it doesn't erase the applet;
> // this will avoid flickering when the applet is redrawn
> paint(g);
> }
> ---
>
> Hinkmond
>
>
yes, better to point it out, beware: my example in the thread was just
meant to discuss performance! Thread.sleep(60) is just a value large
enough to make sure the test has enough time to execute, the flag is
used to show the performance difference.

In real cases, for MHP, you will need to synchronize your own paint and
repaint and to use toolkit.sync() and the end of paint() to make sure
all the drawing operation are executed before calling repaint again.

Lorenzo

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net

bddeveloper
Offline
Joined: 2008-01-14
Points: 0

Hi Lorenzo,

Did you figure out the double buffer issue? I am having the same problem.

Thank you,

Scott

Lorenzo Pallara

phonemeadvanced@mobileandembedded.org wrote:
> Hi Lorenzo,
>
> Did you figure out the double buffer issue? I am having the same problem.
>
>

Hi Scott,
I see you are coming from some blu ray experience, so here is it my
full picture of the issue, notice that the xlet should be able to figure
out if the box has double buffering querying HScene.isDoubleBuffered()
so it can decide the best strategy at starting time and notice that blu
ray has a new API introduce on purpose to taking care of these problems
that's SyncFrameAccurateAnimation.

Let me tell before also that obviously if the set top box is without any
hardware acceleration for 2d graphics, has a cpu that is slow compared
to the drawing needs and it doesn't support any form of double buffering
you won't get any good out of it but I have never experienced such a low
profile hardware for interactive television.

If the box and the middleware do support double buffering, and this is
the case of all the mhp boxes in Italy, your xlet needs to wait between
repaints and at the of the paint you need to call also toolkit.sync to
make sure any operation is finished.
This will ensure that: everything you want to draw when calling repaint
is actually displayed after the first vertical blank that happens after
all the draws are done and you won't see drawing on the front buffer
because it happens in the back buffer.
You can also try to avoid the container to be cleared at repaint and to
clear your own area setting no_background_fill to speed up BUT if the
box has a hw way to clear the buffer you could miss it SO you should
measure the performance of the box Hscene clear at initialization and
your own clear.

If the set top box has 2d hardware acceleration or a fast cpu but has
not enough memory for double buffering you could go with the standard
way using an image to draw everything and then blit the image at the end
of the paint however this won't guarantee you won't see the drawing
happening on the front buffer and so on the display because you are
drawing on the same memory you are displaying and I don't think the
drawing occurs synchronized with the vertical scan and it is fast enough
to work with one buffer only. Notice that if the box has enough memory
to have an image large as the front buffer... it should have also memory
for double buffering, so there is something odd going on...

what do you think? which kind of tests did you perform? what was your
experience?

bests,
Lorenzo

---------------------------------------------------------------------
To unsubscribe, e-mail: advanced-unsubscribe@phoneme.dev.java.net
For additional commands, e-mail: advanced-help@phoneme.dev.java.net