Skip to main content

How to do pixel based animation instead of MemoryImageSource?

3 replies [Last post]
ylzhao
Offline
Joined: 2004-08-22

Hi,
in JDK 1.1 days, one always uses MemoryImageSource to form animation from an image pixel data buffer. For example,

=========================================================
int pixels[];
MemoryImageSource source;

public void init() {
int width = 50;
int height = 50;
int size = width * height;
pixels = new int[size];

int value = getBackground().getRGB();
for (int i = 0; i < size; i++) {
pixels[i] = value;
}

source = new MemoryImageSource(width, height, pixels, 0, width);
source.setAnimated(true);
image = createImage(source);
}

public void run() {
Thread me = Thread.currentThread( );
me.setPriority(Thread.MIN_PRIORITY);

while (true) {
try {
thread.sleep(10);
} catch( InterruptedException e ) {
return;
}

// Modify the values in the pixels array at (x, y, w, h)

// Send the new data to the interested ImageConsumers
source.newPixels(x, y, w, h);
}
}
=========================================================

But use MemoryImageSource suffers performance problem. For example, if the pixel data array is large, then the animation fps is slow, or if want to do an fullscreen animation, MemoryImageSource is weak.

In JDK 1.5, there are BufferedImage, VolatileImage and BufferStrategy which can be hardware accelerated. However, if operate on the pixel data buffer directly, Java 2D engine can't use hardware acceleration.

So, my question is : Is there a method can produce pixel based animation and use hardware acceleration instead of MemoryImageSource?

Message was edited by: ylzhao

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Jim Graham

Hi ylzhao,

There are some missing details in your post that affect the type of
solution that can achieve your goals. For instance, how often are the
pixels updated as compared to how often the updated image is drawn to
the screen? Also, what percentage of the pixels are modified on a
typical update? Are the values of any of the pixels in the image read
back during the modification phase?

These issues matter since they can affect how much hardware acceleration
can help you. For example, if your updates touch most of the pixels in
the image, and they are read/modify/write types of updates, and you
update the pixels once per frame rendered to the screen, then hardware
acceleration will not give you much benefit and may actually hurt your
performance as the read performance of most video cards is very poor
compared to system memory.

Unfortunately we are still working on a way to provide better
acceleration-friendly access to the pixels of various image types. See
the bug 6205557 for more information:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6205557

Right now if you get the DataBuffer from a Raster for a BufferedImage
the image will become permanently non-acceleratable. This means that
any of the relatively efficient methods on DataBuffer for updating the
pixels are out of reach if you want to maintain acceleration. Using the
data modification methods on the WritableRaster class will allow your
image to remain acceleratable, but each time you modify the pixels it
will take 2 copies to the screen before the copies become accelerated
again. The reason the image is only accelerated on the second and
subsequent copies is that the copy of the image data to the cached vram
surface is unnecessary work if the image is always changing.

Probably the best bet would be a solution based on an INT_RGB
BufferedImage and modifying the pixels using the method:

WritableRaster.setDataElements(x, y, int[]);

...jim

> In JDK 1.5, there are BufferedImage, VolatileImage and BufferStrategy
> which can be hardware accelerated. However, if operate on the pixel
> data buffer directly, Java 2D engine can't use hardware acceleration.
>
> So, my question is : Is there a method can produce pixel based
> animation and use hardware acceleration instead of MemoryImageSource?
> [Message sent by forum member 'ylzhao' (ylzhao)]

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".

ylzhao
Offline
Joined: 2004-08-22

Hi Jim,

> Hi ylzhao,
>
> There are some missing details in your post that
> affect the type of
> solution that can achieve your goals. For instance,
> how often are the
> pixels updated as compared to how often the updated
> image is drawn to
> the screen? Also, what percentage of the pixels are
> modified on a
> typical update? Are the values of any of the pixels
> in the image read
> back during the modification phase?

This is my applet prototype:
=========================================================
public class Viewer extends Applet implements Runnable {
int[] idata = null; // image pixel data array

Image view = null; // viewport
int[] vdata = null; // view data array

Image offImage = null; // Offscreen rendering image

Graphics offGraphics = null; // Offscreen rendering graphics context

int offwidth = 0; // Width of the offscreen graphics

int offheight = 0; // Height of the offscreen graphics

MemoryImageSource source = null; // View is calcaulated here before dispaly

public Viewer() {
// use PixelGrabber to read pixels data into
// idata array
init();
}

public void init() {
// some init code
idata = readImagePixelData(imageFileName);
}

public void update(Graphics g) {
paint(g);
}

public void paint(Graphics g) {
// Setup offscreen rendering environmnent
offwidth = getWidth();
offheight = getHeight();
offImage = createImage(offwidth, offheight);
offGraphics = offImage.getGraphics();

vdata = new int[vwidth * vheight];
source = new MemoryImageSource(vwidth, vheight, vdata, 0, vwidth);
source.setAnimated(true);
view = createImage(source);

// read pixel data from idata and
// use some warp transform to
// extract a new view according
// to user input
transform(idata, vdata, mouse_x, mouse_y);
source.newPixels();

offGraphics.drawImage(view, 0, 0, this);
g.drawImage(offImage, 0, 0, this);
}
=========================================================

And, according to the mouse actions, like drag and drop on the applet, it will call repaint and update the view.
When the view data is calculated completed, then show the view image.

> These issues matter since they can affect how much
> hardware acceleration
> can help you. For example, if your updates touch
> most of the pixels in
> the image, and they are read/modify/write types of
> updates, and you
> update the pixels once per frame rendered to the
> screen, then hardware
> acceleration will not give you much benefit and may
> actually hurt your
> performance as the read performance of most video
> cards is very poor
> compared to system memory.
>
> Unfortunately we are still working on a way to
> provide better
> acceleration-friendly access to the pixels of various
> image types. See
> the bug 6205557 for more information:
>
>
>
>
>
>
>
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=62
> 05557
>
> Right now if you get the DataBuffer from a Raster for
> a BufferedImage
> the image will become permanently non-acceleratable.
> This means that
> any of the relatively efficient methods on DataBuffer
> for updating the
> pixels are out of reach if you want to maintain
> acceleration. Using the
> data modification methods on the WritableRaster class
> will allow your
> image to remain acceleratable, but each time you
> modify the pixels it
> will take 2 copies to the screen before the copies
> become accelerated
> again. The reason the image is only accelerated on
> the second and
> subsequent copies is that the copy of the image data
> to the cached vram
> surface is unnecessary work if the image is always
> changing.
>
> Probably the best bet would be a solution based on an
> INT_RGB
> BufferedImage and modifying the pixels using the
> method:
>
> WritableRaster.setDataElements(x, y, int[]);
>
> ...jim
>
> > In JDK 1.5, there are BufferedImage, VolatileImage
> and BufferStrategy
> > which can be hardware accelerated. However, if
> f operate on the pixel
> > data buffer directly, Java 2D engine can't use
> e hardware acceleration.
> >
> > So, my question is : Is there a method can produce
> pixel based
> > animation and use hardware acceleration instead of
> f MemoryImageSource?
> > [Message sent by forum member 'ylzhao' (ylzhao)]
>
> ======================================================
> =====================
> To unsubscribe, send email to listserv@java.sun.com
> and include in the body
> of the message "signoff JAVA2D-INTEREST". For
> general help, send email to
> listserv@java.sun.com and include in the body of the
> message "help".

Message was edited by: ylzhao

Jim Graham

There are really 2 questions here:

- How do I manipulate a BufferedImage without interfering with its
ability to be accelerated under the covers?

- Is there a type of image that lives in hardware accelerated memory and
to which I can write directly to its pixels?

To answer the first question, first note that the type of hardware
acceleration provided by a BufferedImage is very limited. It is what we
call a "managed image" which means that we can cache a copy of that
image in display memory if you copy it there frequently, but that copy
in the display memory is not where the pixels live - it is only a cached
copy. To avoid interfering with this "managed image" mechanism, you
would currently have to use the relatively high level methods on
BufferedImage and Raster and these are not as fast as manipulating the
pixel array directly. This concern doesn't really apply to your case,
though, since you modify all of the pixels in the image each time you
copy it to the screen and a managed image only provides benefit on the
second and subsequent times you render it to the screen. (The first
time you render it to a screen or VolatileImage, the data must be
uploaded to the display card whether it is going directly to the
screen/VolatileImage or whether it is going to the cached copy. Either
way, you still pay the performance penalty to get the pixels across the
bus to the card.)

The answer to the second question is that we currently have no image
type which has directly accessible pixels and which also lives in
accelerated memory - sort of an "uploadable texture" type of object.
That kind of image is on our wish list, but we have no schedule for
delivery of it yet...

...jim

> This is my applet prototype:
> =========================================================
> public class Viewer extends Applet implements Runnable {
> int[] idata = null; // image pixel data array
>
> Image view = null; // viewport
> int[] vdata = null; // view data array
>
> Image offImage = null; // Offscreen rendering image
>
> Graphics offGraphics = null; // Offscreen rendering graphics context
>
> int offwidth = 0; // Width of the offscreen graphics
>
> int offheight = 0; // Height of the offscreen graphics
>
> MemoryImageSource source = null; // View is calcaulated here before dispaly
>
> public Viewer() {
> // use PixelGrabber to read pixels data into
> // idata array
> init();
> }
>
> public void init() {
> // some init code
> idata = readImagePixelData(imageFileName);
> }
>
> public void update(Graphics g) {
> paint(g);
> }
>
> public void paint(Graphics g) {
> // Setup offscreen rendering environmnent
> offwidth = getWidth();
> offheight = getHeight();
> offImage = createImage(offwidth, offheight);
> offGraphics = offImage.getGraphics();
>
> vdata = new int[vwidth * vheight];
> source = new MemoryImageSource(vwidth, vheight, vdata, 0, vwidth);
> source.setAnimated(true);
> view = createImage(source);
>
> // read pixel data from idata and
> // use some warp transform to
> // extract a new view according
> // to user input
> transform(idata, vdata, mouse_x, mouse_y);
> source.newPixels();
>
> offGraphics.drawImage(view, 0, 0, this);
> g.drawImage(offImage, 0, 0, this);
> }

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".