Skip to main content

drawImage for video rendering

1 reply [Last post]
remmeier
Offline
Joined: 2007-05-07

I'm currently working on a new JNI wrapper for ffmpeg to play videos in Java without using the out-dated (and buggy) JMF library. This would allow to play virtually any audio/video file from the local file system, http, or rtp. The SWT version is already working fine, but the Java2D-based version for Swing does not have the needed performance. I have to display 25 frames per second and get the data either as int[] or byte[]. Any of a rather large number of palettes can be choosen within ffmpeg and scaling is also already done. In Java2D, I allocate a new image before the first frame is shown:

int[] data = new int[width * height];
DataBufferInt dataBuffer = new DataBufferInt(data, data.length);
DirectColorModel colorModel = new DirectColorModel(24, rMask, ...);
SampleModel sm = new SinglePixelPackedSampleModel(DataBuffer.TYPE_INT, width, height, new int[] { rMask, gMask, bMask });
WritableRaster wr = Raster.createWritableRaster(sm, dataBuffer, new Point(0, 0));
image = new BufferedImage(colorModel, wr, true, null);

Afterwards I just fill the data array with the new data and call

g2d.drawImage(image, 0, 0, null);

However, drawImage(..) takes about 15 to 30ms for a frame of size 608x336. Doing it almost the same way in SWT with an ImageData object takes only 0.5ms. Is there a way to improve performance? Or do I have to use either DirectX or OpenGL?

It would be nice if it is compatible with JXPanel to support, for example, overlays. Moreover, as far as I have seen there is no NIO support in Java2D? Because this way I could get the data from ffmpeg without having to copy anything.

Thanks, Remo

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Jim Graham

What values are you using for rMask, gMask and bMask?

Internally we have native support for packed formats of xRGB, xBGR,
ARGB, and RGBx. It the masks you are using don't match one of those
formats then we may back off to some pretty generalized "method call per
pixel" code which might explain the slow frame times you are seeing.

If that is the case, then it may be faster for you to rearrange the
components into one of those supported orders as you fill the array (a
few shifts and masks) rather than rely on drawImage to do it
automatically (with method calls, etc.).

I submitted bug 6554473 on this which should show up on the bug parade
in the next day or so. In the meantime, the workaround I mention above
(doing your own bit shifting) can improve the performance to within
about 3x of one of the supported formats. Even better would be to get
ffmpeg to return the data in a format that matches one of our supported
formats better...

...jim

java2d@javadesktop.org wrote:
> I'm currently working on a new JNI wrapper for ffmpeg to play videos in Java without using the out-dated (and buggy) JMF library. This would allow to play virtually any audio/video file from the local file system, http, or rtp. The SWT version is already working fine, but the Java2D-based version for Swing does not have the needed performance. I have to display 25 frames per second and get the data either as int[] or byte[]. Any of a rather large number of palettes can be choosen within ffmpeg and scaling is also already done. In Java2D, I allocate a new image before the first frame is shown:
>
> int[] data = new int[width * height];
> DataBufferInt dataBuffer = new DataBufferInt(data, data.length);
> DirectColorModel colorModel = new DirectColorModel(24, rMask, ...);
> SampleModel sm = new SinglePixelPackedSampleModel(DataBuffer.TYPE_INT, width, height, new int[] { rMask, gMask, bMask });
> WritableRaster wr = Raster.createWritableRaster(sm, dataBuffer, new Point(0, 0));
> image = new BufferedImage(colorModel, wr, true, null);
>
> Afterwards I just fill the data array with the new data and call
>
> g2d.drawImage(image, 0, 0, null);
>
> However, drawImage(..) takes about 15 to 30ms for a frame of size 608x336. Doing it almost the same way in SWT with an ImageData object takes only 0.5ms. Is there a way to improve performance? Or do I have to use either DirectX or OpenGL?
>
> It would be nice if it is compatible with JXPanel to support, for example, overlays. Moreover, as far as I have seen there is no NIO support in Java2D? Because this way I could get the data from ffmpeg without having to copy anything.
>
> Thanks, Remo
> [Message sent by forum member 'remmeier' (remmeier)]
>
> http://forums.java.net/jive/thread.jspa?messageID=215811
>
> ===========================================================================
> To unsubscribe, send email to listserv@java.sun.com and include in the body
> of the message "signoff JAVA2D-INTEREST". For general help, send email to
> listserv@java.sun.com and include in the body of the message "help".

===========================================================================
To unsubscribe, send email to listserv@java.sun.com and include in the body
of the message "signoff JAVA2D-INTEREST". For general help, send email to
listserv@java.sun.com and include in the body of the message "help".