Skip to main content

J2KImageReader parameters for scalable memory usage

14 replies [Last post]
scyudits
Offline
Joined: 2005-07-15

Hi,

Having successfully implemented scalable JPEG2000 compression for my very large tiled image (140MB), I am now trying to decompress a JPEG2000 compressed stream that is just over 65 MB in length. My environment, however, can only allocate at most 64MB of contiguous memory (which is the reason why I need JPEG2000 compression/decompression to be scalable). I am getting the following error:

java.lang.RuntimeException: An uncaught runtime exception has occurred
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.initializeRead(J2KReadState.java:604)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.(J2KReadState.java:153)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KRenderedImage.(J2KRenderedImage.java:41)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readAsRenderedImage(J2KImageReader.java:379)
Caused by: java.io.IOException: Out of memory to cache input data
at jj2000.j2k.util.ISRandomAccessIO.growBuffer(ISRandomAccessIO.java:169)
at jj2000.j2k.util.ISRandomAccessIO.readInput(ISRandomAccessIO.java:198)
at jj2000.j2k.util.ISRandomAccessIO.length(ISRandomAccessIO.java:277)
at jj2000.j2k.fileformat.reader.FileFormatReader.readFileFormat(FileFormatReader.java:195)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.initializeRead(J2KReadState.java:373)
... 7 more

It seems that J2KImageReader has been implemented to put all the compressed data into memory before reading. Is this true? Are there any parameter settings that can prevent J2KImageReader from allocating arrays according to the length of the input compressed data? ISRandomAccessIO does not look scalable at all.

Please help,

Sophia

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Brian Burkhalter

Brad,

> Providing a better set of base JPEG2000 encoder and decoder
> classes is certainly a worthwhile endeavor. However, it
> seems that the JJ2000 reference implementation at least
> works reasonably well. The same can not be said of the J2K
> plugins.

Do you mean to say that the problems appear to be primarily in the
jai-imageio-core code which wraps the JJ2000 implementation, i.e., the adapter
classes which make JJ2000 into JPEG 2000 reader and writer plug-ins?

Brian

----------------
Brian Burkhalter
Java Media, Imaging, and Graphics
Sun Microsystems, Inc.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This email message is for the sole use of the intended recipient(s)
and may contain confidential and privileged information. Any
unauthorized review, use, disclosure or distribution is prohibited.
If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Ken Warner

Brad,

How are you going to do all that stuff? Are you working by yourself or for a large company with lots of money?

Sorry, I don't mean to sound harsh. I'm really just curious. You've bitten off a big chunk.

jai-imageio@javadesktop.org wrote:
> Providing a better set of base JPEG2000 encoder and decoder
> classes is certainly a worthwhile endeavor. However, it
> seems that the JJ2000 reference implementation at least
> works reasonably well. The same can not be said of the J2K
> plugins. I have an urgent need to get at least the pure Java
> plugin working well and, if at all possible, consistently
> with the native plugin. This seems to be quite feasible in
> a reasonably short amount of time. I've already started.
>[stuff deleted]

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

castalia
Offline
Joined: 2005-12-06

Providing a better set of base JPEG2000 encoder and decoder
classes is certainly a worthwhile endeavor. However, it
seems that the JJ2000 reference implementation at least
works reasonably well. The same can not be said of the J2K
plugins. I have an urgent need to get at least the pure Java
plugin working well and, if at all possible, consistently
with the native plugin. This seems to be quite feasible in
a reasonably short amount of time. I've already started.

Note that the application that is using J2K requires tiling
and resolution control.

With reliable J2K plugins in place this will set the stage
for further enhacement of the JPEG2000 capabilities of the
Image I/O tools. For example, possible on-the-fly
resolution control, complete JP2 metadata, finer area
selection granularity (precint and/or code block level),
complete JP2 specification parameters for the writer, etc.
I also intend to implement a JPIP plugin for remote
server access to JP2 files (currently via the Kakadu
kdu_server).

My sense of things is that if the plugins work right other
enhancements will follow much more readily.

Brad Castalia

johnvaltus
Offline
Joined: 2006-03-31

Is there a reason that JJ2000 code is being used for the native plugins instead of kakadu?

castalia
Offline
Joined: 2005-12-06

Tests show that the native J2K plugin behaves as expected
when a RenderedImage is used: Intialization does not cause
the codestream to be rendered into pixels (as does the
pure Java plugin) and only those tiles requested are
decoded.

The native plugin, when registered, gives itself priority
over the pure Java plugin. This happens automatically
when the native plugin codec library can be found.

Getting these two versions of the J2K plugin to behave the
same, or at least correctly, seems to be needed. I'm going
to do what I can. If there is anyone else interested in
working to upgrade the J2K plugins, especially the pure
Java version, please let me know so we can coordinate our
efforts.

Thnx,

Brad Castalia

Brian Burkhalter

Brad - thanks.

Everyone else, if you have any interest in JPEG 2000 then any help you could
provide to Brad would be greatly appreciated by everyone. JPEG 2000 is the
compression methodology in which we have the least expertise so any
contributions would be most beneficial in this area.

Brian

> Getting these two versions of the J2K plugin to behave the
> same, or at least correctly, seems to be needed. I'm going
> to do what I can. If there is anyone else interested in
> working to upgrade the J2K plugins, especially the pure
> Java version, please let me know so we can coordinate our
> efforts.

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Ken Warner

As I understand it -- and I haven't looked at the code in a while -- the pure Java version of the JPEG2000 codec is based on the reference implementation JJ2000. JJ2000 is Java 1.0.x. Info on that is on the web.

I've hacked JJ2000 to a degree and it is what you might expect for a reference implementation. Lots of logging and error messages for every possible error. I estimated that if one simply removed all the intricate logging and error handling -- still leaving what is essential -- you could reduce the size of the JJ2000 implementation by half.

But I really think that if you got three or four people who really understood wavelett compression, you could write a series of codecs that ranged from just handling the j2c or codestream part to the tiles and multiple resolutions defined in the standard. I estimate that most applets don't need thumbnails or tiling or multiple resolutions.

I for one think that starting with a simple j2c decoder that could be packaged with an applet would be a major step forward. Like I said, I hacked JJ2000 and did just that but I did it as a demonstration. The result was a 150kb jar file. On the edge of impractibility for an applet. And my hack was not complete in that it left a lot of the logging and error handling I mentioned earlier. But it worked...

I just think that it would be more fruitful to start from scratch and use the latest greatest Java and start small and simple with a decoder for j2c. Then progess on to the encoder and j2p etc.

I don't know if this helps or muddies the waters...

Brian Burkhalter wrote:
> Brad - thanks.
>
> Everyone else, if you have any interest in JPEG 2000 then any help you
> could provide to Brad would be greatly appreciated by everyone. JPEG
> 2000 is the compression methodology in which we have the least expertise
> so any contributions would be most beneficial in this area.
>
> Brian
>
>> Getting these two versions of the J2K plugin to behave the
>> same, or at least correctly, seems to be needed. I'm going
>> to do what I can. If there is anyone else interested in
>> working to upgrade the J2K plugins, especially the pure
>> Java version, please let me know so we can coordinate our
>> efforts.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Brian Burkhalter

On Mon, 5 Jun 2006, Ken Warner wrote:

> As I understand it -- and I haven't looked at the code in a while -- the pure
> Java version of the JPEG2000 codec is based on the reference implementation
> JJ2000. JJ2000 is Java 1.0.x. Info on that is on the web.

That is correct. The Java JPEG 2000 plug-in more or less wraps the JJ2000
code. Unfortunately when this wrapper was written some of the JJ2000 code was
itself modified to accomplish it. I think this is less than ideal. Ideally the
JJ2000 code would be independently built and the wrapper would just invoke its
methods. It might be that this structure was not possible, I can't say.

There used to be a JJ2000 open source project on freshmeat or somewhere but
last I looked it had disappeared.

> I've hacked JJ2000 to a degree and it is what you might expect for a
> reference implementation. Lots of logging and error messages for every
> possible error. I estimated that if one simply removed all the intricate
> logging and error handling -- still leaving what is essential -- you could
> reduce the size of the JJ2000 implementation by half.
>
> But I really think that if you got three or four people who really understood
> wavelett compression, you could write a series of codecs that ranged from
> just handling the j2c or codestream part to the tiles and multiple
> resolutions defined in the standard. I estimate that most applets don't need
> thumbnails or tiling or multiple resolutions.
>
> I for one think that starting with a simple j2c decoder that could be
> packaged with an applet would be a major step forward. Like I said, I hacked
> JJ2000 and did just that but I did it as a demonstration. The result was a
> 150kb jar file. On the edge of impractibility for an applet. And my hack
> was not complete in that it left a lot of the logging and error handling I
> mentioned earlier. But it worked...
>
> I just think that it would be more fruitful to start from scratch and use the
> latest greatest Java and start small and simple with a decoder for j2c.
> Then progess on to the encoder and j2p etc.
>
> I don't know if this helps or muddies the waters...

It might not be a bad idea to start from the JJ2000 code, re-hack that, and
then move on from there eventually to the plug-ins.

Brian

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Ken Warner

Link to JJ2000 Java implementation.

http://jpeg2000.epfl.ch/

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

praveen87
Offline
Joined: 2009-04-03

Hi i am also doing project in jpeg2000 images but i would not do the woi part because the jp2 image file is not having resolution details...can anyone help me its urgent...Is there way to open it in IE or opera as such....

castalia
Offline
Joined: 2005-12-06

The pure Java J2K plugin (J2KImageReader) does, indeed,
behave badly by decoding the entire source image during
initialization. The native plugin (J2KImageReaderCodecLib)
does not have this problem.

I am working to fix this.

johnvaltus
Offline
Joined: 2006-03-31

>The native plugin
> (J2KImageReaderCodecLib)
> does not have this problem.
>
Castalia, So there is a way to use the native plugin to decode only the region of interest? I thought both the pure java and the native implementations had the same problem.
Do we have to do anything different that just having the native libraries accessible?

James Cheng

> Do we have to do anything different that just having the native libraries accessible?

By default, the native JPEG2000 plugin is ahead of the pure Java
plugin in the registry. You can find it out with some code like
this:

Iterator iter =
ImageIO.getImageReadersByFormatName("jpeg2000");
do {
jiioReader = (ImageReader)iter.next();
System.out.println(jiioReader.getClass().getName());
} while(iter.hasNext());

Here are the two JPEG2000 plugins when the native library is
available:

com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReaderCodecLib
com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader

To have the native plugin accessible, you need to have
clibwrapper_jiio.jar and its associated .dll/.so installed
properly. There is also the permission issue in the case of
applets.

HTH,
-James

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

johnvaltus
Offline
Joined: 2006-03-31

> > Do we have to do anything different that just
> having the native libraries accessible?
>
> By default, the native JPEG2000 plugin is ahead of
> the pure Java
> plugin in the registry. You can find it out with
> some code like
> this:
>
> Iterator iter =
> ImageIO.getImageReadersByFormatName("jpeg2000");
> do {
> jiioReader = (ImageReader)iter.next();
>
> System.out.println(jiioReader.getClass().getName());
> } while(iter.hasNext());
>
> Here are the two JPEG2000 plugins when the native
> library is
> available:
>
> com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageRea
> derCodecLib
> com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageRea
> der
>
> To have the native plugin accessible, you need to
> have
> clibwrapper_jiio.jar and its associated .dll/.so
> installed
> properly. There is also the permission issue in the
> case of
> applets.
>
> HTH,
> -James
>
This in my code:

[b]ImageReader reader = findReader(filename);
System.out.println(reader);[/b]

Gives this:
[b]com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReaderCodecLib[/b]

But I still get "out of memory" errors when I try to decode a portion of a jpeg2000 file. The errors don't happen when I decode the same portion of the same data when it's stored as tiff.
I assume this means the whole file is being decoded when I read from the jpeg2000.

Are these the right methods to be calling to use the native library properly?

param.setSourceRegion(region);
param.setSourceSubsampling( sample, sample, 0, 0);

Thanks,
John