Skip to main content

Re: [JAI-IMAGEIO] Transcoding large image files

8 replies [Last post]
Anonymous

The JPEG almost certainly needs to be completely in memory (does not support tiling), so you are not going to be able to do this... (at least not without a 64 bit JVM), and even then some other limitations will probably prevent it.

Since a BMP is normally uncompressed, it is implicitly tiled, thus a small region can be read at a time.

-----Original Message-----
>From: jai-imageio@javadesktop.org
>Sent: Dec 13, 2007 10:39 AM
>To: interest@jai-imageio.dev.java.net
>Subject: [JAI-IMAGEIO] Transcoding large image files
>
>Hi all,
>
>I have followed the example listed here: http://java.sun.com/products/java-media/jai/forDevelopers/jai1_0_1guide-...
>
>In my case, however, I am trying to turn a 4gb bmp into a jpg. I get java out of heap space errors on the line encoder.encode(img) in the function encodeImage. I don't understand why, as nothing is being buffered.
>
>Any pointers greatly appreciated.
>
>John
>[Message sent by forum member 'jppowell' (jppowell)]
>
>http://forums.java.net/jive/thread.jspa?messageID=249952
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
>For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Simone Giannecchini

It is not the case that it is impossible in Java, it is just that it
has not been implemented that way.

Simone.

On Dec 14, 2007 1:40 PM, wrote:
> Indeed, I made the large bitmap with hundreds of tiffs and tiling in the first place. So, excluding a 64-bit jvm, the only formats that I can transcode to are tiff, bmp and jpeg2000?
>
> We have some proprietary code that will happily concert a 10gb bmp to a jpeg. It is written in C and is clearly not doing it all in memory. Sadly, I can't see the source, but I wonder how it is possible to do this in C and not Java.
> [Message sent by forum member 'jppowell' (jppowell)]
>
> http://forums.java.net/jive/thread.jspa?messageID=250106
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>
>

--
-------------------------------------------------------
Eng. Simone Giannecchini
President /CEO GeoSolutions S.A.S.
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it

-------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

jppowell
Offline
Joined: 2007-03-14

Simone you are correct. However, it amounts to the same thing for those of us who don't have the time to write a jpeg codec plugin: you can't write large jpegs in JAI.

Nidel, Mike

You should file an issue I would say. This is something a lot
of people ask about from time to time. Search the issue DB first
of course to make sure you're not duplicating.

> -----Original Message-----
> From: jai-imageio@javadesktop.org
> [mailto:jai-imageio@javadesktop.org]
> Sent: Friday, December 14, 2007 7:54 AM
> To: interest@jai-imageio.dev.java.net
> Subject: Re: [JAI-IMAGEIO] Transcoding large image files
>
> Simone you are correct. However, it amounts to the same thing
> for those of us who don't have the time to write a jpeg codec
> plugin: you can't write large jpegs in JAI.
> [Message sent by forum member 'jppowell' (jppowell)]
>
> http://forums.java.net/jive/thread.jspa?messageID=250109
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail:
> interest-help@jai-imageio.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

jxc
Offline
Joined: 2005-02-24

There is already an issue filed against the codecLib JPEG codec:
https://jai-imageio-core.dev.java.net/issues/show_bug.cgi?id=117
Feel free to add comment and/or vote for it.

In the meantime, I wonder if you have tried the Java SE built-in
JPEG plugin, for example by doing
java -Dcom.sun.media.imageio.disableCodecLib=true YourApp

Thanks,
-James

jppowell
Offline
Joined: 2007-03-14

Thank you jxc. I have voted for that issue.

-Dcom.sun.media.imageio.disableCodecLib=true switch still leads to out of heap space error.

I think this is a good excuse for getting started on Jpeg 2000 anyway, which does support tiling and has other advantages over standard jpeg.

Thank you all for the prompt replies,

John

jppowell
Offline
Joined: 2007-03-14

Indeed, I made the large bitmap with hundreds of tiffs and tiling in the first place. So, excluding a 64-bit jvm, the only formats that I can transcode to are tiff, bmp and jpeg2000?

We have some proprietary code that will happily concert a 10gb bmp to a jpeg. It is written in C and is clearly not doing it all in memory. Sadly, I can't see the source, but I wonder how it is possible to do this in C and not Java.

jppowell
Offline
Joined: 2007-03-14

I was writing this before I saw Mike's reply. So, it would seem it is a java transcoder limitation and not a limitation of the jpg format itself.

Nidel, Mike

The limitations of a SampleModel are that it cannot be of size
greater than Integer.MAX_VALUE. So you probably cannot create
a jpeg that large even on 64-bit JVMs.

I believe (not certain) that the requirement for the JPEG to
be entirely in memory is a limitation of the available Java
JPEG readers/writers rather than the format itself.

> -----Original Message-----
> From: Robert Engels [mailto:rengels@ix.netcom.com]
> Sent: Thursday, December 13, 2007 11:53 AM
> To: interest@jai-imageio.dev.java.net
> Subject: Re: [JAI-IMAGEIO] Transcoding large image files
>
> The JPEG almost certainly needs to be completely in memory
> (does not support tiling), so you are not going to be able to
> do this... (at least not without a 64 bit JVM), and even then
> some other limitations will probably prevent it.
>
> Since a BMP is normally uncompressed, it is implicitly tiled,
> thus a small region can be read at a time.
>
> -----Original Message-----
> >From: jai-imageio@javadesktop.org
> >Sent: Dec 13, 2007 10:39 AM
> >To: interest@jai-imageio.dev.java.net
> >Subject: [JAI-IMAGEIO] Transcoding large image files
> >
> >Hi all,
> >
> >I have followed the example listed here:
> >http://java.sun.com/products/java-media/jai/forDevelopers/jai
> 1_0_1guide
> >-unc/Encode.doc.html#55335
> >
> >In my case, however, I am trying to turn a 4gb bmp into a
> jpg. I get java out of heap space errors on the line
> encoder.encode(img) in the function encodeImage. I don't
> understand why, as nothing is being buffered.
> >
> >Any pointers greatly appreciated.
> >
> >John
> >[Message sent by forum member 'jppowell' (jppowell)]
> >
> >http://forums.java.net/jive/thread.jspa?messageID=249952
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> >For additional commands, e-mail:
> interest-help@jai-imageio.dev.java.net
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail:
> interest-help@jai-imageio.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net