Skip to main content

extremely large images

5 replies [Last post]
ashmore
Offline
Joined: 2006-05-24

Hello, I discovered JAI not too long ago, and it looks like a powerful API and I am excited to do some work with it. I have a question though, regarding working with large images. I do not know enough about the capability of JAI/IO, so I want to see if what I want to do is even possible.

The image I am working with is 50000 pixels square, and is in 24 bit color space. Uncompressed, it is about 7G on my hard disk. Right now, the image is made of a 10x10 grid of 5000 pixel square images each. What I want to do is to merge these together, and then scale them down using bicubic downsampling, and save the result. The tiles can not be scaled down individually, because this will create seams.

Will JAI/IO allow me to work with this without running out of memory? I think it is necessary for Java to use disk paging in order for this to work because it cannot fit the whole uncompressed image into the VM memory.

Help or input would be greatly appreciated. Cheers!

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Bob Deen

I think so, but it might take a little work.

First, you need to be able to create a full (virtual) image out of the
image tiles on the disk. The best way to do this might be to use
TiledImage, where when a tile is requested, you use imageio to read that
tile file and present it as an image tile (I believe you'd override
getTile() to do that). I know the idea of a virtual image where each
tile is a separate disk file has been talked about before, but I don't
recall if there's an implementation out there somewhere.

Once you have that, it's relatively easy to attach a scale and output to
that. They should work a tile at a time, but of course each tile will
require 9 source tiles to cover all the overlaps. If that's still too
much memory, you could put a format operator after the load, to make the
tile size smaller for the downstream operations. Worst case doing that
would be 4 source tiles (if you're working in a corner).

That's just an overview, but I wanted to let you know it is possible...
probably not terribly hard... but it's not trivial.

-Bob

jai-imageio@javadesktop.org wrote:
> Hello, I discovered JAI not too long ago, and it looks like a powerful API and I am excited to do some work with it. I have a question though, regarding working with large images. I do not know enough about the capability of JAI/IO, so I want to see if what I want to do is even possible.
>
> The image I am working with is 50000 pixels square, and is in 24 bit color space. Uncompressed, it is about 7G on my hard disk. Right now, the image is made of a 10x10 grid of 5000 pixel square images each. What I want to do is to merge these together, and then scale them down using bicubic downsampling, and save the result. The tiles can not be scaled down individually, because this will create seams.
>
> Will JAI/IO allow me to work with this without running out of memory? I think it is necessary for Java to use disk paging in order for this to work because it cannot fit the whole uncompressed image into the VM memory.
>
> Help or input would be greatly appreciated. Cheers!
> [Message sent by forum member 'ashmore' (ashmore)]
>
> http://forums.java.net/jive/thread.jspa?messageID=343264
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

ashmore
Offline
Joined: 2006-05-24

Thank you Bob Deen, that is exactly what I needed to know!
Despite being nontrivial, that seems very straightforward.

I have realized since that I should have made the source tiles much smaller. If I continue to have memory issues I might just have to break them up or re-render.

I now have two of these 50000 square images. I finished merging the first in Photoshop and it took days and required a lot of attention. I imagine that using JAI will go much more swiftly.

This is the sort of stuff I am working on if there is any curiosity:
http://www.icosilune.com/GeneticImage/gallery.php

castalia
Offline
Joined: 2005-12-06

At the HiRISE Operations Center we have been producing rather large images and storing them in JP2 format for distribution (http://hirise.lpl.arizona.edu). Though we do not process these data products in their JP2 form I have researched various issues associated with using JP2 files and the JPEG2000 codestream it contains.

It is possible to append image data to an existing JPEG2000 codestream under certain circumstances and by being careful to maintain the the validity of any codestream marker segments with data values that need to be updated by the change to the codestream. This is also true of JP2 box values with data values that will need to be updated. Simply appending image codestream data to an existing codestream or JP2 file does not produce a valid codestream or JP2 file.

JPEG2000 codestream tiling can have a significantly adverse affect on the performance of decoders. At first we generated tiled JPEG2000 codestreams with the expectation that this would be beneficial for for the tile oriented image data management system of JAI and the data processing pipelines and viewers (such as JadeDisplay) that would consume image tiles. We found, however, that JPEG2000 tile access is much less efficient than accessing an image composed of a single tile (i.e. "untiled"). This problem is particularly acute when image data access is via a JPIP server (which is what is used at HiROC) that intelligently delivers only that part of an image that is requested by the client. JPEG2000 precincts are a much better match to the intent of JAI tiles and can be accessed much more efficiently than JPEG2000 codestream tiles (which contain precincts) while providing the same piecemeal data delivery effect. Unfortunately, the JAI software interface to the JP2 file and its JPEG2000 codestream content only relates tiles to tiles instead of relating precincts to tiles, so the benefits of JPEG2000 precincts are not being realized by the JAI tile management software. This creates a dilemma for JAI users needing to efficiently access large JPEG2000 codestreams: Tiling the JPEG2000 codestream can be expected to reduce the data rendering performance, but not tiling the codestream defeats the resource efficiency effects of the JAI tile cache system. It should be possible for the application software to take a makeshift approach by getting the precinct size corresponding to the resolution level being accessed (its easiest to simply assume access at full resolution; dynamic resolution access is another powerful feature of JPEG2000 that JAI does not handle well) and then pulling image chunks corresponding to codestream precincts, possibly holding them in JAI tiles. This does not take advantage of the JAI automatic tile management, but until the JAI interface to JP2/JPEG2000 image data sources is taught to handle these sources intelligently there will remain a significant functional mismatch.

Bradford Castalia
Principal Systems Analyst
HiRISE Operations Center
University of Arizona

Ken Warner

Have you looked at third party JPEG2000 API's? Have you thought about using
Flash?

KRPano.com has a tiling capability that is very flexible. Of course it only
uses JPEG but given the flexibility of the tiling, not a bad trade off.

-----
# Rendering of up to several giga-pixel large panoramas
Additionally to the "Piecewise" loading the panorama image can be
"multi-resolution". Only the tiles of the "current" resolution will be loaded.
So it will be possible to render gigantic panoramas without loading the whole
panorama.
more information...
----

jai-imageio@javadesktop.org wrote:
> At the HiRISE Operations Center we have been producing rather large images and storing them in JP2 format for distribution (http://hirise.lpl.arizona.edu).
> Bradford Castalia
> Principal Systems Analyst
> HiRISE Operations Center
> University of Arizona
> [Message sent by forum member 'castalia' (castalia)]
>
> http://forums.java.net/jive/thread.jspa?messageID=343989
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Simone Giannecchini

You need to:
1> mosaic the single tiles
2> scale down the resulting mosaic
3> write down the result

Simone.
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: +39 0584983027
fax: +39 0584983027
mob: +39 333 8128928

http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

On Wed, Apr 22, 2009 at 6:35 PM, wrote:
> Hello, I discovered JAI not too long ago, and it looks like a powerful API and I am excited to do some work with it. I have a question though, regarding working with large images. I do not know enough about the capability of JAI/IO, so I want to see if what I want to do is even possible.
>
> The image I am working with is 50000 pixels square, and is in 24 bit color space. Uncompressed, it is about 7G on my hard disk. Right now, the image is made of a 10x10 grid of 5000 pixel square images each. What I want to do is to merge these together, and then scale them down using bicubic downsampling, and save the result. The tiles can not be scaled down individually, because this will create seams.
>
> Will JAI/IO allow me to work with this without running out of memory? I think it is necessary for Java to use disk paging in order for this to work because it cannot fit the whole uncompressed image into the VM memory.
>
> Help or input would be greatly appreciated. Cheers!
> [Message sent by forum member 'ashmore' (ashmore)]
>
> http://forums.java.net/jive/thread.jspa?messageID=343264
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
> For additional commands, e-mail: interest-help@jai-imageio.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net