Skip to main content

[JAI-IMAGEIO] Extending CameraRaw support.

4 replies [Last post]
Anonymous

First of all, jrawio has moved and now is on java.net, as a child project of
imageio (jrawio.dev.java.net). Thanks to Brian and the imageio staff for
hosting. :-)

As v1.0 is going thru some Release Candidates for testing, I'm planning the
next major version with new features.

I've got in touch with the developers of some Java applications that process
photos by using ImageIO and they are interested in adding Camera Raw
support. What is emerging is that a reasonable feature would be that this
support should be implemented by just adding jrawio.jar in the classpath.
After all this is the typical behaviour of ImageIO SPIs. These apps don't
want to handle cameraraw files in a specific fashion. I'd call this class of
applications 'raw-unaware'.

But jrawio on purpose provides the application with an unprocessed image. If
you deal with camera raw images, in most cases it's up to the application to
process them in the best way - this is why camera manufacturers have
introduced camera raw formats. I'd call this class of applications
'raw-aware'.

The point is that jrawio 1.0 supports only raw-aware applications. So I'd
like to introduce support for raw-unaware applications in v1.5.

I've already got the code and the knowledge to do the required number
crunching for some formats. I'm just asking an opinion on which is the best
architectural way to go. I see two options:

1. introducing a specific ImageReadParam subclass with something like a
'processed' flag; raw-aware would set it to false and raw-unaware apps would
set it to true.

2. providing a separate 'wrapper' SPI (also in a separate jarfile) - for
simplicity I call this jrawio2.jar. This would register ImageReaderSpis that
supersede those in jrawio.jar; they would internally call jrawio.jar readers
and add postprocessing. raw-aware apps would only put jrawio.jar in the
classpath; raw-unaware apps would add also jrawio2.jar.

The former approach looks like simpler and clearer. Nevertheless it puzzles
me about what should be the default settings for the 'processed' flag. On
one hand defaulting it to 'false' would mantain jrawio philosophy of being
natively designed to exploit cameraraw formats features; but raw-unaware
apps should be explicitly set the flag to true, so that just dropping
jrawio,jar in the classpath would not be enough.

Perhaps the best solution would be:

3. go with the ImageReadParam approach and to provide two jar versions,
jrawio.jar and jrawio2.jar. No more wrappers here, but mutually exclusive
purposes: the former defaults the 'processed' flag to false, the latter to
true. Raw-aware apps would use the former, raw-unaware apps the latter.

Opinions?

Thanks.

--
Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
http://www.tidalwave.it/blog - Fabrizio.Giudici@tidalwave.it
mobile: +39 348.150.6941 - fax: +39 027.005.105.36

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Bob Deen

> I've already got the code and the knowledge to do the required number
> crunching for some formats. I'm just asking an opinion on which is the
> best architectural way to go. I see two options:

...

> The former approach looks like simpler and clearer. Nevertheless it
> puzzles me about what should be the default settings for the 'processed'
> flag. On one hand defaulting it to 'false' would mantain jrawio
> philosophy of being natively designed to exploit cameraraw formats
> features; but raw-unaware apps should be explicitly set the flag to
> true, so that just dropping jrawio,jar in the classpath would not be
> enough.

Personally I like option 1 with the default set to "true". That way unaware
apps can simply and magically read the images and do something with
them (display, standard processing, whatever) without even knowing that
there's something special about the image. Aware apps need to know
about the RAW API anyway so having them set the flag to "false" to get
raw access doesn't seem to be a big deal.

So the general philosophy would be, work in the simple case, with access
available for the more complex case.

I wouldn't worry about the "natively designed to exploit cameraraw formats
features" part. Just because that's not the default case doesn't negate in
any way the value of having those features. It should be obvious that it is
in fact designed to exploit the raw features from the documentation...
I suspect 90%+ of it will talk about those features. So I don't think
anyone
will get the wrong impression.

I very much dislike on an architectural basis the idea of having two jars,
*especially* mutually exclusive ones. It would prevent you from writing
apps
that do standard things to images that don't care about the details (say,
showing thumbnails of what's available) and at the same time allowing other
parts of the code to do more sophisticated raw-aware processing. It's easy
to say, well if part of the app is aware why not make the whole thing
aware...
but that makes it impossible to reuse components to do the simple stuff.

You also might be able to make use of an ImageTypeSpecifier rather than
a flag, in a similar way to what has been discussed here recently regarding
use of the embedded ICC profile for... I think it was TIFF images. But
whatever mechanism you choose, make sure the defaults are such that an
unaware app will get the processed (but correct) image.

-Bob

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Fabrizio Giudici

On Feb 6, 2006, at 23:01 , Bob Deen wrote:

>> I've already got the code and the knowledge to do the required
>> number crunching for some formats. I'm just asking an opinion on
>> which is the best architectural way to go. I see two options:
>
> ...
>
>> The former approach looks like simpler and clearer. Nevertheless
>> it puzzles me about what should be the default settings for the
>> 'processed' flag. On one hand defaulting it to 'false' would
>> mantain jrawio philosophy of being natively designed to exploit
>> cameraraw formats features; but raw-unaware apps should be
>> explicitly set the flag to true, so that just dropping jrawio,jar
>> in the classpath would not be enough.
>
> Personally I like option 1 with the default set to "true". That
> way unaware
> apps can simply and magically read the images and do something with
> them (display, standard processing, whatever) without even knowing
> that
> there's something special about the image. Aware apps need to know
> about the RAW API anyway so having them set the flag to "false" to get
> raw access doesn't seem to be a big deal.

Ok, I'm going this way. Which unfortunately leads me to another
problem. :-)

I'm explaining it with an example. A given digital camera has a
sensor made of
3030 x 2020 points. So, an unprocessed image has getWidth() = 3030,
getHeight() = 2020.
When the image is processed (developed), the size shrinks to 3000 x
2000. So the
processed image has getWidth() = 3000, getHeight() = 2000.

But the problem is that getWidth() and getHeight() don't have a
ImageReaderParam
to specify if we are loading the raw raster or the processed data. I
have to choose
which pair of values I have to return - I'd say 3000x2000, in
coherence with the
approach of returning the processed image by default.

Now what happens is:

ImageReader ir = ...
ir.setInput(...);

int w = ir.getWidth(0); // w = 3000
int h = ir.getHeight(0); // h = 2000

CameraRawImageReaderParam param = (CameraRawImageReaderParam)
ir.getDefaultReadParam();
ir.setProcessingEnabled(false);
BufferedImage image = ir.read(0, param);

int w2 = image.getWidth(); // w2 = 3030
int h2 = image.getHeight(); // h2 = 2020

assert w == w2; // assertion fails
assert h == h2; // assertion fails

The programmer could learn about the raw image size by using
getRAWImageType(0).getSampleModel().getWidth(), getHeight().

But is it acceptable that ir.getWidth(0) != ir.read(0, param).getWidth
() or is this breaking
ImageReader's contract?

PS This is not a special case: this little shrinking happens with
every camera raw format
as it deals with demosaicing. Furthermore, some cameras allow the
photographer to choose
between a large and small photo size (the smaller is just a crop of
the large).

--
Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
http://www.tidalwave.it/blog - Fabrizio.Giudici@tidalwave.it
mobile: +39 348.150.6941 - fax: +39 027.005.105.36

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Robert Engels

Can you give a BRIEF overview of the camera raw format, and how it differs
from other image types?

I thought it was just that the pixels as raw values (r,g,b or c,m,y,k, or
however the color is represented) along with an ICC profile so they can be
converted to other color spaces?

-----Original Message-----
From: fabrizio.giudici@tidalwave.it
[mailto:fabrizio.giudici@tidalwave.it]
Sent: Friday, February 03, 2006 6:16 AM
To: interest@jai-imageio.dev.java.net
Subject: [JAI-IMAGEIO] Extending CameraRaw support.

First of all, jrawio has moved and now is on java.net, as a child project of
imageio (jrawio.dev.java.net). Thanks to Brian and the imageio staff for
hosting. :-)

As v1.0 is going thru some Release Candidates for testing, I'm planning the
next major version with new features.

I've got in touch with the developers of some Java applications that process
photos by using ImageIO and they are interested in adding Camera Raw
support. What is emerging is that a reasonable feature would be that this
support should be implemented by just adding jrawio.jar in the classpath.
After all this is the typical behaviour of ImageIO SPIs. These apps don't
want to handle cameraraw files in a specific fashion. I'd call this class of
applications 'raw-unaware'.

But jrawio on purpose provides the application with an unprocessed image. If
you deal with camera raw images, in most cases it's up to the application to
process them in the best way - this is why camera manufacturers have
introduced camera raw formats. I'd call this class of applications
'raw-aware'.

The point is that jrawio 1.0 supports only raw-aware applications. So I'd
like to introduce support for raw-unaware applications in v1.5.

I've already got the code and the knowledge to do the required number
crunching for some formats. I'm just asking an opinion on which is the best
architectural way to go. I see two options:

1. introducing a specific ImageReadParam subclass with something like a
'processed' flag; raw-aware would set it to false and raw-unaware apps would
set it to true.

2. providing a separate 'wrapper' SPI (also in a separate jarfile) - for
simplicity I call this jrawio2.jar. This would register ImageReaderSpis that
supersede those in jrawio.jar; they would internally call jrawio.jar readers
and add postprocessing. raw-aware apps would only put jrawio.jar in the
classpath; raw-unaware apps would add also jrawio2.jar.

The former approach looks like simpler and clearer. Nevertheless it puzzles
me about what should be the default settings for the 'processed' flag. On
one hand defaulting it to 'false' would mantain jrawio philosophy of being
natively designed to exploit cameraraw formats features; but raw-unaware
apps should be explicitly set the flag to true, so that just dropping
jrawio,jar in the classpath would not be enough.

Perhaps the best solution would be:

3. go with the ImageReadParam approach and to provide two jar versions,
jrawio.jar and jrawio2.jar. No more wrappers here, but mutually exclusive
purposes: the former defaults the 'processed' flag to false, the latter to
true. Raw-aware apps would use the former, raw-unaware apps the latter.

Opinions?

Thanks.

--
Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
http://www.tidalwave.it/blog - Fabrizio.Giudici@tidalwave.it
mobile: +39 348.150.6941 - fax: +39 027.005.105.36

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net

Fabrizio Giudici

On Feb 3, 2006, at 15:17 , Robert Engels wrote:

> Can you give a BRIEF overview of the camera raw format, and how it
> differs
> from other image types?
>
> I thought it was just that the pixels as raw values (r,g,b or
> c,m,y,k, or
> however the color is represented) along with an ICC profile so they
> can be
> converted to other color spaces?

Also. But most of all you don't have all of the r-g-b samples for
each pixel.
The sensor is usually made of patterns such as

GR
BG

that is every four pixels you have only two green, one blue and one red.
So, before applying an ICC profile, missing samples must be
interpolated.

Look here: http://en.wikipedia.org/wiki/Bayer_filter, pictures say it
most
than many words.

Another conceptual point: a raw format is intended as a "digital
negative":
it has to be "developed". That is, all relevant postprocessing
(including
sharpening at least) is recorded as a metainfo by the camera (e.g.
sharpen=90%)
and it should be responsibilty of a raw- aware application to apply
it (this
is done so everything is reversible and you can override settings at
shot time
without introducing another bunch of noise).

--
Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere."
http://www.tidalwave.it/blog - Fabrizio.Giudici@tidalwave.it
mobile: +39 348.150.6941 - fax: +39 027.005.105.36

---------------------------------------------------------------------
To unsubscribe, e-mail: interest-unsubscribe@jai-imageio.dev.java.net
For additional commands, e-mail: interest-help@jai-imageio.dev.java.net