Skip to main content

Solve Jar Hell (JDK10?)

12 replies [Last post]
aslom
Offline
Joined: 2003-06-27

i think that this alone even if 80/20 solution would be a big productivity boost for non trivial applications that requires more than 10 JAR files and have cascading JAR dependencies.

what i want is some way to declare that my app requires JAR of this version (or newer) and each JAR file can also declare what it requires - when JVM is started with special option it is then doing full dependency check and classloader throws exception if there are unfulfilled or conflicting dependencies.

i think current package versioning could be extended for more details see: http://www.extreme.indiana.edu/~aslom/blog/java/2003/02/20/JavaPackageVe...

this change alone is 10x more valuable than generics and i would not mind to call it then JDKX and skip version 6-9 :)

thanks,

alek

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
paulbrickell
Offline
Joined: 2003-09-17

Maybe this thread and the one here....

http://forums.java.net/jive/thread.jspa?forumID=23&threadID=75

Should be combined. This problem need sorting. How can we apply pressure?

mthornton
Offline
Joined: 2003-06-10

My first gripe is the number of people who don't bother to put any form of version information in their Manifests.
Once you have sensible version information, WebStart is a great way of ensuring that you get exactly what you expect in your class path.

vhi
Offline
Joined: 2004-10-11

I couldn't agree more. Look at Xerces, Xalan, even Sun supplied Java libraries. None of them have version information. I think this is partly because under the current situations, it is useless. I have to write custom classloader, non standard extensions and all just to ensure that I load the correct classes. I cannot use WebStart in many cases (server-side).

My requirement is this:

1. I should be able to say what version of a Java package I want in my code (I am assuming that a Java package, and not class is the basic unit of versionable entity), and use the classes from that version.
E.g.

@Version(EQUALS_OR_GREATER, 1.2)
import org.apache.xpath.*;

2. It should be possible to have multiple versions of the same packages in the classpath, and the JVM should load classes from the specified package version.

3. The JVM should support the package management, and dependency tracking by default. I do not want to specify configurations, classpath hacks to get it working. If it is not easy to use, nobody will use it.

We need to make Java run-time behaviour much more reliable. I don't know how JVM sharing can be done gracefully and reliably until the package versioning is fixed. Versioning becomes more and more necessary as the number of Java components increase.

mthornton
Offline
Joined: 2003-06-10

> I couldn't agree more. Look at Xerces, Xalan, even
> Sun supplied Java libraries. None of them have
> version information. I think this is partly because
> under the current situations, it is useless.

Not entirely useless. My JNLP deployment process extracts whatever version information is present and uses this to form the file name (so that I have things like forms__v1.0.2.jar and forms__v1.0.4 in my JNLP server area), and inserts the relevant version into the JNLP file being deployed. Unfortunately unless I insert the version myself, antlr.jar remains just as it is.

> classes. I cannot use WebStart in many cases
> (server-side).

In my opinion WebStart ought to be extended so that it can be used for server side processes and command line applications, everywhere in fact. There is no reason why the JNLP mechanism can't be used in these cases, just that the current client deployed with Java doesn't permit it.

vhi
Offline
Joined: 2004-10-11

But this still does not solve multiple versions problem.
Suppose I use Xerces 1.0 for my application, and the plugin from my customer uses Xerces 2.0. At present, it is not possible because only the classes encountered first are used. Thus, I cannot have multiple versions of same class in the same JVM.

I think what we need is an integrated, whole hearted support within JVM.

patrikbeno
Offline
Joined: 2004-10-11

As you might know, this is possible with multple classloaders. But it may not solve the problem you describe.

If the mentioned plugin uses xerces 2, you may load whole plugin's jar with special classloader that also loads xerces 2 libraries upon plugin's demand for xerces' classes. So far so good, everybody's happy.

But this might not work in some scenarios.

If the plugin parses XML with xerces2 and returns loaded DOM tree to your application, your application may fail on ClassCastException if it attempts to cast returned DOM tree to Xerces1 class. (If app<->plugin communication is based on common classes/interfaces, this problem will not occur)

What I am trying to say is that version conflict cannot be solved so easily. You may not have two different versions of the same class within the same classloader's scope. [b]This is unsolvable and it cannot be changed, AFAIK.[/b]

Partial solution of the problem would be loading conflicting classes using separate classloaders, as I mentioned. However, if code bases from different classloaders interact (and interaction is usually data exchange), class version incompatibilities may occur
. [b]And this problem is also unsolvable, i think[/b]

IMHO we may not expect Java to provide us with any kind of magic that will solve our own configuration and design problems/failures.

[b]This was just an attempt to provide people who just want too much with a little in-depth analysis :-)
As I already said, [i]I agree that package/jar/module dependecies and requirements should be more precisely specifiable and application/runtime should be able to verify these requirements.[/i][/b] Well, said it twice to avoid confusion :-)

Duh, I am too verbose but hopefully I made myself clear :-)

vhi
Offline
Joined: 2004-10-11

>> (If app<->plugin communication is based on common classes/interfaces, this problem will not occur)

Actually, I mean only the "common interface, different implementations" scenario. Of course if the interfaces are different, the contract is different, and the plugin must confirm.

>> What I am trying to say is that version conflict cannot be solved so easily. You may not have two different versions of the same class within the same classloader's scope. This is unsolvable and it cannot be changed, AFAIK.

I do not understand why a classloader cannot have different versions of a class, IF the version information is with the classloader. Of course, the caller would have to mention which version of class is required, and the classloader can provide it. In Java we can say that a class is a basic versionable entitiy. Consider how DLLs are being loaded in .NET with side-by-side versioning.

>> Partial solution of the problem would be loading conflicting classes using separate classloaders, as I mentioned. However, if code bases from different classloaders interact (and interaction is usually data exchange), class version incompatibilities may occur
. And this problem is also unsolvable, i think

I agree that using different classloaders are unreliable.
And IMHO, different classloaders are not required because if the version information is with the classloader, it can load different versions of the same class and provide it to the caller which demands a specific version.

>> IMHO we may not expect Java to provide us with any kind of magic that will solve our own configuration and design problems/failures.
Of course, Java is not magical ;-)

patrikbeno
Offline
Joined: 2004-10-11

Quoting: [i]A bug first appears in some version of a vendors package and may or may not continue to be a problem in subsequent versions.....[/i]

damn it, so your rationale is: let's extend "feature backward compatible" to "bug backward compatible"
And this is supposed to solve any hell? (Exaggerating :-))

Anyway, [b]I agree[/b] that package/jar/module dependecies and requirements should be more precisely specifiable and application/runtime should be able to verify these requirements.

[b]Off topic:[/b] Ad. your sarcastic attack on generics :-): jar hell can be solved right now by properly configuring your runtime environment, that's all. For generics, we need compiler and JVM's support. As for value: you configure your runtime properly only once and you're done while generics can help you throughout the whole development process

vhi
Offline
Joined: 2004-10-11

Judging by the luke-warm response to this request, I guess that solving JAR-Hell is of lesser priority to most of the developers around here. But, if you feel that solving JAR-Hell is an important issue, please indicate your approval in this thread so that Sun may notice it. If there are sufficient votes, then we can write a RFC for this purpose.

zander
Offline
Joined: 2003-06-13

I suggest you write an application (in java) that opens a Jar and loads (creates a Class object) each .class file in there. Each exception you get means you miss a dependency; print missing class name and your done.

Should be easy enough to create. Why don't you create it and make it open source :)
Submit it to Maven if you like! (if its not already available)

aslom
Offline
Joined: 2003-06-27

haha that makes me laugh and envy you as you apparently have not been in (Sun-endorsed) JarHell ;->

the problem is not in missing classes or dependency on one class - i can easily do such checks in ant with

problem is in maintaining version dependencies and discovering potential problems that do not show up
until application is already (partially) loaded and running - when you see lovely MethodInvocationException, ClassNotFound, IncompatibleClassChangeError or some variation with hundreds lines of useless stack traces ...

try google Jar Hell and you find more discussion and interesting examples:
http://www.neward.net/ted/weblog/index.jsp?date=20030215
http://www.mail-archive.com/general@jakarta.apache.org/msg04506.html
http://www.oreillynet.com/cs/user/view/cs_msg/8632
http://www.almaer.com/blog/archives/000368.html
http://aspn.activestate.com/ASPN/Mail/Message/Apache-Soap-Dev/740007
http://www.realityinteractive.com/rgrzywinski/archives/000089.html
http://wiki.apache.org/cocoon/EndorsedLibsProblem
http://www.dehora.net/journal/2003/02/xerceshell.html
http://www.dehora.net/journal/2002/12/classpath_hell.html
http://www.clarkware.com/cgi/blosxom/2002/12/17#Software/Tools/JavasAchi...
http://www.freeroller.net/page/fate/?anchor=the_evils_of_commons_logging

there are already attempts under way to get this fixed
http://incubator.apache.org/depot/version/jar-hell.html
http://krysalis.org/version/jar-hell.html
http://classworlds.codehaus.org/

but this whole mess seems to me like a perfect example where standard is required so it is easy to start JVM in "dependency version checking" mode that will reject JARs that do not have enough metadata, do not meet dependency requirements, or there is more than one JAR file for the same package but with different versions (very common ...)

i hope the issue is now clarified ...

thanks,

alek

vhi
Offline
Joined: 2004-10-11

There is another thread "Administration of Libaries" which you may want to refer to. Maybe if we can have sufficient support, Sun would do something about it.