Skip to main content

Cache pack200 bugs, mistakes http port, messses up with nativelibs, slow

8 replies [Last post]
jayv
Offline
Joined: 2007-08-14

With the new plugin out I was anxious to give it a go, unfortunately I stumbled upon some misbehaving code in the jar/cache downloader...

When tracing turned on in console I detected this, (as I suspected something wrong was happening, startup was unreasonably slow) The only thing I did was change my applet to use pack200. Before that it didn't have this problem.

It's currently deployed on localhost port 8080 please note the connection refused errors, the applet is going to port 80 first for version check ??!?!!

Both my applet tag and jnlp file have the correct port number in the codebase param. Jar hrefs are relative paths so no issue there either.
.
The 2nd problem I notice is that my native library doesn't get cached at all, unfortunately it's 5MB so it hurts each time :(

3rd problem I noticed that the non-pack200 version downloaded native libs for other plaforms (linux, macx86, macppc), although when I check the "merged" JNLP file from in the Java Cache Viewer only the windows nativelib entry is present (as expected)!?? That means 15MB download penalty in my case, really a blocker if you ask me :(

Apart from that and the branding problem (http://forums.java.net/jive/message.jspa?messageID=267888) it's sweet :)

Running java version "1.6.0_10-beta"
Java(TM) SE Runtime Environment (build 1.6.0_10-beta-b14)
Java HotSpot(TM) Client VM (build 11.0-b11, mixed mode, sharing)

Logs:
-------
network: Cache entry found [url: http://localhost:8080/testapp/lib/ezmorph-1.0.4.jar.pack.gz, version: null]
network: Connecting http://localhost/testapp/lib/ezmorph-1.0.4.jar.pack.gz.pack.gz with proxy=DIRECT
java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at com.sun.deploy.net.HttpUtils.followRedirects(Unknown Source)
at com.sun.deploy.net.BasicHttpRequest.doRequest(Unknown Source)
at com.sun.deploy.net.BasicHttpRequest.doGetRequestEX(Unknown Source)
at com.sun.deploy.net.DownloadEngine.actionDownload(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResource(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadJarFiles(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadEagerorAll(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.downloadResources(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.prepareLaunchFile(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.loadJarFiles(Unknown Source)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
network: Connecting http://localhost:8080/testapp/lib/ezmorph-1.0.4.jar.pack.gz with proxy=DIRECT
network: ResponseCode for http://localhost:8080/testapp/lib/ezmorph-1.0.4.jar.pack.gz : 200
network: Encoding for http://localhost:8080/testapp/lib/ezmorph-1.0.4.jar.pack.gz : null
network: Disconnect connection to http://localhost:8080/testapp/lib/ezmorph-1.0.4.jar.pack.gz

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
ngthomas
Offline
Joined: 2004-06-03

Looking at the trace, it seems like pack200 may not be properly enabled. How do you deploy the pack200 JARs for your applet ?

For 6u10, the best way is to use the new java system property "jnlp.packEnabled":

http://java.sun.com/javase/downloads/ea/6u10/newJavaSystemProperties.jsp

For the problem that you are running into with native library, can you please provide more information on how to reproduce the problem ? Do you use the jnlp nativelib tag for the native JAR? A testcase would be very helpful for us to look into it further.

thanks,
Thomas

jayv
Offline
Joined: 2007-08-14

> For 6u10, the best way is to use the new java system
> property "jnlp.packEnabled":

That's the one I used, without that parameter it complained it couldn't find or unzip the JARs because I only deployed the *.jar.pack.gz files. If I remember correctly.

> For the problem that you are running into with native
> library, can you please provide more information on
> how to reproduce the problem ? Do you use the jnlp
> nativelib tag for the native JAR? A testcase would
> be very helpful for us to look into it further.

Correct, below is a snippet from my non-Pack200 JNLP (the pack200 JNLP is the same except for the pack.gz extensions and the packEnabled param). As I said before I didn't actually see the plugin fetch all those jars, but I noticed them while using Java Cache Viewer > Resources:












I checked again to make sure for a webstart app (where I didn't had this behaviour with Java6u4) and then again with my current applet (without pack200) and actually no libs for other platforms than windows are downloaded. This is strange because I had them there at some point.

Is it perhaps possible that pack200 unpacks its nativelibs in the cache so that there are both entries with and without .pack.gz? This way it's possible that what I saw has nothing to do with the non-pack200 version, but would indicate that the pack200 mechanism downloads and unpacks all nativelibs without respecting the OS param.

I will try to setup my app for pack200 again by tomorrow because I don't understand what's going on.

ngthomas
Offline
Joined: 2004-06-03

> below is a snippet from my non-Pack200 JNLP (the pack200 JNLP is the same except for the pack.gz extensions and the packEnabled param).

This sounds like the problem. To enable pack200, you only need to set the system property, the jar href in the jnlp file should remains the same (should not add the .pack.gz extension to it)

e.g:



Notice only "foo.jar" is used, which is correct and required. We will automatically contact the server for foo.jar.pack.gz, when the jnlp.packEnabled property is set.

Can you please correct your pack200 enabled jnlp file, and try again ?

thanks,
Thomas

Message was edited by: ngthomas

jayv
Offline
Joined: 2007-08-14

I adapted my JNLP according to your suggestions and managed to get it to work, no more slowdowns or wrong libraries are downloaded. But I still believe there's a bug with Pack200 and a non-standard HTTP port.

I'm using the Netbeans 6.1 Dev Pack200ExtendedTask with ANT to do the repack / pack operation. I've verified my JARs (both regular and pack200) manually with unpack200 and jarsigner -verify... they all verify OK.

It does seems to work fine when I use file:// URLs to my tomcat's deployment folder, but when I use a HTTP URL to my tomcat port 8080 it fails.
I then tried flipping the jnlp.packEnabled to false and that does fix the problem,

I then again suspected a combination of Pack200 with the port 8080, so I reconfigured my tomcat to port 80 and now it works over HTTP with Pack200.
I've also disabled all proxy servers for Java to rule them out.

The codebase (if it's of any help) is specified in the applet tag like this:
codebase="http://localhost:8080/mytest/"

My perception of the situation: it tries to download the .jar.pack.gz file from port 80 first, it fails because it should be 8080. Then it falls back to the regular .jar URL using the correct port but after download it tries to unpack200 the regular JAR (notice content-encoding:pack200-gzip on the JAR download).

I see possibly 2 areas for bugs, the pack200 URL construction adding .pack(.gz) to the URL loses the port, and the content-encoding that doesn't get reset on fallback.

Below you'll find the stacktrace when using Pack200-enabled JNLP over HTTP with port 8080.

java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at com.sun.deploy.net.HttpUtils.followRedirects(Unknown Source)
at com.sun.deploy.net.BasicHttpRequest.doRequest(Unknown Source)
at com.sun.deploy.net.BasicHttpRequest.doGetRequestEX(Unknown Source)
at com.sun.deploy.net.DownloadEngine.actionDownload(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResource(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadJarFiles(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadEagerorAll(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.downloadResources(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.prepareLaunchFile(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.loadJarFiles(Unknown Source)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
network: Connecting http://localhost:8080/mytest/mytestFX.jar with proxy=DIRECT
network: ResponseCode for http://localhost:8080/mytest/mytestFX.jar : 200
network: Encoding for http://localhost:8080/mytest/mytestFX.jar : null
network: Sever response: (length: 110647, lastModified: Tue Apr 08 11:22:08 CEST 2008, downloadVersion: null, mimeType: application/java-archive)
network: Downloading resource: http://localhost:8080/mytest/mytestFX.jar
Content-Length: 110.647
Content-Encoding: pack200-gzip
Applet Status: exception: null.
exception: null.
com.sun.deploy.net.FailedDownloadException: Unable to load resource: http://localhost:8080/mytest/mytestFX.jar
at com.sun.deploy.net.DownloadEngine.actionDownload(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResource(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadJarFiles(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadEagerorAll(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.downloadResources(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.prepareLaunchFile(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.loadJarFiles(Unknown Source)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by:
java.io.IOException: Not in GZIP format
at java.util.zip.GZIPInputStream.readHeader(Unknown Source)
at java.util.zip.GZIPInputStream.(Unknown Source)
at java.util.zip.GZIPInputStream.(Unknown Source)
at com.sun.deploy.net.HttpDownloadHelper.download(Unknown Source)
at com.sun.deploy.cache.Cache.downloadResourceToCache(Unknown Source)
at com.sun.deploy.net.DownloadEngine.actionDownload(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResourceCacheEntry(Unknown Source)
at com.sun.deploy.net.DownloadEngine.getResource(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadJarFiles(Unknown Source)
at com.sun.javaws.LaunchDownload.downloadEagerorAll(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.downloadResources(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.prepareLaunchFile(Unknown Source)
at sun.plugin2.applet.JNLP2Manager.loadJarFiles(Unknown Source)
at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Exception: com.sun.deploy.net.FailedDownloadException: Unable to load resource: http://localhost:8080/mytest/mytestFX.jar

ngthomas
Offline
Joined: 2004-06-03

Good to know it's working now.

I will look into the non-standard port issue. Sounds like a bug.

Thanks for the report!

ngthomas
Offline
Joined: 2004-06-03

I tried on my own tomcat, and pack200 works fine on non-standard HTTP port as well. I tested with port 8080 also, and it works for me.

Is there any special setup on your tomcat server ? Any servlet running ?

For the mimetype issue that you report, I noticed in the trace you provide:

network: Connecting http://localhost:8080/mytest/mytestFX.jar with proxy=DIRECT
network: ResponseCode for http://localhost:8080/mytest/mytestFX.jar : 200
network: Encoding for http://localhost:8080/mytest/mytestFX.jar : null
network: Sever response: (length: 110647, lastModified: Tue Apr 08 11:22:08 CEST 2008, downloadVersion: null, mimeType: application/java-archive)
network: Downloading resource: http://localhost:8080/mytest/mytestFX.jar

The above are correct and printed by plugin. mimeType is correct.

Content-Length: 110.647
Content-Encoding: pack200-gzip

These two lines are not printed by java plugin I think ? Do you know where they come from ?

thanks,
Thomas

jayv
Offline
Joined: 2007-08-14

> 8080 also, and it works for me.

:-s

> Is there any special setup on your tomcat server ?

Pretty much out of the box 6.0.14 running some webapps in other contexts. But in the folder hosting the applet there is nothing else but jars and jar.pack.gzs and the htm and jnlp.

To make sure I even installed a basic Apache HTTPD 2.2 with the same results... I'm starting to fear that maybe some anti-virus or firewalling software installed on my laptop is interfering (I can't turn it of unfortunately). If I have time later this week(end) I'll try my case on a personal laptop without all that crap to see if it behaves differently and I'll try to make a smaller archive to send to you so you can investigate.

> For the mimetype issue that you report, I noticed in
> the trace you provide:

The mimetype is OK, it was the content-encoding that worried me:

> Content-Length: 110.647
> Content-Encoding: pack200-gzip
>
> These two lines are not printed by java plugin I
> think ? Do you know where they come from ?

These do pop up in my Java Console window if I'm fast enough to enable full tracing (5) before it tries to download. I'm not sure how something that's not part of the JRE can log there . It's certainly not in my code, it can't even find my JAR ;-) And the filesize matches my non-pack200 JAR's size, but the content-encoding indicates a packed JAR, coincidence ;-)

> thanks,
> Thomas

No problem, it's not blocking me anymore but I still want to help you find the cause of this.

Jo

ngthomas
Offline
Joined: 2004-06-03

Thanks for the reply. You are right, content-encoding is actually printed from plugin, but it looks correct to me in my testcase (see below the my trace).

HelloWorld.jar.pack.gz -> pack200-gzip
jnlp-servlet.jar -> null

I was using an older version of tomcat. I will try with 6.0.14 or apache and see if I can produce the bug.

network: Downloading resource: http://capoon.sfbay.sun.com:8080/HelloWorld/HelloWorld.jar.pack.gz
Content-Length: 1,072
Content-Encoding: pack200-gzip
network: Downloading resource: http://capoon.sfbay.sun.com:8080/HelloWorld/jnlp-servlet.jar
Content-Length: 63,462
Content-Encoding: null