Smaller Java Core
Many times users don't need JNDI, JDBC, CORBA, XML, etc. Extract them as libraries separate from the minimal core to be distributed with the applications that use them (for offline distributions) or downloaded upon installation (for online distributions).
That is not to say that I disagree with having an "uber" install available. Developers would want the full install. PCs with Java pre-installed should come with all the packages in the current JRE. But requiring users to download a large runtime over a slow connection just to run a simple applet is a LARGE barrier to entry that has hurt Java's market penetration.
A modularized runtime would also allow more frequent bug fixes. If the problem is fully contained within a module, just release the updated module. Only do Java point updates when the core needs patching or when multiple modules need to be simultaneously changed to fix a single bug.