Skip to main content

vector primitives

3 replies [Last post]
Joined: 2003-09-22

I'm convinced that Java isn't really much slower than native code for most purposes....... except......
What continues to drive me to use native code sometimes is the lack of support within Java for the vector processing units found on modern CPUs. It's not really viable to use Java to write applications involving signal processing (audio apps, for eg.), video processing, games etc. until we can use the hardware that's available for doing that kind of thing.
I use a G5 Macintosh which has a monster vector processor in it, and I do a lot of code that can often run more than 5 times faster when the vector unit is being used properly. But if I code in Java I have to ignore all that power - it just sits idle, and for quite a few applications I have to abandon Java and descend back into the swamp because Java just isn't fast enough.

Implementing vector types in Java (and ideally a bunch of useful vector code like FFTs etc.) and letting the VM worry about the quirks of the actual hardware would be a dream. Ideally the Java VM would have a full set of vector operators, and would either emulate the operation in software or use the hardware processor if it was available - we'd still have "write once, run anywhere", except that if you're running on a machine with a decent vector unit you'll run a hell of a lot faster.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Joined: 2003-06-17

This is stuff is far too "niche" to justify extensions in the language, typesystem, bytecode & other low-level specs. We can have this with new APIs.

For example, JSR-84 (now withdrawn) asked support for operations like FMA. I guess this JSR failed because it proposed a large chage (a new fastfp modifier / math behavior), as automatic replacement of a*b+c by an FMA instructions would violate the rules of Java's maths (even in non-strict maths, stuff like FMA cannot be used).

But an API-based solution, like a method java.lang.Math.fma(), wouldn't create any problem:

* Existing code is not affected;
* New API can specify whatever semantics they desire, including expected precision, rounding behavior, support for meta-numbers (signed zeroes, infinites and NaNs);
* Performance would be the same if the JIT compiler intrinsifies calls to these special methods;
* Doesn't create a burden for slower JVM implementers: if the JIT optimizations like above are difficult, it's trivial to implement these methods with JNI code in order to be compliant, only diasdvantage is performance.

The single disadvantage is easiness. Of course it would be better if the JIT compiler could detect opportunities for special instructions and vectorization in arbitrary code. But these optimizations are very difficult, if completely automatic (without new basic types, language syntax or special bytecode constructs to make vectorizable code explicit), so the more seamless implementation would certainly be only viable in server VMs like HotSpot Server even thoush these things are really cool for client-side code like games with advanced graphics.

In addition to fma, vectorization and similar things, I'd like to have new efficient APIs to emulate many efficient constructs that are usually possible only in Assembly languages. For example, sometimes you want to compute an integer division and remainer for the same values. Or you want to know if there is any carry after an integer addition... in Assembly this is trivial, just look at the carry flag, but in Java you typically need to perform a new operation after the addition, masking values and testing specific bits of the input values to discover if there's a carry. Example of pure-Java code that could become much simpler and faster with new APIs for these things: BigInteger -- just look at the sources, it's insane, it does double or even triple of the work, compared to a hand-coded ASM implementation.

I think that good JITs like HotSpot will detect many patterns of bytecode that results of programming idioms required to do these things, and to emulate unsigned arithmetics etc., but optimizers have limitations and once again only the server JVMs do it all. Run a numeric benchmark like SciMark2 testing HotSpot Client against Server. HotSpot Server 5.0 produces exactly the same overall score of the C-language version (hoorrah!), but HotSpot Cient is 2,4X slower (170Mflops against 404Mflops in my machine), and this disadvantage translates DIRECTLY to the performance of any client-side app that must run on the Client VM, either because Server requires additional setup, or because it has worse footprint and loading time. But with a little help of a few efficient primitives, implemented as new APIs, even in a client-optimized JIT could produce very good code without a big effort -- because we're offloading optimizations from the JIT compiler to the programmer, who decides manually where to use critical instructions or constructs. The purist inside me hates that, for it's the pragmatic solution, and much less ugly than needing some JNI to implement the kernel of your multimedia, DSP or 3D SFX library (like Sun itself does, for example in JMF).

Joined: 2003-09-22

To clarify - I'm not proposing that things like ffts etc. be added to the language sytnax as operators. I totally agree that that would be craziness. I'm also not really proposing automatic vectorisation (though if you could declare vectors, then this option would obviously be opened up).

What I'm proposing is just some kind of extension to the array syntax, i guess.

Then you could write APIs to do operations on vectors (your FFTs etc.), and these APIs would presumably be written in native code, passing the vectors across as NIO buffers, but could also have fallback reference implementations in Java.

I disagree that vectors are "niche". I would say that >70% of the programs I work with day to day fit in that niche. Let's see - I use photoshop, sometimes do a bit of video editing, play music, play games - in fact all of the things that people (consumers) actually buy a computer for (apart from web browsing and email). I suspect that you're generally more used to server side code, and mostly deal with text - more and more people are working with pictures, sound, movies and even 3d, and all these things can benefit from vectorisation.

There are huge domains where vectors are the natural representation.The only reason they're "niche" is that they're usually rather difficult to work with, partly because people aren't used to them and partly because of generally poor language support. That's what I'm proposing a solution for.

Joined: 2004-04-27

Yes, you are right. FFT, image filtering and other image processing (don't want a iLife like applicaions written in java?), voice processing, scientific apps (imagine a java grid distributed environment running on G5s of Hammers)... and GUIs for all of us...
..all this problems long for a good support for new instructions and registers.