Skip to main content

[Java] BigDecimal Limitation

16 replies [Last post]
jfanatic
Offline
Joined: 2008-08-20

Hi all,

I would like to point out a real limitation in Java related to floating point calculation which for such a rich language is really unfortunate.
We cannot use doubles or floats if we want exact calculations, say in a financial app, because of underlying binary floating point representation, so we should use BigDecimals right? But as you know its immutable so for every bit of calculation we are creating new objects just for fun. For strings one can understand the reason behind immutability that is we do not want to create redundancy in memory for same string literals and avoid unnecessary garbage but for BigDecimal and even for Wrappers making them immutable is actually encouraging garbage as every add, subtract operation will result in a new object. That's a luxury some people cannot afford if they would like to write a performant application. There should be a class just like StringBuffer for repeated manipulations of same numeric object. Your thoughts please.

JFanatic

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
dansiviter
Offline
Joined: 2005-09-26

Right, just to save face on my part, it looks like I got the wrong end of the stick with JVM optimisations [further reading suggests the BigDecimal internals were improved], but my point still stands that BigDecimal is still very quick.

Though a mutable version of BigDecimal would be nice, I feel it's still not a limitation of Java. There are a lot of clever developers out there writing financial applications in Java real-time JVM with no need for this. I agree with a number of other posters here that maybe worth looking at tuning the GC.

stevegal
Offline
Joined: 2004-07-10

Are you talking about garbage collection being a potential issue or the performance of BigDecimal itself? If it's garbage collection, there are lots of things you can do to tune the performance of the gc if you need to, but again, be sure that you need to before you start.You really need to be sure of your problem before you go implementating a solution to an issue that may not be there in reality.

jfanatic
Offline
Joined: 2008-08-20

For the interested, here are the performance test results on jdk1.5.0_11 with default VM options i.e. GC type(presumably serial collector), young generation & heap sizes. I just want to get the sum of 1 million atm transaction's amount in Java and hopefully accurate to a cent (we do care about money :)).

1) Using java primitives the result came in millisecs but not sure about accuracy.
2) With Commons mutable wrappers (thanks ryanh0) approx. same performance but accuracy is the issue.
3) And using BidDecimals i got java.lang.OutOfMemoryError: Java heap space :( just simply because of garbage involved while using BigDecimals.

All of you may agree that one of the factors that can effect the performance of an app is the performance of GC. If an app is to spent most of its execution time in GC we can imagine the impact on performance which is exactly the case here. GC is trying desperately to get some memory free so that app can work but when BigDecimal + associated String instances are taking 99% of heap what else can be done.

Can anyone please explain, the reason behind making these numerics immutable, we should forget multithreaded usage as being the reason. This should be left to app developer to ensure synchronization. Besides all the apps are not multithreaded. When folks at apache can think of a usage of mutable wrappers in real life definitely SUN can also.

public class BigDecimalTest {
public static void main(String[] args) {
final int SIZE = 1000000;
final Random r = new Random();
BigDecimal[] bdList = new BigDecimal[SIZE];
BigDecimal sum = new BigDecimal("0.00");

System.out.println("Initialization Start--->" + System.nanoTime());
for (int i = 0; i < bdList.length; i++) {
bdList[i] = new BigDecimal(Double.toString(r.nextDouble() * 100));
}
System.out.println("Initialization End--->" + System.nanoTime());

System.out.println("Start Summing--->" + System.nanoTime());
for (int i = 0; i < bdList.length; i++) {
sum = sum.add(bdList[i]);
}
System.out.println("Result--->" + System.nanoTime());
}
}

jfanatic
Offline
Joined: 2008-08-20

There is something wrong with the editor here, test code got corrupted. But it should not be hard to imagine what might be going on. just to avoid confusion if it may arise

1st for loop is initializing the BDs with timestamp printed before & after
2nd is just summing the list in sum variable again with timestamp printed before & after

stevegal
Offline
Joined: 2004-07-10

Well, your test code is a bit extreme. You are not allowing anything (much) to be gc'd as you hold all the references to every(most) BigDecimal during the lifecycle of your main. In real life you are unlikely to be in this situation, with an array of 1000000 items held in memory (which by itself must take you close to the default limit of the jvm). I don't think there is anything special about BigDecimal that will prevent it from being gc'ed.

Message was edited by: stevegal

jfanatic
Offline
Joined: 2008-08-20

To stevegal & dansiviter:

Its true that in the test code all the declarations are outside the loops but these are just reference declarations and assuming my java knowledge is correct assigning a reference previously pointing to object o1 to a new object o2 makes o1 garbage collectable.

As for holding 1million objects at the same time, well MutableDoublez are performing with the same heap size. Problem is BigDecimal requires 2million to do the same job :).

Yes we can move to tuning jvm, increase heap size etc. Unfortunately that is all left to us today just because we wanted to stick to "standard options" available in java to do simple additions with reasonable performance and accuracy. Inventing homegrown solutions is not a viable option sometimes and most of us here are app developers not system developers.

Thanks

kcpeppe
Offline
Joined: 2003-06-15

This is beginning to sound like a microbenchmark and what stevegal and dansiviter are hinting at is; microbenchmarking is notoriously difficult. If you are not careful, you will be measuring some pathology that has nothing to do with your initial question. It would be useful if you could post your code so that we can have a look at it.

Regards,
Kirk

jfanatic
Offline
Joined: 2008-08-20

Hi kirk

I think we are beginning to get astray from the real topic, performance test was only done because some of the posters here objected that you are saying something about BD w/o actually giving proof of its performance which all of us should know will be very slow no matter what. The question is not about how many objects we are allocating at 1 time to avoid OutOfMemory exception 1Milllion,10Million or even mere 100, BD will remain slow. The question is not about why i am not tuning my JVM to allocate more memory etc, i know this is a way to go even when i feel that memory i allocate for my app should be used to hold app specific data and not some garbage.My real question to which no one is giving any answers is why BD in java is immutable and should we have a mutable version as a standard option because BD is the only option in standard java to get accurate floating point results. I know that SUN has planned to provide some usability benefits for BD as part of SE 7.0 like operator overloading how wonderful it would be if some attention should be paid towards its performance also.

Thanks

stevegal
Offline
Joined: 2004-07-10

Okay, okay, I'll bite :D

are you asking why immutable objects are a good thing? Have you thought about what might happen in a highly multithreaded app if you are passing around objects that mutate from underneath you? At least with an immutable object you can be sure that when you for example

if (myBigDecimal.compareTo(someOtherBigDecimal)>1)
{
final BigDecimal newValue = this.doSomething(myBigDecimal,someOtherBigDecimal);
}

then the values in the variable myBigDecimal and someOtherBigDecimal have not changed by the time you get to the this.doSomething method. Without immutability how can you be sure. Especially important in a finiancial app I would have though!!!

but then again I do like the keyword final :D

hontvari
Offline
Joined: 2005-03-31

Do you have actual performance problems? Have you done performance test with an alternative implementation?

The results of performance tests are frequently surprising. Even when JRE doesn't optimize code. If BigDecimal were mutable it still has to allocate an array for the result. Not always, but the overhead of filtering these cases and the additional complexity of calculating in place may give worser performance. Without measurement you cannot tell whether a mutable version would be faster.

chrishurst
Offline
Joined: 2008-02-26

Do you have any benchmarks to back this assertion up ?

My understanding of Java garbage collection is that its very efficient at object reuse unlike C++ new / delete and should be very fast at such reusue ie they don't just throw your object away because they can.

'How expensive is allocation?'
http://www.ibm.com/developerworks/java/library/j-jtp01274.html

'Urban performance legend #3: Immutable objects are bad for performance'
http://www.ibm.com/developerworks/java/library/j-jtp04223.html

Also in theory the JVM does not have to allocate a new object on the heap at all and could benefit from escape analysis and allocate on the stack.

http://www.ibm.com/developerworks/java/library/j-jtp09275.html

kcpeppe
Offline
Joined: 2003-06-15

> Do you have any benchmarks to back this assertion up
> ?
>
> My understanding of Java garbage collection is that
> its very efficient at object reuse unlike C++ new /
> delete and should be very fast at such reusue ie they
> don't just throw your object away because they can.

This is incorrect. GC will reclaim what it can when it can. Also object reuse should be avoided. References should be released as soon as possible. This allows collection in young. The young collector work by copying live objects out of Eden/Survivors. One could almost argue that this isn't garbage collection, it's live object harvesting. From this point of view you should see that short lived objects will not be harvested == very little work to reclaim the space. Longer lived objects will needed to be harvested (read copied) and that is an expensive operation which involves work to swizzle pointers and so on.
>
>
> 'Urban performance legend #3: Immutable objects are
> bad for performance'
> http://www.ibm.com/developerworks/java/library/j-jtp04
> 223.html

Brian has been pushing immutability because it works better in multi-threaded applications. There is no question that immutability is a drain on performance. However, I would not thread safety for performance. The advice offered by Brian is mostly good. You're going to have to evaluate it in your particular context to see what's best.
>
> Also in theory the JVM does not have to allocate a
> new object on the heap at all and could benefit from
> escape analysis and allocate on the stack.

This isn't working just yet.

Regards,
Kirk

mthornton
Offline
Joined: 2003-06-10

I think this depends on the operation mix and the size of the values involved. For multiplication with large values the limitation is the use of the classical algorithm and not gc. Adding up a series of values might benefit from a mutable version, but perhaps a better solution would be to provide a method which adds up an array of (Number) values. This could then use a mutable value internally (which already exists) and return an immutable result.

Often though you won't save much because Java's allocation of short lived memory is rather fast.

dansiviter
Offline
Joined: 2005-09-26

How bizarre you should point this out as I've been discussing this with a colleague today.

Unfortunately, the story is even less black and white as you've pointed out. When using the BigDecimal it's not treated as a normal class by the JVM internals. After a quick Google it turns out JVM vendors have apparently put some magical code that streamlines the usage of the class. Don't bother asking me how... but it works.

By creating some basic tests an internally written mutable decimal class [with zero internal class creation] was outperformed by BigDecimal. BigDecimal even got better when you add the '-server' flag (using JDK 1.5.0_14).

I've not had the heart to tell the guy who wrote it yet. :oP

Caveat: I've only tried this with the normal JVM and not the real-time one so cannot vouch for its behaviour in that environment, but I'd imagine it's the same.

jdouglas
Offline
Joined: 2004-03-12

[i]> After a quick Google it turns out JVM vendors have
> apparently put some magical code that streamlines the
> usage of the class. Don't bother asking me how... but
> it works.[/i]

I'd be interested in those URLs.

ryanh0
Offline
Joined: 2008-07-14

I know jakarta commons lang has some Mutable numbers, but unfortunately no BigDecimal impl.
http://commons.apache.org/lang/apidocs/index.html
Maybe they have one planned or a reason for not having one...