Skip to main content

Shrinking New Generation

10 replies [Last post]
samuelto
Offline
Joined: 2005-03-15
Points: 0

I'm testing JDK 1.5.0_09 on a Solaris 9 box (12-CPUs) with the following options:
-server -XX:+PrintClassHistogram -verbose:gc -XX:+PrintGCTimeStamps -x768M -Xms768M -XX:NewRatio=2 -XX:PermSize=64M -XX:MaxPermSize=160M -XX:+UseParallelGC -XX:ParallelGCThreads=2

I used LoadRunner to run a particular test script. I monitored the heap usage using jstat. Since I am specifying NewRatio=2, I expect the total size of the new generation to be about 250MB.

I summed S0C + S1C + EC from jstat (from what I understand, this should be the size of the new generation). The jstat readings are taken every 15s:
S0C S1C EC S0C+S1C+EC
2768.0 32768.0 196608.0 Total: 262144
32768.0 32768.0 196608.0 Total: 262144
32768.0 32768.0 196608.0 Total: 262144
32768.0 32768.0 196608.0 Total: 262144
...
10304.0 13760.0 118976.0 Total: 143040
13056.0 13760.0 117888.0 Total: 144704
8960.0 12544.0 116864.0 Total: 138368
...
10432.0 6336.0 91392.0 Total: 108160
10432.0 16384.0 87872.0 Total: 114688
26048.0 24512.0 84224.0 Total: 134784
...
65216.0 63232.0 111296.0 Total: 239744
72320.0 68736.0 111808.0 Total: 252864
75584.0 75648.0 110848.0 Total: 262080

So you see, the new generation started around 260MB, and shrunk to about 108MB before growing back to 262MB. The capacity of the old generation (OC) remains constant at 524MB during the test.

Running the exact same test with JDK 1.6, I noticed the same behavior, except the size of new generation shrunk to an even smaller value (57MB).

Why didn't the size of the new generation stay somewhat constant during the test? The shrinking new generation caused additional minor GC during the test.

I can understand the survivor space and eden to change size, but shouldn't the sum of their capacities to remain constant?

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
jon999
Offline
Joined: 2006-02-15
Points: 0

Would it be possible to run with -XX:+PrintHeapAtGC and include the gc
log here or submit the gc log to

http://java.sun.com/docs/forms/gc-sendusmail.html

samuelto
Offline
Joined: 2005-03-15
Points: 0

Done...there is no place for me to upload the entire GC log, so I just pasted in a few sections. If you want the entire log, let me know how to send it to you.

In response to briand, I added NewRatio in addition to -XX:NewSize because it won't let me just use -Xmx512m -Xms512m -Xmn=172m. It gives the following error:

Invalid initial eden size: -Xmn=172m
Could not create the Java virtual machine.

briand
Offline
Joined: 2005-07-11
Points: 0

I think you need to drop the equal sign - i.e. change -Xmn=172m to -Xmn172m.

samuelto
Offline
Joined: 2005-03-15
Points: 0

Using -Xmn172m works, and the new generation capacity stayed constant at 176128K:

NGCMN NGCMX NGC S0CMX S0C S1CMX S1C ECMX EC YGC FGC
176128.0 176128.0 176128.0 22016.0 58688.0 58688.0 22016.0 176000.0 132096.0 2 0
...
176128.0 176128.0 176128.0 6528.0 58688.0 58688.0 6720.0 176000.0 162560.0 75 11

jon999
Offline
Joined: 2006-02-15
Points: 0

The failure of the young generation to maintain a proper minimum value is a bug
and is being fixed under 6524727. As you note the workaround is to use
-Xmn (or the equivalent -XX:NewSize and -XX:MaxNewSize).

briand
Offline
Joined: 2005-07-11
Points: 0

It looks like there may be a bug here - where one of the survivor space's maximum
and current capacity jvmstat counters are potentially swapped. However, I'd have to
study the code more to determine if this is indeed what's happening.

In general the sum of the three spaces current capacities should be equal to the total
capacity of the young generation, assuming that -Xms == -Xmx. However, the counters
are sampled by jstat without any synchronization and at any time jstat could be sampling
a transient state. In practice, that doesn't happen very often, but it's not beyond the
realm of possibility. So, summing the current capacities like this could result in incorrect
results, regardless of whether there's a bug here.

A better way to check the size of the young generation is to look at the generation
size with jstat -gcnewcapacity. The following was generated with your same heap sizing
parameters:

[code]
# jstat -gcnewcapacity 22242
NGCMN NGCMX NGC S0CMX S0C S1CMX S1C ECMX EC YGC FGC
262144.0 262144.0 262144.0 32768.0 87360.0 87360.0 32768.0 262016.0 196608.0 2 2
[/code]

NGCMN is the minimum capacity of the young generation and NGCMX is the maximum
capacity of the young generation. NGC is the current capacity. They are all the same here
because -Xmx==-Xms and you can see that it is a 256m young generation.

For the second part of your question, why the current capacities of the spaces vary
over time, that's a factor of the Adaptive Size Policy (-XX:[+-]UseAdaptiveSizePolicy),
which is enabled by default with -XX:+UseParallelGC. This sizing policy allows the
garbage collector to vary the distribution of young gen space between the Eden
and Survivor spaces based on various allocation and collection statistics. This reduces
the need to size the survivor spaces for optimal garbage collection performance, as
they are dynamically sized based on application behavior .

When the adaptive size policy is in effect, the maximum capacities of the Eden and
Survivor Spaces are set to their theoretical maximums, which can be a bit misleading
when looking at the maximum space capacity values - the sum of the young gen space
maximum capacities will exceed the young generation maximum capacity.

If you don't want this dynamic space sizing, then you need to add -XX:-UseAdaptiveSizePolicy
to the command line. However, you'll likely need to size your survivor spaces to match
application behavior in this case. To size the survivor spaces in JDK 6 and later, just
use -XX:SurvivorRatio=N. Prior to JDK 6, you need to use InitialSurvivorRatio and/or
MaxSurvivorRatio instead of SurvivorRatio to statically size the survivor spaces for
UseParallelGC.

Message was edited by: briand

Message was edited by: briand

samuelto
Offline
Joined: 2005-03-15
Points: 0

Thanks for the reply. I tried using -gcnewcapacity and I'm seeing the same pattern. (This time I used -Xms512m -Xmx512m instead of 768m). The NGC value is actually pretty close to S0C + S1C + EC from jstat -gc.

Output from jstat -gcnewcapacity every 15s:

NGCMN NGCMX NGC S0CMX S0C S1CMX S1C ECMX EC
YGC FGC
176128.0 176128.0 167936.0 36160.0 58688.0 58688.0 3648.0 176000.0 92352.0 15 0
176128.0 176128.0 163840.0 32064.0 58688.0 58688.0 35328.0 176000.0 91456.0 16 0
... ...
176128.0 176128.0 102400.0 11648.0 58688.0 58688.0 12224.0 176000.0 75392.0 37 0
176128.0 176128.0 98304.0 7552.0 58688.0 58688.0 11200.0 176000.0 74752.0 38 0
... ...
176128.0 176128.0 57344.0 5312.0 58688.0 58688.0 5504.0 176000.0 43520.0 129 5
176128.0 176128.0 53248.0 4672.0 58688.0 58688.0 1728.0 176000.0 43136.0 131 5
... ...
176128.0 176128.0 77824.0 18496.0 58688.0 58688.0 15040.0 176000.0 37376.0 153 11
176128.0 176128.0 106496.0 32832.0 58688.0 58688.0 29056.0 176000.0 38336.0 159

So you see the new generation capacity shrunk to about 53MB before bouncing back. This pattern is reproducible for my particular test (which is a business application for creating purchase requisitions).

In my original question, I was asking why NGC changes at all. I can understand EC and S0C and S1C changing, but NGC should remain fairly constant based on the new ratio setting. Any ideas why that is not the case?

briand
Offline
Joined: 2005-07-11
Points: 0

Can you try -Xmn172m instead of -XX:NewRatio=2. I'm wondering if NewRatio
doesn't result in NewSize==MaxNewSize.

samuelto
Offline
Joined: 2005-03-15
Points: 0

I tried using -XX:NewSize=170m -XX:NewRatio=2 and the new generation capacity was constant:

NGCMN NGCMX NGC S0CMX S0C S1CMX S1C ECMX EC
YGC FGC
176128.0 176128.0 176128.0 22016.0 58688.0 58688.0 22016.0 176000.0 132096.0
1 0
...
176128.0 176128.0 176128.0 5440.0 58688.0 58688.0 5568.0 176000.0 164864.0
75 11

Does this suggest a problem with the NewRatio logic that without an explicit -XX:NewSize, old generation capacity / new generation capacity is not always equal to the NewRatio value?

briand
Offline
Joined: 2005-07-11
Points: 0

In this case, your NewRatio setting was overriding NewSize due to the order
on the command line. NewRatio is just a more convenient way of setting NewSize
based on the total heap size. It may have resulted in the same net effect, but it
was not quite what I was after. I was looking for you to drop NewRatio and replace
it with just -Xmn. -Xmn is just a more convenient way of setting both NewSize and
MaxNewSize to the same value.

Regardless, the effect here may or may not be intentional. When -Xms != -Xmn,
the garbage collector is allowed to grow and shrink the heap size as it sees fit.
This happens only on full garbage collections. When NewSize != MaxNewSize,
which is the default condition, the same is true of the new generation (and it too
only gets resized on full collections - ignoring the UseAdaptiveSizePolicy component,
which just redistributes the current size among the three spaces). The question here
is whether NewSize != MaxNewSize is intentional when -Xms == -Xmx. It seems
to me that this may be a bug, but it may have been intentional. I'll need to take
it up with the GC team to be sure.