Skip to main content

Increased GC frequency with CMS

5 replies [Last post]
shulei
Offline
Joined: 2009-04-28
Points: 0

I have a server that processes a large number of requests (about 4000 requests/sec). It creates a lot of short lived objects in a short period of time. I used CMS garbage collector with the following parameters:

-server -Xms4g -Xmx4g -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70

I am running JDK6 on Linux:
java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) 64-Bit Server VM (build 11.0-b15, mixed mode)

Linux usc101 2.6.24-etchnhalf.1-amd64 #1 SMP Tue Oct 14 03:11:45 UTC 2008 x86_64 GNU/Linux

The GC throughput is 99.57% and GC collects 8.5G/sec. Over time, both the ParNew and the CMS GC frequencies rise while my test load is steady. In a period of 15 hours, minor GC (ParNew) increases from once every 40 seconds to every 15 seconds; full GC (CMS) increases from once every hour to once every 15 minutes. After full/CMS GCs, the memory always resets back to the same level. As the GC frequency increases, I noticed that my CPU usage also increases from 5% to 20% and my response time increases.

Has anyone seen this type of behavior with CMS? What could cause the GC frequencies to increase?

Message was edited by: shulei

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
shulei
Offline
Joined: 2009-04-28
Points: 0

Problem was solved. I used jstat to watch the memory allocation and usage for different generations and found that my survivor generation is 100% full all the time. After I tune the survivor generation size, the GC has been regular.

ruchirc
Offline
Joined: 2008-07-20
Points: 0

My Friend You can try this option:

java -Xmx4096m -Xms4096m -Xmn2048m -Xss128k -XX:ParallelGCThreads=20 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:+AggressiveOpts

If you can publish the capacity of the box It will be great to suggest you something better.

Ruchir Choudhry

tomcatuser
Offline
Joined: 2011-01-14
Points: 0

Java 1.6 - -XX:CMSInitiatingOccupancyFraction=70 -
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 429522944 (409.625MB)
used = 429522944 (409.625MB)
free = 0 (0.0MB)
100.0% used
Eden Space:
capacity = 322174976 (307.25MB)
used = 322174976 (307.25MB)
free = 0 (0.0MB)
100.0% used
From Space:
capacity = 107347968 (102.375MB)
used = 107347968 (102.375MB)
free = 0 (0.0MB)
100.0% used
To Space:
capacity = 107347968 (102.375MB)
used = 0 (0.0MB)
free = 107347968 (102.375MB)
0.0% used
concurrent mark-sweep generation:
capacity = 3221225472 (3072.0MB)
used = 3221053616 (3071.8361053466797MB)
free = 171856 (0.1638946533203125MB)
99.99466488758723% used
Perm Generation:
capacity = 565047296 (538.87109375MB)
used = 318075232 (303.3401794433594MB)
free = 246972064 (235.53091430664062MB)
56.29178906821988% used

richiemarshall
Offline
Joined: 2009-05-07
Points: 0

-------------
> I would also suggest trying a lower CMSInitiatingOccupancyFraction to see if this reduces the number of full GC
-------------

Wouldn't this increase the number of full GCs as it will tell the JVM to start a full GC when occupancy is lower?

If you want to decrease the number of Full GCs then the fraction should be higher than 70%. (as is specified in the startup params). It may be worth removing the CMSInitiatingOccupancyFraction option entirely and let the JVM decide when to kick of a Full GC. It's default is somewhere around the 90% occupancy mark (as described here: http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html#gene...)

peter__lawrey
Offline
Joined: 2005-11-01
Points: 0

If it is cleaning up to the same level, more often, then it is cleaning up more data. I would guess, the behaviour of your application changes over time, even though the test load hasn't changed. (It will only clean up objects the application creates)

I would also suggest trying a lower CMSInitiatingOccupancyFraction to see if this reduces the number of full GC