Skip to main content

Memory areas contributing to the real memory footprint

8 replies [Last post]
st10470
Offline
Joined: 2008-05-23
Points: 0

Hello,

I wonder what is the real weight of the different memory areas in the real memory footprint (retrieved by TOP egs.).
For instance, the JMX API gives access to the "CMS Perm Gen" area, with "init", "used", "committed", "max" attributes.
Which of these attributes is the one contributing to the real footprint ?
What is the effect of setting the max, and what happens if the max is reached by setting a too low value ? What is the value of the additionnal allocated size ? Is it tunable ?
I set the JVM memory options such as...
-XX:MaxPermSize=size
-XX:ReservedCodeCacheSize=size
etc.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
peter__lawrey
Offline
Joined: 2005-11-01
Points: 0

It worth noting that unless you are talking about GB's you are not talking about memory which costs much. You may be able to tune down the memory only to run into problems much later. Note: 100 MB costs about $3, so how much time is it worth spending to save this much memory?

mthornton
Offline
Joined: 2003-06-10
Points: 0

Even GB does not cost much. The trouble comes if you aren't allowed to require a 64 bit OS. In this case you have a hard limit at 2 or 3GB (usually 2GB in the case of Windows).

st10470
Offline
Joined: 2008-05-23
Points: 0

Thanks for the advice unfortunately the context is fixed: no hardware tuning allowed! I cannot describe it all but you just have to know that it is an embedded system and that the hardware is chosen and certified for safety reasons, years before software is designed (which runs against the market evolution...) so that we cannot just go to the next PC corner shop and change a RAM slot during the week end ;)
Interesting though how it reveals opposite approaches on tuning.
Some questions remain:
1- Is there a way to know/change the value of the increment in allocation of memory pools such as Perm Gen ?
2- Is bytecode included in perm gen ?
3- if yes, do you think class unloading is a good approach to tune down memory ?

briand
Offline
Joined: 2005-07-11
Points: 0

> 1- Is there a way to know/change the value of the increment in allocation of memory
pools such as Perm Gen ?

What do you mean by 'Increment in allocation'. There's two interpretations in my mind.

1 - object allocation sizes. From the application's perspective, this is more under you
control than the JVM's control, though you have little control over object sizes for any class
libraries that you might use. You may have some control over buffer and array sizes, though
use of their corresponding allocation APIs and tunables. Still, these types of things don't live
in the perm gen. Interned String sizes and application class size are about all you have
control over for the things that live in the Perm Gen.

2 - the size by which the JVM grows the heap space, where grow means moving pages
from strictly a reserved address space state into a committed address space state. How
this works depends on the garbage collector you are using. See the following paper for
details.

http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html

For the old gen and perm gen, and to some extent the young gen, there are various
command line options that control the grow/shrink behavior. The -Xms parameter
indicates the initial committed memory size for the old and young gens. The -Xmn
size indicates the initial reserved memory for the old and young gens. When the
heap utilization exceeds -XX:MinHeapFreeRatio after a full collection, the heap
will be grown. Likewise, if after a collection, the heap utilization drops below
-XX:MaxHeapFreeRatio, the heap will be shrunk. Growing and shrinking will occur
in units of -XX:MinHeapDeltaBytes, however I believe that the ParallelGC young
gen collector has it's own policy for grow/shrink of the young gen.

The Perm Gen is not included in the -Xms/-Xmx parameters and needs to be
sized separately. There are similar grow/shrink parameters for the perm gen:

-XX:MinHeapDeltaBytes - grow/shrink amount - applies to both the perm and old gens
-Xms :: -XX:PermSize
-Xmx ::-XX:MaxPermSize
-XX:MinHeapFreeRatio :: -XX:MinPermHeapExpansion
-XX:MaxHeapFreeRatio :: -XX:MaxPermHeapExpansion

To be frank, I have no experience with [Min|Max]PermHeapExpansion, so I'm
not sure that they are 1:1 with [Min|Max]HeapFreeRatio. You'd have to play with
them to figure out how they actually work. The [Min|Max]PermHeapExpansion
are not widely used options (in fact, I don't recall ever seeing them be used),
so caveat emptor.

Note: Grow (aka expand) means that memory moves from reserved to committed,
while shrink means that it moves from committed to reserved. Committed memory
requires backing store, and therefore swap space and generally RAM. These
commit/uncommit memory operations are not necessarily free from a performance
perspective. How they behave depends on the work performed by the underlying
OS. For example, attempting to commit a page of memory could result in the OS
paging out the memory of some other process or even result in a page coalescing
operation, both of which could be expensive. For pause time sensitive apps, we
generally recommend that you set these parameters such that committed == reserved
to avoid the costs associated with these operations.

Memory size oriented parameters are generally automatically and silently rounded
to the page size being used, so if you use a size that is not a multiple of the page
size in use, then it may appear that the systems is using a size different from what
you specified.

> 2- Is bytecode included in perm gen ?

No, it's in the class files themselves, which are loaded into memory via native I/O operations.

> 3- if yes, do you think class unloading is a good approach to tune down memory ?

Class unloading is orthogonal to where the byte codes are stored. There's classes
in the Perm gen that describe the classes that are loaded, and these classes can
be unloaded on full collections when there are no longer any objects in the heap
of that type. This tends to take pressure off of the Perm Gen and will reduce the
number of full gc events that occur in a system (compared to an implementation
that doesn't unload unused classes). CMS, however, can collect the perm gen
concurrently, meaning a full gc is not required to collect class objects from the
perm gen. We see a lot of people running with -Xnoclassgc on their command
lines; this tends to be a hold over from the pre-JDK 1.8 days and is generally a
really bad idea. Not only does it result in additional pressure on the Perm gen,
but it is also known to result in some odd (read: bad) performance pathologies.

st10470
Offline
Joined: 2008-05-23
Points: 0

Thanks a lot for the time spent and this complete information. I'll try to give feedback on our trials!
1-
We use the following parameters:
java -server -XX:NewSize=32m -Xms75m -Xmx300m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC [...]
which means we set the initial heap to a size slightly above the needs in a nominal use case.
We do have a low GC pause constraint to guarantee our network acquisition throughput, this is why we use CMS GC and a big NewSize.
To set committed==reserved, we would set Xms=Xmx, did I get that right ?
But then we'll have a huge heap (300m) all the time, whereas it would be really useful one out of 100 times.
(I guess you meant Xmx instead of Xmn here: "The -Xmn
size indicates the initial reserved memory for the old and young gens".)
In fact the problem we regularly face is that after hours of running, the used heap reaches the initial old available size ((75m-32m)=43m), and full garbage is done frequently (period=11s), thus increasing the CPU consumption. We will try to tune the GC as described in the sun link you gave (-XX:CMSIncrementalSafetyFactor=, -XX:CMSIncrementalDutyCycleMin=, ...). Could you give a typical value to set for these parameters, since I have no idea of the default one ?
Besides, using "-Xms100m", the full GC happens less often, and the CPU decreases. The strange thing is that the RAM increases by 15MB, while the heap increases by 25MB.

2- Ok, I asked because Perm gen contains classes and information about the classes, which is a little confusing...

3- If I sum it up, class unloading is a good way to reduce Perm gen space (plus loaded byte code), so we will give it a try then. I was a little surprised that people advice to run several System.gc() to ensure class unloading, until there's no more memory to free. In our case we'd better let the GC work maybe a little later but ensuring our acquisition throughput.

briand
Offline
Joined: 2005-07-11
Points: 0

> 1-
> We use the following parameters:
> java -server -XX:NewSize=32m -Xms75m -Xmx300m
> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC [...]

The "..." might be important here. What version of the JDK are
you using? What OS platform are you running on? How many
CPUs do you have?

> which means we set the initial heap to a size
> slightly above the needs in a nominal use case.
> We do have a low GC pause constraint to guarantee our
> network acquisition throughput, this is why we use
> CMS GC and a big NewSize.
> To set committed==reserved, we would set Xms=Xmx, did
> I get that right ?

Yes.

> But then we'll have a huge heap (300m) all the time,
> whereas it would be really useful one out of 100
> times.

It's important to realize that CMS never shrinks the heap. This is
a design decision. So, if you hit that 1 out of 100 case just once,
you are at 300m forever after that. So, you might as well design
for the worst case from the get go, IMHO. You'll get more consistent
performance throughout the lifetime of the application.

> (I guess you meant Xmx instead of Xmn here: "The
> -Xmn
> size indicates the initial reserved memory for the
> old and young gens".)

Yep - sorry about that. sometimes I type to fast for my own good ;-)

> In fact the problem we regularly face is that after
> hours of running, the used heap reaches the initial
> old available size ((75m-32m)=43m), and full garbage
> is done frequently (period=11s), thus increasing the
> CPU consumption. We will try to tune the GC as
> described in the sun link you gave
> (-XX:CMSIncrementalSafetyFactor=,
> -XX:CMSIncrementalDutyCycleMin=, ...). Could you

These parameters are for CMS incremental mode (iCMS), and only apply
if you have -XX:+CMSIncrementalMode set on your command line.
I didn't see that above. If you are not using iCMS, then these parameters
are ignored.

> give a typical value to set for these parameters,
> since I have no idea of the default one ?

Default Duty Cycle is 10, which is a percentage.
Default Safety Factor is also 10, and is also a percentage.

There's a lot of complexity around these, so I'd like to know
if you are using iCMS before getting into this.

> Besides, using "-Xms100m", the full GC happens less
> often, and the CPU decreases. The strange thing is

> that the RAM increases by 15MB, while the heap
> increases by 25MB.

How are you measuring "RAM" - is this RSS? We would reserve
additional address space during a grow operation, which would
increase the virtual memory size of the heap, but most operating
systems don't make those pages resident until we first touch the
page. So, if we grew the heap by 25MB, but we only utilized (touched)
15 MB of that 25MB, the virtual size would have increased by 25MB,
but the RSS size would only increase by as much as we've touched,
in this case 15MB.

>
> 2- Ok, I asked because Perm gen contains classes and
> information about the classes, which is a little
> confusing...

It contains information about classes and interned strings. The
classes them selves, including byte codes, are in .class and .jar
files. In older JVMs (1.5.0 and earlier, IIRC) the jar/class files were
mapped into the process address space on Solaris and Linux
and a pmap of the process would see all these mappings. On
Windows, we've always used buffered I/O to read in class files,
and that's what we do in later JVMs on Solaris and Linux.

>
> - If I sum it up, class unloading is a good way to
> reduce Perm gen space (plus loaded byte code), so we
> will give it a try then. I was a little surprised
> that people advice to run several System.gc() to

This is usually a recommendation for testing and for benchmarks,
where you want to start in a clean state. I don't think this is generally
a good idea in production systems, unless you are performing it
during an initialization phase, before transitioning into steady state
operation (assuming your app behaves that way).

> ensure class unloading, until there's no more memory
> to free. In our case we'd better let the GC work
> maybe a little later but ensuring our acquisition
> throughput.

Generally, letting the GC do it's job is the best way to use GC.
Calls to System.gc() tend to cause problems. With CMS, there's
tuning variables to influence whether System.gc() results in a
stop the world full gc or whether it initiates a CMS cycle, so you'd
need to understand the consequences of each.

Some older versions of CMS (1.4, IIRC) didn't collect the Perm Gen.
Newer versions do. Even in recent JVMs, you might want to try
-XX:CMSPermGenSweepingEnabled.

3dfan
Offline
Joined: 2009-05-21
Points: 0

very interesting information, post more of such great messages for such newbies as I am

linuxhippy
Offline
Joined: 2004-01-07
Points: 0

used = actually used (information stored there) memory
commited = memory currently allocated ("footprint")
max = maximum allowed

If you set max too low, you program will fail to run, or with suboptimal performance.
However the JVM will good do a good job allocating the right amount of memory, so if you give it enough free room it won't eat up all your system resources ;)

lg Clemens