Skip to main content

Help On Memory Leak.

4 replies [Last post]
saiprasadt
Offline
Joined: 2006-12-02
Points: 0

Hi,

Can any one answer my question.

As part of my application performance testing one of the container in my application using more memory.(Java Based Application).

This application is on solaris platform. When the application started this container normally uses 400 MB. When the load increases on the application this container starts eating memory. This will go up to 2345 MB. We have specified vmparam -Xmx2048m for this container.But still it crossed 2048 MB. My total system memory itself 8 GB.This container alone is eating 2.4 GB.

After this I have kept my application idle for more than 24 hours.This used memory never come down.It was like that.

1.Once GC runs it will free up this memory?
2.After GC freeing up memory unused memory by java will be given to OS?(In Solaris).
3.Memory can be occupied more than what we have specified in Xmx?.
4.Is this memory Leak?

Can any one answer.

Regards
Sai.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
briand
Offline
Joined: 2005-07-11
Points: 0

> 1.Once GC runs it will free up this memory?

It depends on what you mean by 'free up this memory'. If you mean garbage collect it, then yes it will free it up
any unreferenced heap memory when the right type of GC event occurs. For some objects, like those with finalizers, it may take a couple of the right type of GC events to fully free up that memory.

> 2.After GC freeing up memory unused memory by java will be given to OS?(In Solaris).

If you mean will the memory be returned to the OS and the virtual size of the process (and subsequently the RSS size of the process as well) be reduced to some smaller value, then maybe ....

For the JVM to actually reduce its memory usage, a number of conditions need to occur. First, must have have -Xms != -Xmx. If -Xms == -Xmx, then we allocate and commit the full size of the heap and never grow or shrink the heap (which is typically the best thing for server apps)

When -Xms != -Xmx, we will grow and shrink the heap based on the settings of -XX:MinHeapFreeRatio= (default 40%) and -XX:MaxHeapFreeRatio= (default 70%). See:

http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html#0.0.0.%20Total%2...

However, these grow and shrink operations will only happen after a Full GC event. That's the tricky part and if your application had a spurt of work to do, grew the heap to it's full size and then went completely idle, you won't get the Full GC events to trigger the release of the memory resources.

If you are running with the CMS garbage collector, it only grows the heap; it does not currently shrink it.

> 3.Memory can be occupied more than what we have
> specified in Xmx?.

Again, it depends on what you mean by 'occupied'. The Java heap itself will not grow beyond the -Xmx value.

There are other memory resources used by the JVM. There's the Perm Gen (-XX:PermSize -XX:MaxPermSize), which store class meta data and interned strings, and the Code Cache, which stores JITed methods. In addition, your application can directly or indirectly use native memory resources (via use of native libraries or the use of Direct or Mapped ByteBuffer objects). The JVM also has it's own text and data.

One particularly large contributor to the Solaris (and Linux) virtual memory size of a Java process is the fact that we mmap every jar file into the process address space. For apps with lots of jar files, this can really inflate the VSIZE of the process. Interestingly, these mappings are read only and any common jar files between running Java apps (like rt.jar, for instance) are completely shared between instances. In JDK 6 and later, we don't do this anymore - instead, we only map in the 'central index' of the jar files. So, you have to be careful on how you interpret memory metrics. RSS is typically a better estimate of real memory usage than VSIZE (though it is still an estimate as it doesn't separate out actually shared, sharable, and private RSS).

> 4.Is this memory Leak?

Maybe, maybe not. 2345M is not really much beyond 2048M and things like the Perm Gen and Code Cache may contribute significantly to that difference (depending on PermGen sizing). jconsole will help you see the size of the PermGen and the code cache. You may have grown to the maximum heap size because of a memory leak; we can't tell you if that's the case, though. You could use jmap -histo on the process when it's in this state to get a heap dump that may help in identifying if there's a memory leak. Alternatively, you could use hprof with heap=sites to get similar information.

If it's a native component thats growing the address space, you'll need to use native tools to resolve it. Since you are on Solaris, you are probably in the best place to resolve this issue as you have tools like pmap (pmap -xs and pmap -r) and dtrace at your disposal to help.

>
> Can any one answer.
>

HTH

Brian

thomasolsen
Offline
Joined: 2007-07-18
Points: 0

Hi,

I have a simliar question regarding the memory manangment and what Linux displays. I read that using ps to display what memory is consumed by your process might not give you the correct result.

When I look at ps for my java process I get as mentioned that same results where RSS used by the process slowly increases and it appears never decrease. I tryed to use pmap and when the RSS increases every shared library etc within the address space reported by pmap stays the same. Does this then indicate that it isn't a memory leak?

I do actually have -Xmx == -Xms so there was a hint in the previous post to why the RSS and Virtual Size never decreases.

I have used a profiler to look into my java program and everything seems ok. It doesn't appear to have any leaks and I have never got an Out Off Memory exception.

So my question is really, whether or not I should be concerned with the fact that RSS increses?

Any help would be greatly appriciated.

Thank
Thomas

You mentioned in section 4 about the memory leak that

briand
Offline
Joined: 2005-07-11
Points: 0

RSS can increase only up to the virtual size of the processs. If you have nothing
else running on the system and there's plenty of RAM to satisfy all the virtual memory
committed by the process, then an increasing RSS is not a bad thing. It just means
that more and more of your virtual address space is occupying physical RAM. That
in turn means fewer page faults on non-resident pages. (It's interesting to note that
Windows places restrictions on the total amount of memory that can be resident for
a process - and provides APIs that allow that limit to be changed - but Solaris and
Linux do not have such per-process limits or APIs; instead they take a more global
view of memory utilization and rely on LRU policies to determine what should be
evicted from RAM).

If the increase in RSS is accompanied by continued increases in the virtual memory
size of the process, then that might be an indication of a memory leak. In your situation,
where -Xms==-Xmx, is not likely to be in the Java heap, though. However, it could be
the result of things that your java application is doing directly (such as FileChannel.map()
operations or use of Buffer.allocateDirect()) or indirectly, though some class library or
framework that it uses. It could also be the result of native methods, again either directly
or indirectly, that induce memory leaks through the use of malloc().

The RSS size of a process can increase signficantly during a full garbage collection,
as the garbage collector pretty much touches every page of the heap, and for large
heaps and/or apps with infrequent full gc events, this can increase the RSS of the
process.

I'm not booted into my linux partition at the moment, so I don't recall the specifics
of Linux's pmap command, but I do recall that it is not as robust as the Solaris pmap
command. I think that it only shows you virtual memory reservations mappings and
doesn't provide a way to distinguish between committed and resident portions of
those mappings. Without being able to distinguish between these two, it's difficult to
identify which mappings are contributing to additional committed and RSS memory.

linuxhippy
Offline
Joined: 2004-01-07
Points: 0

Cross-Posting is clearly a do-net, except you've good reaons and you explain why you did so.

In fact if you read at: http://www.javalobby.org/java/forums/t85324.html

You'll get the answers to you question. If they don't help you next time ask more specifically. (If you never get OOM its most likely not a leak but a performance optimization so that the collector does not have to collect too often).

lg Clemens