Skip to main content

Glassfish 3.0.1 GZIP problem - Threads apparently spinning, 100% CPU use

Please note these java.net forums are being decommissioned and use the new and improved forums at https://community.oracle.com/community/java.
7 replies [Last post]
steveglennon
Offline
Joined: 2008-04-24

We are in a situation where we find our Glassfish 3.0.1 instance permanently running at 100% CPU, with heap memory use normal. A thread dump (successive times) shows that two of our Glassfish threads are spinning indefinitely in the GZIP compression phase. There appear to be others who are seeing similar behaviors (but with no answers) and it is starting to look like a Glassfish GZIP bug. We are simply using the @GZIP annotation on our web service, and have enabled gzip,deflate in the network/http listener in the glassfish admin console.

Normally all is well (we see gzipped payloads), so this is NOT the issue mentioned here:

Others are seeing similar (maybe exactly the same) problems:
and there are several "clues" in various other stack dumps in a google search on (for example "Deflater.java:306") that suggest that either this is a "normal" situation (like socket reads being marked as runnable - I think not in this case), or that there is some systematic problem with GZIP support in the JVM. I have found no answers that seem to address this specific problem.

We have two threads that are in basically the same situation, with different locks but otherwise stuck at the same place:

"http-thread-pool-8080-(18)" - Thread t@950

java.lang.Thread.State: RUNNABLE
at java.util.zip.Deflater.deflateBytes(Native Method)
at java.util.zip.Deflater.deflate(Deflater.java:306)
- locked <21b0c5> (a java.util.zip.ZStreamRef)
at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)
- locked <132ba84> (a java.util.zip.GZIPOutputStream)
at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
at org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
at org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)
at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)
at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
- locked <1ab9b7e> (a org.apache.cxf.phase.PhaseInterceptorChain)
at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)
- locked <16129e9> (a org.apache.cxf.phase.PhaseInterceptorChain)
at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
- locked <1649a52> (a org.apache.cxf.phase.PhaseInterceptorChain)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)
at org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
at org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
at com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:662)
Locked ownable synchronizers:
- locked <1cfb6a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

I would love to have someone smarter and more experienced than me to shed some light on this.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
oleksiys
Offline
Joined: 2006-01-25

Hi,

the problem seems to happen inside CXF, so it may be not (I'd even say
just "not" :) ) GF related.
One of the links you referenced have a stacktrace, which suggests that
spinning may occur when more than 1 thread tries to deal with the same
GZip stream... may be CXF has the same problem?

What I can suggest you to try is to disable gzip on the CXF level
(remove @GZIP annotation) but enable gzip on GF level via admin console.

Thanks.

WBR,
Alexey.

On 10/13/2011 11:48 PM, forums@java.net wrote:
> We are in a situation where we find our Glassfish 3.0.1 instance
> permanently
> running at 100% CPU, with heap memory use normal. A thread dump
> (successive
> times) shows that two of our Glassfish threads are spinning
> indefinitely in
> the GZIP compression phase. There appear to be others who are seeing
> similar behaviors (but with no answers) and it is starting to look like a
> Glassfish GZIP bug. We are simply using the @GZIP annotation on our web
> service, and have enabled gzip,deflate in the network/http listener in
> the
> glassfish admin console.
>
> Normally all is well (we see gzipped payloads), so this is NOT the issue
> mentioned here:
>
> http://www.java.net/forum/topic/glassfish/glassfish/gzip-compression-gla...
>
> [1] Others are seeing similar (maybe exactly the same) problems:
> https://forums.oracle.com/forums/thread.jspa?messageID=4627097 [2] and
> there
> are several "clues" in various other stack dumps in a google search on
> (for
> example "Deflater.java:306") that suggest that either this is a
> "normal" situation (like socket reads being marked as runnable - I think
> not in this case), or that there is some systematic problem with GZIP
> support
> in the JVM. I have found no answers that seem to address this specific
> problem.
> We have two threads that are in basically the same situation, with
> different
> locks but otherwise stuck at the same place:
>
> "http-thread-pool-8080-(18)" - Thread t@950
>
> java.lang.Thread.State: RUNNABLE at
> java.util.zip.Deflater.deflateBytes(Native Method) at
> java.util.zip.Deflater.deflate(Deflater.java:306) - locked <21b0c5> (a
> java.util.zip.ZStreamRef) at
> java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
> at
> java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118) at
>
> java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72) - locked
> <132ba84> (a java.util.zip.GZIPOutputStream) at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)
>
> at
> org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) at
>
> org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)
>
> at
> org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
> - locked <1ab9b7e> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)
>
> - locked <16129e9> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
> - locked <1649a52> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)
>
> at
> org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)
>
> at
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)
>
> at
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
>
> at
> org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)
>
> at
> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
> at
> com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97) at
> com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
>
> at
> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
>
> at
> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
> at
> com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693) at
> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954) at
> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
>
> at
> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
> at
> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
> at
> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>
> at
> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>
> at com.sun.grizzly.ContextTask.run(ContextTask.java:69) at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
> at java.lang.Thread.run(Thread.java:662) Locked ownable synchronizers: -
> locked <1cfb6a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> I would love to have someone smarter and more experienced than me to shed
> some light on this.
>
>
> [1]
> http://www.java.net/forum/topic/glassfish/glassfish/gzip-compression-gla...
> [2] https://forums.oracle.com/forums/thread.jspa?messageID=4627097
>
> --
>
> [Message sent by forum member 'steveglennon']
>
> View Post: http://forums.java.net/node/853023
>
>

steveglennon
Offline
Joined: 2008-04-24

Alexey,
Thanks for the feedback. We took a similar tack. We had set up the "Enable Compression" ON on Glassfish (Network Listeners - Advanced) in addition to using the CXF annotations. We took a similar path to you - that mybe somehow the two are conflicting. I had thought that the GF compression was to be used in combination with the CXF compression (ie GF compression ON is required to allow the CXF annotations to work). This appears to not be the case.

We tried one server with the GF compression off, and an otherwise identical server running with GF compression on. We found the GF Compression OFF server still running after the weekend, and the GF compression ON server back in the same state (100% CPU, 2 threads stuck in GZIP compression)

So at this point I'm willing to just turn off GF compression and see how long this lasts, but I don't feel like I have an answer as to where the problem lies.

Regards

Steve

oleksiys
Offline
Joined: 2006-01-25

Hi Steve,

> We tried one server with the GF compression off, and an otherwise
> identical
> server running with GF compression on. We found the GF Compression OFF
> server still running after the weekend, and the GF compression ON server
> back in the same state (100% CPU, 2 threads stuck in GZIP compression)
I see. Can you pls. post a complete threads dump, when spinning appears?
As I said, looks like several (more than 1) threads try to write data
simultaneously on the same connection.

Are you running any asynchronous tasks within the web service?

WBR,
Alexey.

steveglennon
Offline
Joined: 2008-04-24

Alexey,

Last question first - We are not creating separate threads within the web service (I understand that the conventional wisdom on that is "BAD IDEA"). We do have an HTTP thread pool configured within Glassfish for multiple requests.

The complete thread dump is as follows.

It is starting to look like either:

* The two http pool threads are stuck inside the gzip native code or

* The two http pool threads are bouncing in an out of the gzip native code.

We found some wierdness (bug?) in java.util.zip.DeflaterOutputStream where it appears that on write(buf,off,len) the result appears to be that deflate is called which works on the protected member buf[] rather than the passed in buffer b[].

Full thread dump:

2011-10-13 10:28:55

Full thread dump Java HotSpot(TM) Client VM (19.0-b09 mixed mode):

"RMI TCP Connection(idle)" - Thread t@1822

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1d29b42> (a java.util.concurrent.SynchronousQueue$TransferStack)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)

at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)

at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-79" - Thread t@1821

java.lang.Thread.State: BLOCKED

at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)

- waiting to lock <31a517> (a org.jvnet.hk2.osgiadapter.OSGiModuleImpl) owned by "Thread-73" t@1794

at com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)

Locked ownable synchronizers:

- None

"RMI TCP Connection(25112)-10.4.0.88" - Thread t@1820

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)

at java.io.BufferedInputStream.read(BufferedInputStream.java:237)

- locked <1a966ca> (a java.io.BufferedInputStream)

at java.io.FilterInputStream.read(FilterInputStream.java:66)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <9b9212> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"RMI TCP Connection(25115)-10.49.5.34" - Thread t@1819

java.lang.Thread.State: RUNNABLE

at sun.management.ThreadImpl.dumpThreads0(Native Method)

at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:374)

at sun.reflect.GeneratedMethodAccessor2900.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)

at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)

at com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)

at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)

at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)

at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)

at javax.management.StandardMBean.invoke(StandardMBean.java:391)

at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)

at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)

at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)

at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)

at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)

at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)

at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)

at sun.reflect.GeneratedMethodAccessor1010.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)

at sun.rmi.transport.Transport$1.run(Transport.java:159)

at java.security.AccessController.doPrivileged(Native Method)

at sun.rmi.transport.Transport.serviceCall(Transport.java:155)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <192f091> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"JMX server connection timeout 1818" - Thread t@1818

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <11e7538> (a [I)

at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"RMI TCP Connection(25116)-10.49.5.34" - Thread t@1815

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)

at java.io.BufferedInputStream.read(BufferedInputStream.java:237)

- locked <1b8bcfc> (a java.io.BufferedInputStream)

at java.io.FilterInputStream.read(FilterInputStream.java:66)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <17f7683> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"JMX server connection timeout 1814" - Thread t@1814

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <9653ac> (a [I)

at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"RMI TCP Connection(24909)-10.4.1.125" - Thread t@1803

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)

at java.io.BufferedInputStream.read(BufferedInputStream.java:237)

- locked <ca13f2> (a java.io.BufferedInputStream)

at java.io.FilterInputStream.read(FilterInputStream.java:66)

at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <e39a33> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-74" - Thread t@1796

java.lang.Thread.State: BLOCKED

at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)

- waiting to lock <31a517> (a org.jvnet.hk2.osgiadapter.OSGiModuleImpl) owned by "Thread-73" t@1794

at com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)

Locked ownable synchronizers:

- None

"Thread-73" - Thread t@1794

java.lang.Thread.State: BLOCKED

at oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3403)

- waiting to lock <1dd20e> (a oracle.jdbc.driver.T4CConnection) owned by "connector-timer-proxy" t@68

at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:347)

at com.sun.enterprise.resource.allocator.LocalTxConnectorAllocator.destroyResource(LocalTxConnectorAllocator.java:118)

at com.sun.enterprise.resource.pool.ConnectionPool.deleteResource(ConnectionPool.java:884)

at com.sun.enterprise.resource.pool.datastructure.ListDataStructure.removeAll(ListDataStructure.java:208)

- locked <a0b0ae> (a java.util.ArrayList)

- locked <1af824b> (a java.util.ArrayList)

at com.sun.enterprise.resource.pool.ConnectionPool.emptyPool(ConnectionPool.java:1183)

at com.sun.enterprise.resource.pool.PoolManagerImpl.killPool(PoolManagerImpl.java:442)

at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.killPool(ConnectorConnectionPoolAdminServiceImpl.java:396)

at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.deleteConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:349)

at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.deleteConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:302)

at com.sun.enterprise.connectors.ConnectorRuntime.deleteConnectorConnectionPool(ConnectorRuntime.java:239)

at com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer.actualUndeployResource(JdbcConnectionPoolDeployer.java:170)

- locked <1478121> (a com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer)

at com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer.undeployResource(JdbcConnectionPoolDeployer.java:146)

- locked <1478121> (a com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer)

at org.glassfish.javaee.services.ResourceManager.undeployResources(ResourceManager.java:199)

at org.glassfish.javaee.services.ResourceManager.preDestroy(ResourceManager.java:166)

at com.sun.hk2.component.AbstractWombInhabitantImpl.dispose(AbstractWombInhabitantImpl.java:74)

at com.sun.hk2.component.SingletonInhabitant.release(SingletonInhabitant.java:66)

at com.sun.hk2.component.LazyInhabitant.release(LazyInhabitant.java:112)

at com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:393)

at org.jvnet.hk2.osgiadapter.HK2Main.stop(HK2Main.java:154)

at org.apache.felix.framework.util.SecureAction.stopActivator(SecureAction.java:662)

at org.apache.felix.framework.Felix.stopBundle(Felix.java:2071)

at org.apache.felix.framework.BundleImpl.stop(BundleImpl.java:951)

at org.apache.felix.framework.BundleImpl.stop(BundleImpl.java:938)

at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.detach(OSGiModuleImpl.java:249)

at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)

- locked <31a517> (a org.jvnet.hk2.osgiadapter.OSGiModuleImpl)

at com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)

at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)

Locked ownable synchronizers:

- None

"pool-26-thread-1" - Thread t@1782

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1b9a686> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)

at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <16eb1ac> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"pool-24-thread-1" - Thread t@1780

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <163d56f> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)

at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <52b7d7> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"pool-23-thread-1" - Thread t@1779

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <15180aa> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)

at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <a39f17> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"http-thread-pool-8080-(18)" - Thread t@950

java.lang.Thread.State: RUNNABLE

at java.util.zip.Deflater.deflateBytes(Native Method)

at java.util.zip.Deflater.deflate(Deflater.java:306)

- locked <21b0c5> (a java.util.zip.ZStreamRef)

at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)

at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)

at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)

- locked <132ba84> (a java.util.zip.GZIPOutputStream)

at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)

at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)

at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)

at org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)

at org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)

at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)

at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)

at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)

- locked <1ab9b7e> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)

- locked <16129e9> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)

- locked <1649a52> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)

at org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)

at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)

at org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)

at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)

at com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)

at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)

at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)

at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)

at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)

at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)

at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)

at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)

at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)

at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)

at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)

at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)

at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)

at com.sun.grizzly.ContextTask.run(ContextTask.java:69)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <1cfb6a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"JMX server connection timeout 627" - Thread t@627

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <11bd42c> (a [I)

at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Timer-1" - Thread t@549

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <8c7c11> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"p: thread-pool-1; w: 3" - Thread t@545

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <130c9b8> (a com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl)

at com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:120)

at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:506)

Locked ownable synchronizers:

- None

"SelectorThread" - Thread t@536

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <bf3380> (a sun.nio.ch.Util$2)

- locked <71b1d6> (a java.util.Collections$UnmodifiableSet)

- locked <1533ea9> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.corba.ee.impl.transport.SelectorImpl.run(SelectorImpl.java:311)

Locked ownable synchronizers:

- None

"Thread-61" - Thread t@538

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1dc5085> (a com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)

at java.lang.Object.wait(Object.java:485)

at com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive.run(Util.java:827)

Locked ownable synchronizers:

- None

"http-thread-pool-8282-(1)" - Thread t@534

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1ffd1df> (a java.lang.Object)

at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"http-thread-pool-8181-(1)" - Thread t@533

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <19ef62b> (a java.lang.Object)

at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Smack Listener Processor (1)" - Thread t@195

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <9c0c9a> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Smack Listener Processor (0)" - Thread t@194

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <13aac45> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Smack Keep Alive (1)" - Thread t@192

java.lang.Thread.State: TIMED_WAITING

at java.lang.Thread.sleep(Native Method)

at org.jivesoftware.smack.PacketWriter$KeepAliveTask.run(PacketWriter.java:451)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Smack Packet Reader (1)" - Thread t@191

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)

at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)

- locked <36e9b9> (a java.lang.Object)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)

at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)

- locked <51f6c6> (a com.sun.net.ssl.internal.ssl.AppInputStream)

at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)

at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)

at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

- locked <2672c3> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(InputStreamReader.java:167)

at java.io.BufferedReader.fill(BufferedReader.java:136)

at java.io.BufferedReader.read1(BufferedReader.java:187)

at java.io.BufferedReader.read(BufferedReader.java:261)

- locked <2672c3> (a java.io.InputStreamReader)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2992)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3046)

at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)

at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)

at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)

at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)

at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)

Locked ownable synchronizers:

- None

"Smack Packet Writer (1)" - Thread t@190

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <103f91b> (a java.util.concurrent.ArrayBlockingQueue)

at java.lang.Object.wait(Object.java:485)

at org.jivesoftware.smack.PacketWriter.nextPacket(PacketWriter.java:239)

at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:255)

at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)

at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)

Locked ownable synchronizers:

- None

"Smack Keep Alive (0)" - Thread t@188

java.lang.Thread.State: TIMED_WAITING

at java.lang.Thread.sleep(Native Method)

at org.jivesoftware.smack.PacketWriter$KeepAliveTask.run(PacketWriter.java:451)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Smack Packet Reader (0)" - Thread t@187

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)

at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)

- locked <3877e0> (a java.lang.Object)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)

at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)

- locked <1f65aa7> (a com.sun.net.ssl.internal.ssl.AppInputStream)

at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)

at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)

at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

- locked <17905df> (a java.io.InputStreamReader)

at java.io.InputStreamReader.read(InputStreamReader.java:167)

at java.io.BufferedReader.fill(BufferedReader.java:136)

at java.io.BufferedReader.read1(BufferedReader.java:187)

at java.io.BufferedReader.read(BufferedReader.java:261)

- locked <17905df> (a java.io.InputStreamReader)

at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2992)

at org.xmlpull.mxp1.MXParser.more(MXParser.java:3046)

at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)

at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)

at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)

at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)

at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)

Locked ownable synchronizers:

- None

"Smack Packet Writer (0)" - Thread t@186

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <82cd7c> (a java.util.concurrent.ArrayBlockingQueue)

at java.lang.Object.wait(Object.java:485)

at org.jivesoftware.smack.PacketWriter.nextPacket(PacketWriter.java:239)

at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:255)

at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)

at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)

Locked ownable synchronizers:

- None

"pool-16-thread-1" - Thread t@161

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <721925> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)

at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

at java.util.concurrent.FutureTask.run(FutureTask.java:138)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <709821> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"AMXConfigLoader.AMXConfigLoaderThread" - Thread t@160

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <d0ddcd> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at org.glassfish.admin.mbeanserver.PendingConfigBeans.take(PendingConfigBeans.java:96)

at org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.doRun(AMXConfigLoader.java:582)

at org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.run(AMXConfigLoader.java:533)

Locked ownable synchronizers:

- None

"ComplianceMonitor.ValidatorThread" - Thread t@157

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <91f798> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.doRun(ComplianceMonitor.java:240)

at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.run(ComplianceMonitor.java:227)

Locked ownable synchronizers:

- None

"log-rotation-timer" - Thread t@146

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <db4814> (a java.util.TaskQueue)

at java.lang.Object.wait(Object.java:485)

at java.util.TimerThread.mainLoop(Timer.java:483)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"com.google.inject.internal.Finalizer" - Thread t@127

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <f5f4a1> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at com.google.inject.internal.Finalizer.run(Finalizer.java:114)

Locked ownable synchronizers:

- None

"org.hibernate.cache.StandardQueryCache.data" - Thread t@126

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <18f954b> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"org.hibernate.cache.UpdateTimestampsCache.data" - Thread t@125

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1928b6c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Vod.vodImages.data" - Thread t@124

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1ff79e7> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Genre.data" - Thread t@123

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <19c5ada> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Vod.data" - Thread t@122

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <10a844> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Role.data" - Thread t@121

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1974904> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Program.data" - Thread t@119

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <13ae2f2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Image.data" - Thread t@118

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <109b5e4> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Program.programImages.data" - Thread t@117

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <84a6f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Source.data" - Thread t@116

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <ebdf08> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"net.sf.ehcache.CacheManager@1ef41c8" - Thread t@115

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <17fc230> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"http-thread-pool-8080-(2)" - Thread t@112

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <593635> (a java.lang.Object)

at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"http-thread-pool-8080-(1)" - Thread t@110

java.lang.Thread.State: RUNNABLE

at java.util.zip.Deflater.deflateBytes(Native Method)

at java.util.zip.Deflater.deflate(Deflater.java:306)

- locked <13ccea> (a java.util.zip.ZStreamRef)

at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)

at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)

at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)

- locked <194592c> (a java.util.zip.GZIPOutputStream)

at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)

at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)

at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)

at org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)

at org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)

at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)

at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)

at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)

- locked <6cc956> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)

- locked <48de4d> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)

- locked <2858ff> (a org.apache.cxf.phase.PhaseInterceptorChain)

at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)

at org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)

at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)

at org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)

at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)

at com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)

at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)

at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)

at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)

at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)

at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)

at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)

at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)

at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)

at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)

at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)

at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)

at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)

at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)

at com.sun.grizzly.ContextTask.run(ContextTask.java:69)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <ead6a9> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true, service.pid=org.apache.felix.fileinstall.568dcc01-97d1-4e8f-9756-3466c566b7f3, felix.fileinstall.dir=/opt/glassfishv3/glassfish/domains/domain1/autodeploy/bundles/, felix.fileinstall.filename=org.apache.felix.fileinstall-autodeploy-bundles.cfg, service.factorypid=org.apache.felix.fileinstall, felix.fileinstall.debug=1}" - Thread t@109

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <a5f299> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)

at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)

Locked ownable synchronizers:

- None

"SCR Component Actor" - Thread t@107

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <da1c2a> (a java.util.LinkedList)

at java.lang.Object.wait(Object.java:485)

at org.apache.felix.scr.impl.ComponentActorThread.run(ComponentActorThread.java:63)

Locked ownable synchronizers:

- None

"{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true, felix.fileinstall.dir=/opt/glassfishv3/glassfish/modules/autostart/, felix.fileinstall.debug=1}" - Thread t@106

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <16f3905> (a org.apache.felix.fileinstall.internal.DirectoryWatcher)

at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)

Locked ownable synchronizers:

- None

"Configuration Updater" - Thread t@105

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <108b278> (a java.util.LinkedList)

at java.lang.Object.wait(Object.java:485)

at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)

Locked ownable synchronizers:

- None

"telnetconsole.Listener" - Thread t@104

java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

- locked <151a> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:462)

at java.net.ServerSocket.accept(ServerSocket.java:430)

at org.apache.felix.shell.remote.Listener$Acceptor.run(Listener.java:137)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"GlassFish Kernel Main Thread" - Thread t@103

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1fce9f0> (a com.sun.enterprise.v3.server.AppServerStartup$1)

at java.lang.Object.wait(Object.java:485)

at com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:146)

Locked ownable synchronizers:

- None

"RMI RenewClean-[10.49.5.27:8686]" - Thread t@102

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <162c0e6> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"RMI Scheduler(0)" - Thread t@99

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1cf1438> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"AutoDeployer" - Thread t@97

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <389dc6> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"RMI Reaper" - Thread t@95

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <fe5942> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"ping" - Thread t@94

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <9608df> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"RMI TCP Accept-8686" - Thread t@93

java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

- locked <16984f1> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:462)

at java.net.ServerSocket.accept(ServerSocket.java:430)

at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)

at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"DynamicReloader" - Thread t@91

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <209b09> (a java.util.TaskQueue)

at java.lang.Object.wait(Object.java:485)

at java.util.TimerThread.mainLoop(Timer.java:483)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"pool-8-thread-1" - Thread t@82

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <10a5750> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"OracleTimeoutPollingThread" - Thread t@81

java.lang.Thread.State: TIMED_WAITING

at java.lang.Thread.sleep(Native Method)

at oracle.jdbc.driver.OracleTimeoutPollingThread.run(OracleTimeoutPollingThread.java:150)

Locked ownable synchronizers:

- None

"com.google.inject.internal.Finalizer" - Thread t@80

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1ea5622> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at com.google.inject.internal.Finalizer.run(Finalizer.java:114)

Locked ownable synchronizers:

- None

"org.hibernate.cache.StandardQueryCache.data" - Thread t@79

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1bce9ba> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"org.hibernate.cache.UpdateTimestampsCache.data" - Thread t@78

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <194edb2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Vod.vodImages.data" - Thread t@77

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <817e73> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Genre.data" - Thread t@76

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <194da56> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Vod.data" - Thread t@75

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <18a5e8e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Role.data" - Thread t@74

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <145f6de> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Source.data" - Thread t@73

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <52831e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Program.data" - Thread t@72

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <11e426e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Image.data" - Thread t@71

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <19b7698> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"com.mcc.ipr.guide.persistence.Program.programImages.data" - Thread t@70

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1872217> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"net.sf.ehcache.CacheManager@3bcdbd" - Thread t@69

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <6642e7> (a java.util.TaskQueue)

at java.util.TimerThread.mainLoop(Timer.java:509)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"connector-timer-proxy" - Thread t@68

java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.read(SocketInputStream.java:129)

at oracle.net.ns.Packet.receive(Packet.java:239)

at oracle.net.ns.DataPacket.receive(DataPacket.java:92)

at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:172)

at oracle.net.ns.NetInputStream.read(NetInputStream.java:117)

at oracle.net.ns.NetInputStream.read(NetInputStream.java:92)

at oracle.net.ns.NetInputStream.read(NetInputStream.java:77)

at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1023)

at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:999)

at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:584)

at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:194)

at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:785)

at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:860)

at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1186)

at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3381)

at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3425)

- locked <1dd20e> (a oracle.jdbc.driver.T4CConnection)

at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1202)

at com.sun.gjc.spi.ManagedConnectionFactory.isValidByTableQuery(ManagedConnectionFactory.java:421)

at com.sun.gjc.spi.ManagedConnectionFactory.isValid(ManagedConnectionFactory.java:306)

at com.sun.gjc.spi.ManagedConnectionFactory.getInvalidConnections(ManagedConnectionFactory.java:253)

at com.sun.enterprise.resource.allocator.AbstractConnectorAllocator.getInvalidConnections(AbstractConnectorAllocator.java:107)

at com.sun.enterprise.resource.pool.ConnectionPool.getInvalidConnections(ConnectionPool.java:1128)

at com.sun.enterprise.resource.pool.resizer.Resizer.removeInvalidResources(Resizer.java:280)

at com.sun.enterprise.resource.pool.resizer.Resizer.removeIdleAndInvalidResources(Resizer.java:240)

at com.sun.enterprise.resource.pool.resizer.Resizer.resizePool(Resizer.java:110)

at com.sun.enterprise.resource.pool.resizer.Resizer.run(Resizer.java:90)

at java.util.TimerThread.mainLoop(Timer.java:512)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"Timer-0" - Thread t@67

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <14ff65b> (a java.util.TaskQueue)

at java.lang.Object.wait(Object.java:485)

at java.util.TimerThread.mainLoop(Timer.java:483)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"http-thread-pool-4848-(2)" - Thread t@66

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <b6f0d6> (a java.lang.Object)

at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-32" - Thread t@65

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <7d7f4d> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"p: thread-pool-1; w: 2" - Thread t@64

java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

- locked <96f73f> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:462)

at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)

at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)

at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)

at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)

at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)

Locked ownable synchronizers:

- None

"p: thread-pool-1; w: 1" - Thread t@63

java.lang.Thread.State: RUNNABLE

at java.net.PlainSocketImpl.socketAccept(Native Method)

at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

- locked <142799d> (a java.net.SocksSocketImpl)

at java.net.ServerSocket.implAccept(ServerSocket.java:462)

at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)

at com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)

at com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)

at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)

at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)

Locked ownable synchronizers:

- None

"GC Daemon" - Thread t@61

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <b77a6> (a sun.misc.GC$LatencyLock)

at sun.misc.GC$Daemon.run(GC.java:100)

Locked ownable synchronizers:

- None

"Thread-31" - Thread t@59

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <dc3977> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-30" - Thread t@50

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <40d6f2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-29" - Thread t@49

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <dfd61b> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-28" - Thread t@48

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <161a4cf> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-27" - Thread t@47

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <18b3184> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-26" - Thread t@46

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <15ffcfc> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Grizzly" - Thread t@45

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <da178f> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-25" - Thread t@44

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <40ce8c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"http-thread-pool-4848-(1)" - Thread t@43

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <b6f0d6> (a java.lang.Object)

at com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)

at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)

at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"transaction-manager" - Thread t@42

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <b8610d> (a java.util.TaskQueue)

at java.lang.Object.wait(Object.java:485)

at java.util.TimerThread.mainLoop(Timer.java:483)

at java.util.TimerThread.run(Timer.java:462)

Locked ownable synchronizers:

- None

"Thread-23" - Thread t@41

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <13e372a> (a sun.nio.ch.Util$2)

- locked <197dd67> (a java.util.Collections$UnmodifiableSet)

- locked <148b6d5> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <db87be> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-21" - Thread t@39

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <ace60e> (a sun.nio.ch.Util$2)

- locked <133be33> (a java.util.Collections$UnmodifiableSet)

- locked <c3719> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <1f56a22> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-19" - Thread t@37

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <ebad43> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <10ae86a> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-18" - Thread t@36

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <bc1eb2> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <120b2c4> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-17" - Thread t@35

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <1b8823c> (a sun.nio.ch.Util$2)

- locked <6e7099> (a java.util.Collections$UnmodifiableSet)

- locked <a25435> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <1469f65> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-16" - Thread t@33

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <18ac785> (a sun.nio.ch.Util$2)

- locked <3119a9> (a java.util.Collections$UnmodifiableSet)

- locked <18bd37a> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <e93001> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-15" - Thread t@31

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <14eacca> (a sun.nio.ch.Util$2)

- locked <321f2a> (a java.util.Collections$UnmodifiableSet)

- locked <9c2daf> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <17b6caf> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-9" - Thread t@29

java.lang.Thread.State: RUNNABLE

at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)

at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)

at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)

at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)

- locked <1c8622> (a sun.nio.ch.Util$2)

- locked <aa32a6> (a java.util.Collections$UnmodifiableSet)

- locked <1e3493f> (a sun.nio.ch.EPollSelectorImpl)

at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)

at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)

at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)

at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- locked <de0390> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-12" - Thread t@28

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <833c98> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <1bbf1be> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-10" - Thread t@26

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1ff5bf1> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <1fc0b7a> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-8" - Thread t@24

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1ed3ca7> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <55b420> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-6" - Thread t@21

java.lang.Thread.State: TIMED_WAITING

at java.lang.Object.wait(Native Method)

- waiting on <28f9f4> (a java.util.concurrent.atomic.AtomicInteger)

at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)

at com.sun.grizzly.Controller.start(Controller.java:843)

at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)

- locked <a4c3ec> (a java.lang.Object)

at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)

at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)

at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)

at com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)

Locked ownable synchronizers:

- None

"Thread-5" - Thread t@19

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <32293c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-4" - Thread t@18

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1312d86> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-3" - Thread t@17

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1a3d4b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-2" - Thread t@16

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <e2bc79> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)

at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Thread-1" - Thread t@15

java.lang.Thread.State: WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1c3b391> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)

at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)

at com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:510)

at com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:255)

Locked ownable synchronizers:

- None

"pool-2-thread-1" - Thread t@14

java.lang.Thread.State: TIMED_WAITING

at sun.misc.Unsafe.park(Native Method)

- parking to wait for <1421966> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)

at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)

at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)

at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"FelixPackageAdmin" - Thread t@13

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <91782f> (a org.apache.felix.framework.PackageAdminImpl)

at java.lang.Object.wait(Object.java:485)

at org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:314)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"FelixStartLevel" - Thread t@12

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1b273e7> (a java.util.ArrayList)

at java.lang.Object.wait(Object.java:485)

at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"FelixDispatchQueue" - Thread t@11

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <696349> (a java.util.ArrayList)

at java.lang.Object.wait(Object.java:485)

at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:931)

at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)

at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"DestroyJavaVM" - Thread t@9

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"OSGi Framework Launcher" - Thread t@8

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <d750e5> (a org.apache.felix.framework.util.ThreadGate)

at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)

at org.apache.felix.framework.Felix.waitForStop(Felix.java:794)

at org.apache.felix.main.Main.main(Main.java:299)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.sun.enterprise.glassfish.bootstrap.ASMainFelix$1.run(ASMainFelix.java:116)

at java.lang.Thread.run(Thread.java:662)

Locked ownable synchronizers:

- None

"Signal Dispatcher" - Thread t@4

java.lang.Thread.State: RUNNABLE

Locked ownable synchronizers:

- None

"Finalizer" - Thread t@3

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <1baadd4> (a java.lang.ref.ReferenceQueue$Lock)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)

at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

Locked ownable synchronizers:

- None

"Reference Handler" - Thread t@2

java.lang.Thread.State: WAITING

at java.lang.Object.wait(Native Method)

- waiting on <ea6a65> (a java.lang.ref.Reference$Lock)

at java.lang.Object.wait(Object.java:485)

at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

Locked ownable synchronizers:

- None

oleksiys
Offline
Joined: 2006-01-25

> Last question first - We are not creating separate threads within the web
> service (I understand that the conventional wisdom on that is "BAD
> IDEA"). We
> do have an HTTP thread pool configured within Glassfish for multiple
> requests.
Right, I just wondered whether CXF doesn't do that.

> The complete thread dump is as follows.
>
> It is starting to look like either:
>
> * The two http pool threads are stuck inside the gzip native code or
>
> * The two http pool threads are bouncing in an out of the gzip native
> code.
>
> We found some wierdness (bug?) in java.util.zip.DeflaterOutputStream
> where it
> appears that on write(buf,off,len) the result appears to be that
> deflate is
> called which works on the protected member buf[] rather than the
> passed in
> buffer b[].
IMO this part of code is fine. Here is the comment:
// Deflate no more than stride bytes at a time. This avoids
// excess copying in deflateBytes (see Deflater.c)

so the just chunking source byte[] to avoid redundant copying.

I see that GZIPOutputStream.write(byte[], int, int) method is
synchronized, so it's actually thread-safe, but
GZIPOutputStream.finish() is not synchronized. The only thing I can
suspect is GZIPOutputStream.finish() and GZIPOutputStream.write(...) are
getting called simultaneously, which leads to spinning.

Thanks.

WBR,
Alexey.

>
> Full thread dump:
>
>
>
>
>
> 2011-10-13 10:28:55
>
> Full thread dump Java HotSpot(TM) Client VM (19.0-b09 mixed mode):
>
>
>
> "RMI TCP Connection(idle)" - Thread t@1822
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1d29b42> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
>
>
> at
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
>
>
> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-79" - Thread t@1821
>
> java.lang.Thread.State: BLOCKED
>
> at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)
>
> - waiting to lock <31a517> (a
> org.jvnet.hk2.osgiadapter.OSGiModuleImpl) owned
> by "Thread-73" t@1794
>
> at
> com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI TCP Connection(25112)-10.4.0.88" - Thread t@1820
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>
> - locked <1a966ca> (a java.io.BufferedInputStream)
>
> at java.io.FilterInputStream.read(FilterInputStream.java:66)
>
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <9b9212> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "RMI TCP Connection(25115)-10.49.5.34" - Thread t@1819
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.management.ThreadImpl.dumpThreads0(Native Method)
>
> at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:374)
>
> at sun.reflect.GeneratedMethodAccessor2900.invoke(Unknown Source)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at
> com.sun.jmx.mbeanserver.ConvertingMethod.invokeWithOpenReturn(ConvertingMethod.java:167)
>
>
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:96)
>
>
> at
> com.sun.jmx.mbeanserver.MXBeanIntrospector.invokeM2(MXBeanIntrospector.java:33)
>
>
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>
>
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>
> at javax.management.StandardMBean.invoke(StandardMBean.java:391)
>
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>
>
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>
>
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>
>
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>
>
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
>
>
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>
>
> at sun.reflect.GeneratedMethodAccessor1010.invoke(Unknown Source)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <192f091> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "JMX server connection timeout 1818" - Thread t@1818
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <11e7538> (a [I)
>
> at
> com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI TCP Connection(25116)-10.49.5.34" - Thread t@1815
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>
> - locked <1b8bcfc> (a java.io.BufferedInputStream)
>
> at java.io.FilterInputStream.read(FilterInputStream.java:66)
>
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <17f7683> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "JMX server connection timeout 1814" - Thread t@1814
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <9653ac> (a [I)
>
> at
> com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI TCP Connection(24909)-10.4.1.125" - Thread t@1803
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>
> at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
>
> - locked (a java.io.BufferedInputStream)
>
> at java.io.FilterInputStream.read(FilterInputStream.java:66)
>
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:517)
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>
>
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-74" - Thread t@1796
>
> java.lang.Thread.State: BLOCKED
>
> at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)
>
> - waiting to lock <31a517> (a
> org.jvnet.hk2.osgiadapter.OSGiModuleImpl) owned
> by "Thread-73" t@1794
>
> at
> com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-73" - Thread t@1794
>
> java.lang.Thread.State: BLOCKED
>
> at
> oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3403)
>
> - waiting to lock <1dd20e> (a oracle.jdbc.driver.T4CConnection) owned by
> "connector-timer-proxy" t@68
>
> at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:347)
>
> at
> com.sun.enterprise.resource.allocator.LocalTxConnectorAllocator.destroyResource(LocalTxConnectorAllocator.java:118)
>
>
> at
> com.sun.enterprise.resource.pool.ConnectionPool.deleteResource(ConnectionPool.java:884)
>
>
> at
> com.sun.enterprise.resource.pool.datastructure.ListDataStructure.removeAll(ListDataStructure.java:208)
>
>
> - locked (a java.util.ArrayList)
>
> - locked <1af824b> (a java.util.ArrayList)
>
> at
> com.sun.enterprise.resource.pool.ConnectionPool.emptyPool(ConnectionPool.java:1183)
>
>
> at
> com.sun.enterprise.resource.pool.PoolManagerImpl.killPool(PoolManagerImpl.java:442)
>
>
> at
> com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.killPool(ConnectorConnectionPoolAdminServiceImpl.java:396)
>
>
> at
> com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.deleteConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:349)
>
>
> at
> com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.deleteConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:302)
>
>
> at
> com.sun.enterprise.connectors.ConnectorRuntime.deleteConnectorConnectionPool(ConnectorRuntime.java:239)
>
>
> at
> com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer.actualUndeployResource(JdbcConnectionPoolDeployer.java:170)
>
>
> - locked <1478121> (a
> com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer)
>
> at
> com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer.undeployResource(JdbcConnectionPoolDeployer.java:146)
>
>
> - locked <1478121> (a
> com.sun.enterprise.resource.deployer.JdbcConnectionPoolDeployer)
>
> at
> org.glassfish.javaee.services.ResourceManager.undeployResources(ResourceManager.java:199)
>
>
> at
> org.glassfish.javaee.services.ResourceManager.preDestroy(ResourceManager.java:166)
>
>
> at
> com.sun.hk2.component.AbstractWombInhabitantImpl.dispose(AbstractWombInhabitantImpl.java:74)
>
>
> at
> com.sun.hk2.component.SingletonInhabitant.release(SingletonInhabitant.java:66)
>
>
> at com.sun.hk2.component.LazyInhabitant.release(LazyInhabitant.java:112)
>
> at
> com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:393)
>
>
> at org.jvnet.hk2.osgiadapter.HK2Main.stop(HK2Main.java:154)
>
> at
> org.apache.felix.framework.util.SecureAction.stopActivator(SecureAction.java:662)
>
>
> at org.apache.felix.framework.Felix.stopBundle(Felix.java:2071)
>
> at org.apache.felix.framework.BundleImpl.stop(BundleImpl.java:951)
>
> at org.apache.felix.framework.BundleImpl.stop(BundleImpl.java:938)
>
> at
> org.jvnet.hk2.osgiadapter.OSGiModuleImpl.detach(OSGiModuleImpl.java:249)
>
> at org.jvnet.hk2.osgiadapter.OSGiModuleImpl.stop(OSGiModuleImpl.java:227)
>
> - locked <31a517> (a org.jvnet.hk2.osgiadapter.OSGiModuleImpl)
>
> at
> com.sun.enterprise.v3.admin.StopDomainCommand.execute(StopDomainCommand.java:86)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)
>
>
> at
> com.sun.enterprise.v3.admin.CommandRunnerImpl$2.run(CommandRunnerImpl.java:333)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "pool-26-thread-1" - Thread t@1782
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1b9a686> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <16eb1ac> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "pool-24-thread-1" - Thread t@1780
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <163d56f> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <52b7d7> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "pool-23-thread-1" - Thread t@1779
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <15180aa> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "http-thread-pool-8080-(18)" - Thread t@950
>
> java.lang.Thread.State: RUNNABLE
>
> at java.util.zip.Deflater.deflateBytes(Native Method)
>
> at java.util.zip.Deflater.deflate(Deflater.java:306)
>
> - locked <21b0c5> (a java.util.zip.ZStreamRef)
>
> at
> java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
>
> at
> java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
>
> at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)
>
> - locked <132ba84> (a java.util.zip.GZIPOutputStream)
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
>
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)
>
>
> at
> org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
>
> at
> org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)
>
>
> at
> org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
>
> - locked <1ab9b7e> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)
>
>
> - locked <16129e9> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
>
> - locked <1649a52> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)
>
>
> at
> org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)
>
>
> at
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)
>
>
> at
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)
>
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)
>
>
> at
> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
>
> at
> com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
>
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
>
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
>
> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
>
> at
> com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
>
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
>
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
>
> at
> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)
>
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
>
>
> at
> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
>
>
> at
> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
>
> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
>
> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
>
> at
> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
>
> at
> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
>
> at
> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>
>
> at
> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>
>
> at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <1cfb6a0> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "JMX server connection timeout 627" - Thread t@627
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <11bd42c> (a [I)
>
> at
> com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Timer-1" - Thread t@549
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <8c7c11> (a java.util.TaskQueue)
>
> at java.util.TimerThread.mainLoop(Timer.java:509)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "p: thread-pool-1; w: 3" - Thread t@545
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <130c9b8> (a
> com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl)
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.WorkQueueImpl.requestWork(WorkQueueImpl.java:120)
>
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:506)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "SelectorThread" - Thread t@536
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked (a sun.nio.ch.Util$2)
>
> - locked <71b1d6> (a java.util.Collections$UnmodifiableSet)
>
> - locked <1533ea9> (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at
> com.sun.corba.ee.impl.transport.SelectorImpl.run(SelectorImpl.java:311)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-61" - Thread t@538
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1dc5085> (a
> com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive)
>
> at java.lang.Object.wait(Object.java:485)
>
> at com.sun.corba.ee.impl.javax.rmi.CORBA.KeepAlive.run(Util.java:827)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-8282-(1)" - Thread t@534
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1ffd1df> (a java.lang.Object)
>
> at
> com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-8181-(1)" - Thread t@533
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <19ef62b> (a java.lang.Object)
>
> at
> com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Listener Processor (1)" - Thread t@195
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <9c0c9a> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Listener Processor (0)" - Thread t@194
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <13aac45> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Keep Alive (1)" - Thread t@192
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Thread.sleep(Native Method)
>
> at
> org.jivesoftware.smack.PacketWriter$KeepAliveTask.run(PacketWriter.java:451)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Packet Reader (1)" - Thread t@191
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at
> com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
>
> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
>
> at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
>
>
> - locked <36e9b9> (a java.lang.Object)
>
> at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)
>
>
> at
> com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
>
> - locked <51f6c6> (a com.sun.net.ssl.internal.ssl.AppInputStream)
>
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
>
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
>
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
>
> - locked <2672c3> (a java.io.InputStreamReader)
>
> at java.io.InputStreamReader.read(InputStreamReader.java:167)
>
> at java.io.BufferedReader.fill(BufferedReader.java:136)
>
> at java.io.BufferedReader.read1(BufferedReader.java:187)
>
> at java.io.BufferedReader.read(BufferedReader.java:261)
>
> - locked <2672c3> (a java.io.InputStreamReader)
>
> at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2992)
>
> at org.xmlpull.mxp1.MXParser.more(MXParser.java:3046)
>
> at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)
>
> at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
>
> at
> org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)
>
> at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
>
> at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Packet Writer (1)" - Thread t@190
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <103f91b> (a java.util.concurrent.ArrayBlockingQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at org.jivesoftware.smack.PacketWriter.nextPacket(PacketWriter.java:239)
>
> at
> org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:255)
>
> at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
>
> at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Keep Alive (0)" - Thread t@188
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Thread.sleep(Native Method)
>
> at
> org.jivesoftware.smack.PacketWriter$KeepAliveTask.run(PacketWriter.java:451)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Packet Reader (0)" - Thread t@187
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at
> com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
>
> at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
>
> at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
>
>
> - locked <3877e0> (a java.lang.Object)
>
> at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)
>
>
> at
> com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
>
> - locked <1f65aa7> (a com.sun.net.ssl.internal.ssl.AppInputStream)
>
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
>
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
>
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
>
> - locked <17905df> (a java.io.InputStreamReader)
>
> at java.io.InputStreamReader.read(InputStreamReader.java:167)
>
> at java.io.BufferedReader.fill(BufferedReader.java:136)
>
> at java.io.BufferedReader.read1(BufferedReader.java:187)
>
> at java.io.BufferedReader.read(BufferedReader.java:261)
>
> - locked <17905df> (a java.io.InputStreamReader)
>
> at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2992)
>
> at org.xmlpull.mxp1.MXParser.more(MXParser.java:3046)
>
> at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)
>
> at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
>
> at
> org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)
>
> at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
>
> at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Smack Packet Writer (0)" - Thread t@186
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <82cd7c> (a java.util.concurrent.ArrayBlockingQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at org.jivesoftware.smack.PacketWriter.nextPacket(PacketWriter.java:239)
>
> at
> org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:255)
>
> at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
>
> at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "pool-16-thread-1" - Thread t@161
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <721925> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:157)
>
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <709821> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "AMXConfigLoader.AMXConfigLoaderThread" - Thread t@160
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at
> org.glassfish.admin.mbeanserver.PendingConfigBeans.take(PendingConfigBeans.java:96)
>
>
> at
> org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.doRun(AMXConfigLoader.java:582)
>
>
> at
> org.glassfish.admin.amx.impl.config.AMXConfigLoader$AMXConfigLoaderThread.run(AMXConfigLoader.java:533)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "ComplianceMonitor.ValidatorThread" - Thread t@157
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <91f798> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at
> org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.doRun(ComplianceMonitor.java:240)
>
>
> at
> org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.run(ComplianceMonitor.java:227)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "log-rotation-timer" - Thread t@146
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.util.TaskQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at java.util.TimerThread.mainLoop(Timer.java:483)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.google.inject.internal.Finalizer" - Thread t@127
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.lang.ref.ReferenceQueue$Lock)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
>
> at com.google.inject.internal.Finalizer.run(Finalizer.java:114)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "org.hibernate.cache.StandardQueryCache.data" - Thread t@126
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <18f954b> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "org.hibernate.cache.UpdateTimestampsCache.data" - Thread t@125
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1928b6c> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Vod.vodImages.data" - Thread t@124
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1ff79e7> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Genre.data" - Thread t@123
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <19c5ada> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Vod.data" - Thread t@122
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <10a844> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Role.data" - Thread t@121
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1974904> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Program.data" - Thread t@119
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <13ae2f2> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Image.data" - Thread t@118
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <109b5e4> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Program.programImages.data" - Thread t@117
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <84a6f0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Source.data" - Thread t@116
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "net.sf.ehcache.CacheManager@1ef41c8" - Thread t@115
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <17fc230> (a java.util.TaskQueue)
>
> at java.util.TimerThread.mainLoop(Timer.java:509)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-8080-(2)" - Thread t@112
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <593635> (a java.lang.Object)
>
> at
> com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-8080-(1)" - Thread t@110
>
> java.lang.Thread.State: RUNNABLE
>
> at java.util.zip.Deflater.deflateBytes(Native Method)
>
> at java.util.zip.Deflater.deflate(Deflater.java:306)
>
> - locked <13ccea> (a java.util.zip.ZStreamRef)
>
> at
> java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
>
> at
> java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118)
>
> at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)
>
> - locked <194592c> (a java.util.zip.GZIPOutputStream)
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
>
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
>
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)
>
>
> at
> org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
>
> at
> org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)
>
>
> at
> org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
>
> - locked <6cc956> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)
>
>
> - locked <48de4d> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
>
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
>
> - locked <2858ff> (a org.apache.cxf.phase.PhaseInterceptorChain)
>
> at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)
>
>
> at
> org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)
>
>
> at
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)
>
>
> at
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)
>
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)
>
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)
>
>
> at
> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
>
> at
> com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
>
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
>
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
>
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
>
> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
>
> at
> com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
>
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
>
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
>
> at
> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)
>
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
>
>
> at
> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
>
>
> at
> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
>
> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
>
> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
>
> at
> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
>
> at
> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
>
> at
> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>
>
> at
> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>
>
> at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true,
> service.pid=org.apache.felix.fileinstall.568dcc01-97d1-4e8f-9756-3466c566b7f3,
>
> felix.fileinstall.dir=/opt/glassfishv3/glassfish/domains/domain1/autodeploy/bundles/,
>
> felix.fileinstall.filename=org.apache.felix.fileinstall-autodeploy-bundles.cfg,
>
> service.factorypid=org.apache.felix.fileinstall,
> felix.fileinstall.debug=1}"
> - Thread t@109
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a
> org.apache.felix.fileinstall.internal.DirectoryWatcher)
>
> at
> org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "SCR Component Actor" - Thread t@107
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.util.LinkedList)
>
> at java.lang.Object.wait(Object.java:485)
>
> at
> org.apache.felix.scr.impl.ComponentActorThread.run(ComponentActorThread.java:63)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true,
> felix.fileinstall.dir=/opt/glassfishv3/glassfish/modules/autostart/,
> felix.fileinstall.debug=1}" - Thread t@106
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <16f3905> (a
> org.apache.felix.fileinstall.internal.DirectoryWatcher)
>
> at
> org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:210)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Configuration Updater" - Thread t@105
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <108b278> (a java.util.LinkedList)
>
> at java.lang.Object.wait(Object.java:485)
>
> at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "telnetconsole.Listener" - Thread t@104
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.PlainSocketImpl.socketAccept(Native Method)
>
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>
> - locked <151a> (a java.net.SocksSocketImpl)
>
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
>
> at java.net.ServerSocket.accept(ServerSocket.java:430)
>
> at org.apache.felix.shell.remote.Listener$Acceptor.run(Listener.java:137)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "GlassFish Kernel Main Thread" - Thread t@103
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1fce9f0> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
>
> at java.lang.Object.wait(Object.java:485)
>
> at
> com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:146)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI RenewClean-[10.49.5.27:8686]" - Thread t@102
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <162c0e6> (a java.lang.ref.ReferenceQueue$Lock)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>
> at
> sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI Scheduler(0)" - Thread t@99
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1cf1438> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "AutoDeployer" - Thread t@97
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <389dc6> (a java.util.TaskQueue)
>
> at java.util.TimerThread.mainLoop(Timer.java:509)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI Reaper" - Thread t@95
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.lang.ref.ReferenceQueue$Lock)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
>
> at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "ping" - Thread t@94
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <9608df> (a java.util.TaskQueue)
>
> at java.util.TimerThread.mainLoop(Timer.java:509)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "RMI TCP Accept-8686" - Thread t@93
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.PlainSocketImpl.socketAccept(Native Method)
>
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>
> - locked <16984f1> (a java.net.SocksSocketImpl)
>
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
>
> at java.net.ServerSocket.accept(ServerSocket.java:430)
>
> at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
>
>
> at
> sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "DynamicReloader" - Thread t@91
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <209b09> (a java.util.TaskQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at java.util.TimerThread.mainLoop(Timer.java:483)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "pool-8-thread-1" - Thread t@82
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <10a5750> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "OracleTimeoutPollingThread" - Thread t@81
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Thread.sleep(Native Method)
>
> at
> oracle.jdbc.driver.OracleTimeoutPollingThread.run(OracleTimeoutPollingThread.java:150)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.google.inject.internal.Finalizer" - Thread t@80
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1ea5622> (a java.lang.ref.ReferenceQueue$Lock)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
>
> at com.google.inject.internal.Finalizer.run(Finalizer.java:114)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "org.hibernate.cache.StandardQueryCache.data" - Thread t@79
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1bce9ba> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "org.hibernate.cache.UpdateTimestampsCache.data" - Thread t@78
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <194edb2> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Vod.vodImages.data" - Thread t@77
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <817e73> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Genre.data" - Thread t@76
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <194da56> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Vod.data" - Thread t@75
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <18a5e8e> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Role.data" - Thread t@74
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <145f6de> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Source.data" - Thread t@73
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <52831e> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Program.data" - Thread t@72
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <11e426e> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Image.data" - Thread t@71
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <19b7698> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "com.mcc.ipr.guide.persistence.Program.programImages.data" - Thread t@70
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1872217> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "net.sf.ehcache.CacheManager@3bcdbd" - Thread t@69
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <6642e7> (a java.util.TaskQueue)
>
> at java.util.TimerThread.mainLoop(Timer.java:509)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "connector-timer-proxy" - Thread t@68
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at oracle.net.ns.Packet.receive(Packet.java:239)
>
> at oracle.net.ns.DataPacket.receive(DataPacket.java:92)
>
> at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:172)
>
> at oracle.net.ns.NetInputStream.read(NetInputStream.java:117)
>
> at oracle.net.ns.NetInputStream.read(NetInputStream.java:92)
>
> at oracle.net.ns.NetInputStream.read(NetInputStream.java:77)
>
> at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1023)
>
> at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:999)
>
> at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:584)
>
> at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:194)
>
>
> at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:785)
>
>
> at
> oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:860)
>
>
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1186)
>
>
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3381)
>
>
> at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3425)
>
>
> - locked <1dd20e> (a oracle.jdbc.driver.T4CConnection)
>
> at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1202)
>
>
> at
> com.sun.gjc.spi.ManagedConnectionFactory.isValidByTableQuery(ManagedConnectionFactory.java:421)
>
>
> at
> com.sun.gjc.spi.ManagedConnectionFactory.isValid(ManagedConnectionFactory.java:306)
>
>
> at
> com.sun.gjc.spi.ManagedConnectionFactory.getInvalidConnections(ManagedConnectionFactory.java:253)
>
>
> at
> com.sun.enterprise.resource.allocator.AbstractConnectorAllocator.getInvalidConnections(AbstractConnectorAllocator.java:107)
>
>
> at
> com.sun.enterprise.resource.pool.ConnectionPool.getInvalidConnections(ConnectionPool.java:1128)
>
>
> at
> com.sun.enterprise.resource.pool.resizer.Resizer.removeInvalidResources(Resizer.java:280)
>
>
> at
> com.sun.enterprise.resource.pool.resizer.Resizer.removeIdleAndInvalidResources(Resizer.java:240)
>
>
> at
> com.sun.enterprise.resource.pool.resizer.Resizer.resizePool(Resizer.java:110)
>
>
> at com.sun.enterprise.resource.pool.resizer.Resizer.run(Resizer.java:90)
>
> at java.util.TimerThread.mainLoop(Timer.java:512)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Timer-0" - Thread t@67
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <14ff65b> (a java.util.TaskQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at java.util.TimerThread.mainLoop(Timer.java:483)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-4848-(2)" - Thread t@66
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.lang.Object)
>
> at
> com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-32" - Thread t@65
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <7d7f4d> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "p: thread-pool-1; w: 2" - Thread t@64
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.PlainSocketImpl.socketAccept(Native Method)
>
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>
> - locked <96f73f> (a java.net.SocksSocketImpl)
>
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
>
> at
> com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
>
>
> at
> com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)
>
>
> at
> com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)
>
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)
>
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "p: thread-pool-1; w: 1" - Thread t@63
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.PlainSocketImpl.socketAccept(Native Method)
>
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
>
> - locked <142799d> (a java.net.SocksSocketImpl)
>
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
>
> at
> com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
>
>
> at
> com.sun.corba.ee.impl.transport.SocketOrChannelAcceptorImpl.getAcceptedSocket(SocketOrChannelAcceptorImpl.java:141)
>
>
> at
> com.sun.corba.ee.impl.transport.ListenerThreadImpl.doWork(ListenerThreadImpl.java:111)
>
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:492)
>
>
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:528)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "GC Daemon" - Thread t@61
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a sun.misc.GC$LatencyLock)
>
> at sun.misc.GC$Daemon.run(GC.java:100)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-31" - Thread t@59
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-30" - Thread t@50
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <40d6f2> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-29" - Thread t@49
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-28" - Thread t@48
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <161a4cf> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-27" - Thread t@47
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <18b3184> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-26" - Thread t@46
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <15ffcfc> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Grizzly" - Thread t@45
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-25" - Thread t@44
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <40ce8c> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "http-thread-pool-4848-(1)" - Thread t@43
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.lang.Object)
>
> at
> com.sun.grizzly.util.SyncThreadPool$SyncThreadWorker.getTask(SyncThreadPool.java:445)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:323)
>
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "transaction-manager" - Thread t@42
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.util.TaskQueue)
>
> at java.lang.Object.wait(Object.java:485)
>
> at java.util.TimerThread.mainLoop(Timer.java:483)
>
> at java.util.TimerThread.run(Timer.java:462)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-23" - Thread t@41
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked <13e372a> (a sun.nio.ch.Util$2)
>
> - locked <197dd67> (a java.util.Collections$UnmodifiableSet)
>
> - locked <148b6d5> (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-21" - Thread t@39
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked (a sun.nio.ch.Util$2)
>
> - locked <133be33> (a java.util.Collections$UnmodifiableSet)
>
> - locked (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <1f56a22> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-19" - Thread t@37
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked <10ae86a> (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-18" - Thread t@36
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked <120b2c4> (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:129)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-17" - Thread t@35
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked <1b8823c> (a sun.nio.ch.Util$2)
>
> - locked <6e7099> (a java.util.Collections$UnmodifiableSet)
>
> - locked (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <1469f65> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-16" - Thread t@33
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked <18ac785> (a sun.nio.ch.Util$2)
>
> - locked <3119a9> (a java.util.Collections$UnmodifiableSet)
>
> - locked <18bd37a> (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-15" - Thread t@31
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked <14eacca> (a sun.nio.ch.Util$2)
>
> - locked <321f2a> (a java.util.Collections$UnmodifiableSet)
>
> - locked <9c2daf> (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked <17b6caf> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-9" - Thread t@29
>
> java.lang.Thread.State: RUNNABLE
>
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
>
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
>
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
>
> - locked <1c8622> (a sun.nio.ch.Util$2)
>
> - locked (a java.util.Collections$UnmodifiableSet)
>
> - locked <1e3493f> (a sun.nio.ch.EPollSelectorImpl)
>
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
>
> at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:513)
>
> at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:185)
>
>
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:130)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - locked (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
>
>
> "Thread-12" - Thread t@28
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <833c98> (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked <1bbf1be> (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-10" - Thread t@26
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1ff5bf1> (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked <1fc0b7a> (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-8" - Thread t@24
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1ed3ca7> (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked <55b420> (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-6" - Thread t@21
>
> java.lang.Thread.State: TIMED_WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <28f9f4> (a java.util.concurrent.atomic.AtomicInteger)
>
> at
> com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1092)
>
>
> at com.sun.grizzly.Controller.start(Controller.java:843)
>
> at
> com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1241)
>
>
> - locked (a java.lang.Object)
>
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1087)
>
> at
> com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1168)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:131)
>
>
> at
> com.sun.enterprise.v3.services.impl.GrizzlyProxy$2.run(GrizzlyProxy.java:317)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-5" - Thread t@19
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <32293c> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-4" - Thread t@18
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1312d86> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-3" - Thread t@17
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1a3d4b8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-2" - Thread t@16
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
>
>
> at com.sun.btrace.BTraceRuntime$1.run(BTraceRuntime.java:317)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Thread-1" - Thread t@15
>
> java.lang.Thread.State: WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1c3b391> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>
> at
> com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:510)
>
>
> at
> com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:255)
>
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "pool-2-thread-1" - Thread t@14
>
> java.lang.Thread.State: TIMED_WAITING
>
> at sun.misc.Unsafe.park(Native Method)
>
> - parking to wait for <1421966> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
>
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
>
>
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
>
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "FelixPackageAdmin" - Thread t@13
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <91782f> (a org.apache.felix.framework.PackageAdminImpl)
>
> at java.lang.Object.wait(Object.java:485)
>
> at
> org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:314)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "FelixStartLevel" - Thread t@12
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1b273e7> (a java.util.ArrayList)
>
> at java.lang.Object.wait(Object.java:485)
>
> at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "FelixDispatchQueue" - Thread t@11
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <696349> (a java.util.ArrayList)
>
> at java.lang.Object.wait(Object.java:485)
>
> at
> org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:931)
>
>
> at
> org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
>
>
> at
> org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "DestroyJavaVM" - Thread t@9
>
> java.lang.Thread.State: RUNNABLE
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "OSGi Framework Launcher" - Thread t@8
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a org.apache.felix.framework.util.ThreadGate)
>
> at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)
>
> at org.apache.felix.framework.Felix.waitForStop(Felix.java:794)
>
> at org.apache.felix.main.Main.main(Main.java:299)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at
> com.sun.enterprise.glassfish.bootstrap.ASMainFelix$1.run(ASMainFelix.java:116)
>
>
> at java.lang.Thread.run(Thread.java:662)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Signal Dispatcher" - Thread t@4
>
> java.lang.Thread.State: RUNNABLE
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Finalizer" - Thread t@3
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <1baadd4> (a java.lang.ref.ReferenceQueue$Lock)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
>
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
>
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
> "Reference Handler" - Thread t@2
>
> java.lang.Thread.State: WAITING
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on (a java.lang.ref.Reference$Lock)
>
> at java.lang.Object.wait(Object.java:485)
>
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
>
>
>
> Locked ownable synchronizers:
>
> - None
>
>
>
>
>
>
> --
>
> [Message sent by forum member 'steveglennon']
>
> View Post: http://forums.java.net/node/853023
>
>

steveglennon
Offline
Joined: 2008-04-24

Alexey,

I am convinced that this is a thread safety issue in the Deflater.java. Others have come to the same conclusion:

http://stackoverflow.com/questions/4660819/java-util-zip-zipinputstream-...

Looking at the source to Deflater.java we can see that the native code has state (dictionary) and the deflateBytes method does not carry the context parameter (strm). As a result the deflateBytes can be executing while another thread (on another GZIPOutputStream object, duly synchronized) can and will change the dictionary mid-deflation.

Steve

steveglennon
Offline
Joined: 2008-04-24

Alexey,

Well I showed my naivety on JNI in the last post. Looking into at least some vintage of the C JNI code for that method it appears that the code is getting the "strm" context parameter by obtaining the variable from the JNIEnv parameter. So I can't find the smoking gun in the code (though I am not sure I am looking at the right code versions).

We were able to confirm that the symptoms re-occur (though much less frequently) with Glassfish compression turned off and CXF @GZIP compression turned on, so Glassfish is absolutely absolved of any blame. So we continue the chase against java.util.zip.GZIPOutputStream.

Thanks for looking into this.

Steve