Skip to main content

OpenMQ JMS backlog issue

Please note these java.net forums are being decommissioned and use the new and improved forums at https://community.oracle.com/community/java.
No replies
NoviceUser
Offline
Joined: 2013-12-11

We are facing high message backlog issue occasionally with our JMS setup.
We have observed that at times, the message backlog seems to be shooting up from 5k to 300k in ~ 1 hr and the application starts showing inconsistent information.
When the JMS broker is bounced, the backlog goes back to normal. This issue happens infrequently and we have not been able to identify any patterns.
If the JMS broker is not bounced, the system would recover on its own - this takes anywhere from 1 to 20 hrs. No information from the JMS/java logs either.
Given below is the information about our JMS setup and machine configurations. Rest all configurations are set to default.

JMS Setup:-
----------------
We have 4 JMS s1, s2, s3 and s4 running. s1 and s2 are clustered and s3 and s4 are clustered.
3500 producers produce 1 message every 30 secs on a JMS topic and 8 asynchronous listeners.
We have used Spring JMS framwork for implementing the producers.

Linux Machine configurations:-
--------------------------------------
Total memory = 70GB

Broker Configurations:-
-----------------------------
imq.jms.max_threads=20000
imq.autocreate.destination.maxNumMsgs=300000
imq.autocreate.destination.limitBehavior=REJECT_NEWEST

Message Listener Configuration:-
----------------------------------------
message.listener.max.concurrent.consumers=5
message.listener.receive.timeout=1000
message.listener.idle.task.execution.limit=1
acknowledge mode= Auto
subscriptionDurable= No

JMS template
-------------------------
Type: org.springbyexample.jms.OpenMQConnectionFactoryBean
deliveryPersistent: false
deliveryMode: NON_PERSISTENT
timeToLive: 60000ms

VM args
----------
-Xms4g -Xmx10g -Xss256k -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=60

Message Information
---------------------------
Type= Object