Skip to main content

JMS Connection Refused using Front End Cluster to Single Instance Back End GF4

No replies
theshowmecanuck
Offline
Joined: 2008-01-31
Points: 0

Files now attached... they didn't upload on the first try.

I have been asked to help 'clusterize' a deployment and can't deploy our app due to JMS issues. Please have some patience I am new to clustering and know jms from a dev viewpoint. But I am trying to include complete info as efficiently as possible.

I keep getting the following errors on our nodes when deploying from the central DAS:

[2013-09-24T22:33:14.821+0000] [glassfish 4.0] [WARNING] [] [javax.jms] [tid: _ThreadID=34 _ThreadName=admin-listener(3)] [timeMillis: 1380061994821] [levelValue: 900] [[
[C4003]: Error occurred on connection creation [staging_jms_backend.myserver.com:27676]. - cause: java.net.ConnectException: Connection refused]]

[2013-09-24T22:33:19.823+0000] [glassfish 4.0] [WARNING] [] [javax.jms] [tid: _ThreadID=34 _ThreadName=admin-listener(3)] [timeMillis: 1380061999823] [levelValue: 900] [[
[C4003]: Error occurred on connection creation [staging_jms_backend.myserver.com:27676]. - cause: java.net.ConnectException: Connection refused]]

[2013-09-24T22:33:19.827+0000] [glassfish 4.0] [SEVERE] [] [] [tid: _ThreadID=34 _ThreadName=Thread-4] [timeMillis: 1380061999827] [levelValue: 1000] [[
com.sun.messaging.jms.JMSException: [C4003]: Error occurred on connection creation [staging_jms_backend.myserver.com:27676]. - cause: java.net.ConnectException: Connection refused
at com.sun.messaging.jmq.jmsclient.ExceptionHandler.throwConnectionException(ExceptionHandler.java:280)
at com.sun.messaging.jmq.jmsclient.ExceptionHandler.handleConnectException(ExceptionHandler.java:226)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.readBrokerPorts(PortMapperClient.java:278)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.init(PortMapperClient.java:156)
at com.sun.messaging.jmq.jmsclient.PortMapperClient.(PortMapperClient.java:98)
at com.sun.messaging.jmq.jmsclient.protocol.tcp.TCPConnectionHandler.(TCPConnectionHandler.java:168)
at com.sun.messaging.jmq.jmsclient.protocol.tcp.TCPStreamHandler.openConnection(TCPStreamHandler.java:141)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:785)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnectionNew(ConnectionInitiator.java:260)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:214)
at com.sun.messaging.jmq.jmsclient.ConnectionInitiator.createConnection(ConnectionInitiator.java:164)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.init(ProtocolHandler.java:874)
at com.sun.messaging.jmq.jmsclient.ProtocolHandler.(ProtocolHandler.java:1597)
at com.sun.messaging.jmq.jmsclient.ConnectionImpl.openConnection(ConnectionImpl.java:2470)

(I have attached a more complete log snippet).

I am trying to set up two servers as 'front end' servers in a cluster (glassfish jms client servers). There is a separate server used as a DAS. i.e. One controller and two nodes all on separate Amazon AWS instances.

The two servers use MDBs to get information from several Topics on a singe REMOTE back end server that collects data from external services. So the front end clustered servers need to connect using JMS to the back end server.

The topics have been created by the devs in glassfish-resources.xml. No ports were configured.

I set up the config on the glassfish clients to 'hit' the remote backend server on port 27676 but when I tried to ping from the jms connector pool in the DAS it failed.
When I set up them up to hit 7676 it worked. And I can see this when monitoring the two nodes' server.log
Then I try to deploy the app and for some reason the glassfish clients want to try to connect to 27676. Then deploy fails with the messages above.

When I telnet the backend I see the status of jms I see this:

ubuntu@ip-10-0-0-126:/opt/glassfish/glassfish/domains/domain1/logs$ telnet 10.0.0.126 27676
Trying 10.0.0.126...
telnet: Unable to connect to remote host: Connection refused

ubuntu@ip-10-0-0-126:/opt/glassfish/glassfish/domains/domain1/logs$ telnet 10.0.0.126 7676
Trying 10.0.0.126...
Connected to 10.0.0.126.
Escape character is '^]'.
101 imqbroker 301
portmapper tcp PORTMAPPER 7676 [imqvarhome=/opt/glassfish4/glassfish/domains/domain1/imq,imqhome=/opt/glassfish4/mq,sessionid=4592899014103577344]
cluster_discovery tcp CLUSTER_DISCOVERY 0
jmxrmi rmi JMX 0 [url=service:jmx:rmi://ip-10-0-0-126/jndi/rmi://ip-10-0-0-126.ec2.internal:7776/ip-10-0-0-126.ec2.internal/7676/jmxrmi]
admin tcp ADMIN 46048
jms tcp NORMAL 33603
cluster tcp CLUSTER 42126
.
Connection closed by foreign host.

The 10... address is the backend server, and I get this response if I run it on this on the backend or the front end.

So I have to conclude the front end is not trying to connect correctly.... it is trying to choose port 27676 when the back end is clearly only set up to receive jms connections on 7676. Even remotely.

I'll attach the domain.xml for the front and back ends as well. I inherited this and hope someone can help me out. :)

AttachmentSize
serverlog.txt65.19 KB
backend_domainxml.txt37.12 KB
frontend_domainxml.txt25.12 KB