Skip to main content

loadbalancer with apache

8 replies [Last post]
peppeme
Offline
Joined: 2007-05-30

I wish to setup my glassfish cluster with an apache loadbalancer.

In order to make it I wish to use the lbplug in.

so I follow this blog
http://blogs.sun.com/sprabhu7/entry/install_configure_lb_on_apache#4

But I can't start Apache because it not work with mod_loadbalancer.dll (I try also in a linux system with mod_loadbalancer.so).

Apache tell me Cannot load
C:/Program Files/Apache Group/Apache2/modules/mod_loadbalancer.dll

I use http://download.java.net/javaee5/external/WINNT/aslb/jars/aslb-9.1.1-b2.jar version of plugin.

In linux apache tell me wrong elf class about mod_loadbalancer.so loading

I tried with apache 2.0.63 and apache 2.2

What's the problem?

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
wc2006
Offline
Joined: 2007-02-20

Hello

I also tried to do load-balancing with Apache.
I followed these steps:
http://blogs.sun.com/jluehe/entry/supporting_apache_loadbalancer_with_gl...

Load-balancing is ok but failover is not. When one machines go down, a new session is created in the other machine instead of using the old session.

Did anyone come across this?

Thank you!

Wilfred

Kshitiz Saxena

Hi,

Please see my comments inline. I hope you are using Apache 2.0.x as
Apache 2.2.x will have different configuration steps.

Thanks,
Kshitiz

glassfish@javadesktop.org wrote:
> I wish to setup my glassfish cluster with an apache loadbalancer.
>
> In order to make it I wish to use the lbplug in.
>
> so I follow this blog
> http://blogs.sun.com/sprabhu7/entry/install_configure_lb_on_apache#4
>
> But I can't start Apache because it not work with mod_loadbalancer.dll (I try also in a linux system with mod_loadbalancer.so).
>
> Apache tell me Cannot load
> C:/Program Files/Apache Group/Apache2/modules/mod_loadbalancer.dll
>

Mostly it will happen when all dependencies are not resolved correctly.
You can use utility like dependency-walker to see if they are all
resolved correctly.

> I use http://download.java.net/javaee5/external/WINNT/aslb/jars/aslb-9.1.1-b2.jar version of plugin.
>
> In linux apache tell me wrong elf class about mod_loadbalancer.so loading
>

You are using 64 bit version of Apache. The load-balancer plugin library
is 32 bit version so ensure you use 32 bit version of Apache as well.
Also not all Linux platform are supported.

> I tried with apache 2.0.63 and apache 2.2
>

As I mentioned configuration steps are different for 2.0.x and 2.2.x.
Please use only Apache 2.0.x.

You can refer to configuration steps available @
http://docs.sun.com/app/docs/doc/819-3679/abdhg?l=en&a=view

> What's the problem?
> [Message sent by forum member 'peppeme' (peppeme)]
>
> http://forums.java.net/jive/thread.jspa?messageID=301020
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

peppeme
Offline
Joined: 2007-05-30

I solve by using Apache 2.0.63 following the guide in the link you have write in the post.
I using dependecy walker, so I copy in modules directory all the dependecy.

Now It works, but about half http request on apache balancer give me the mo_balancer error page.

My cluster have 2 node each with 1 instance.
Every node-agent and instance is running.
I'm try with clusterjsp example application, it works only 50% request.
When it works some time the response is from instance1, some time from instance2 and some time error page.

Why?

Kshitiz Saxena

Please provide error_log file to debug this issue. Whenever a request
fails, there will be logs to indicate the cause of failure. You can turn
on debug logs, by setting log-level to debug, to provide more details.

Thanks,
Kshitiz

glassfish@javadesktop.org wrote:
> I solve by using Apache 2.0.63 following the guide in the link you have write in the post.
> I using dependecy walker, so I copy in modules directory all the dependecy.
>
> Now It works, but about half http request on apache balancer give me the mo_balancer error page.
>
> My cluster have 2 node each with 1 instance.
> Every node-agent and instance is running.
> I'm try with clusterjsp example application, it works only 50% request.
> When it works some time the response is from instance1, some time from instance2 and some time error page.
>
> Why?
> [Message sent by forum member 'peppeme' (peppeme)]
>
> http://forums.java.net/jive/thread.jspa?messageID=301306
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

peppeme
Offline
Joined: 2007-05-30

> You can turn on debug logs, by setting log-level to debug, to provide more details.
Do you referer to apache error-log?
However I collect more data about error:

The cluster with 2 node agent and instances are started.

I attach my loadbalancer.xml

When I start Apache its log entries are:

[notice] Initializing lbplugin BuildId: A701212-164111
[notice] Initializing lbplugin BuildId: A701212-164111
[notice] Apache/2.0.63 (Win32) mod_ssl/2.0.63 OpenSSL/0.9.7m configured -- resuming normal operations
[notice] Server built: Jan 17 2008 22:58:29
[notice] Parent: Created child process 5028
[notice] Initializing lbplugin BuildId: A701212-164111
[notice] Initializing lbplugin BuildId: A701212-164111
[notice] Child 5028: Child process is running
[notice] Child 5028: Acquired the start mutex.
[notice] Child 5028: Starting 250 worker threads.

It's seems ok.
First call on loadbalancer (to apache hostname)

[alert] --------------------Name Trans---------
[alert] apache process id = 5028 ; URI = /clusterjsp
[alert] Within apachelbplugin_name_trans()
[warn] lb.runtime: RNTM2019: Daemon http://mobipc:38080 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://mobipc:38181 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon http://MOBISERVER:38081 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://MOBISERVER:38182 has been intialized.
[crit] lb.runtime: RNTM2003:After LBDaemonManager::init()
[alert] This request is INSECURE...
[crit] lb.runtime: RNTM2005: after LBApacheProxyRequest::LBApacheProxyRequest()
[alert] This request is for APACHE LB Plugin...
[alert] -----------------Request/Response handler---------
[alert] apache process id = 5028 ; URI = /clusterjsp
[crit] lb.runtime: LBRT1002: Executing Runtime method mod_apachelbplugin.cpp:apachelbplugin_handle_req
[crit] lb.runtime: RNTM2017:Header Name = Accept ====> Value = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*
[crit] lb.runtime: RNTM2017:Header Name = Accept-Language ====> Value = en,it;q=0.5
[crit] lb.runtime: RNTM2017:Header Name = Accept-Encoding ====> Value = gzip, deflate
[crit] lb.runtime: RNTM2017:Header Name = User-Agent ====> Value = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
[crit] lb.runtime: RNTM2017:Header Name = Host ====> Value = localhost:85
[crit] lb.runtime: RNTM2017:Header Name = Connection ====> Value = Keep-Alive
[crit] lb.runtime: RNTM2017:Header Name = Cookie ====> Value = JSESSIONID=897d02d1291d49ba3929d787305e; JSESSIONIDVERSION=/clusterjsp:0
[alert] response code ....= 302
[alert] request headers ....= accept="image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*" accept-Language="en,it;q=0.5" accept-Encoding="gzip, deflate" user-Agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)" host="localhost:85" cookie="JSESSIONID=897d02d1291d49ba3929d787305e; JSESSIONIDVERSION=/clusterjsp:0"
[alert] response headers ....= x-powered-by="Servlet/2.5" location="http://localhost:85/clusterjsp/" content-type="text/html; charset=iso-8859-1" date="Thu, 25 Sep 2008 08:26:57 GMT"
[alert] Sending Response Headers to Client ....\n\n
[crit] lb.runtime: RNTM2017:Header Name = X-powered-by ====> Value = Servlet/2.5
[crit] lb.runtime: RNTM2017:Header Name = Location ====> Value = http://localhost:85/clusterjsp/
[crit] lb.runtime: RNTM2017:Header Name = Content-type ====> Value = text/html; charset=iso-8859-1
[crit] lb.runtime: RNTM2017:Header Name = Date ====> Value = Thu, 25 Sep 2008 08:26:57 GMT
[crit] lb.runtime: RNTM2003:After LBDaemonManager::serviceRequest()
[crit] lb.runtime: RNTM2007:Request Succeeded
[alert] --------------------Name Trans---------
[alert] apache process id = 5028 ; URI = /clusterjsp/
[alert] Within apachelbplugin_name_trans()
[warn] lb.runtime: RNTM2019: Daemon http://mobipc:38080 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://mobipc:38181 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon http://MOBISERVER:38081 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://MOBISERVER:38182 has been intialized.
[crit] lb.runtime: RNTM2003:After LBDaemonManager::init()
[alert] This request is INSECURE...
[crit] lb.runtime: RNTM2005: after LBApacheProxyRequest::LBApacheProxyRequest()
[alert] This request is for APACHE LB Plugin...
[alert] -----------------Request/Response handler---------
[alert] apache process id = 5028 ; URI = /clusterjsp/
[crit] lb.runtime: LBRT1002: Executing Runtime method mod_apachelbplugin.cpp:apachelbplugin_handle_req
[crit] lb.runtime: RNTM2017:Header Name = Accept ====> Value = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*
[crit] lb.runtime: RNTM2017:Header Name = Accept-Language ====> Value = en,it;q=0.5
[crit] lb.runtime: RNTM2017:Header Name = Accept-Encoding ====> Value = gzip, deflate
[crit] lb.runtime: RNTM2017:Header Name = User-Agent ====> Value = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
[crit] lb.runtime: RNTM2017:Header Name = Host ====> Value = localhost:85
[crit] lb.runtime: RNTM2017:Header Name = Connection ====> Value = Keep-Alive
[crit] lb.runtime: RNTM2017:Header Name = Cookie ====> Value = JSESSIONID=897d02d1291d49ba3929d787305e; JSESSIONIDVERSION=/clusterjsp:0
[alert] response code ....= 200
[alert] Sending Response Headers to Client ....\n\n
[crit] lb.runtime: RNTM2017:Header Name = X-powered-by ====> Value = JSP/2.1
[crit] lb.runtime: RNTM2017:Header Name = Content-type ====> Value = text/html;charset=ISO-8859-1
[crit] lb.runtime: RNTM2017:Header Name = Content-length ====> Value = 1584
[crit] lb.runtime: RNTM2017:Header Name = Date ====> Value = Thu, 25 Sep 2008 08:26:57 GMT
[crit] lb.runtime: RNTM2017:Header Name = Set-cookie ====> Value = JSESSIONIDVERSION=/clusterjsp:1; Path=/clusterjsp
[crit] lb.runtime: RNTM2003:After LBDaemonManager::serviceRequest()
[crit] lb.runtime: RNTM2007:Request Succeeded

It's seems ok.
Then I Strees(only few thread) apache with jmeter and many request make the error.

Now In the apache error.log continuously appear this entries:
(p.s. MOBISERVER and mobipc are the nodes hostname)

[warn] lb.runtime: RNTM2024: Daemon http://MOBISERVER:38081 is unhealthy.
[warn] lb.runtime: RNTM2030: Daemon Monitor : http://MOBISERVER:38081 : could be because daemon is down
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy.
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy.
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy.
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is detected to be still unHealthy in cluster: cluster
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is detected to be still unHealthy in cluster: cluster
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is detected to be still unHealthy in cluster: cluster
[warn] lb.healthchecker: HLCK3003: Listener: http://mobipc:38080 is detected to be still unHealthy in cluster: cluster
[warn] lb.healthchecker: HLCK3003: Listener: http://mobipc:38080 is detected to be still unHealthy in cluster: cluster
..
[warn] lb.runtime: RNTM2024: Daemon http://mobipc:38080 is unhealthy.
[warn] lb.runtime: RNTM2030: Daemon Monitor : http://mobipc:38080 : could be because daemon is down
[warn] lb.runtime: RNTM2024: Daemon http://mobipc:38080 is unhealthy.
[warn] lb.runtime: RNTM2030: Daemon Monitor : http://mobipc:38080 : could be because daemon is down
[warn] lb.runtime: RNTM2024: Daemon http://MOBISERVER:38081 is unhealthy.
[warn] lb.runtime: RNTM2030: Daemon Monitor : http://MOBISERVER:38081 : could be because daemon is down
[warn] lb.runtime: RNTM2025: Daemon http://mobipc:38080 is healthy.
[warn] lb.healthchecker: HLCK3003: Listener: http://MOBISERVER:38081 is detected to be still unHealthy in cluster: cluster
..

When I call the page by loadbalancer, both when error appear and not appear in glassfish instance server.log I read this entry:

- Log Level:INFO
- Logger: javax.enterprise.system.stream.out
- Name-Value Pairs: _ThreadID=22;_ThreadName=httpSSLWorkerThread-38080-4;
- Record Number: 3750
- Message ID
- Complete Message: No parameter entered for this request

I think that the mod_balancer detect the server sometime healty and some time unhealty, so when it detect unHealthy the error appear.
But I can't understand this because the instance are always running.

Moreover with jmeter test I notice performance downgrade by using loadbalancer, the single cluster instance is better .

kshitiz_saxena
Offline
Joined: 2006-05-03

Have you set values of the parameters StartServers and maxclients to 1 under section prefork MPM of http.conf?

Also have you specified correct machine ip address for Virtual Host?

From logs I can see initialization happening for each request. Below logs indicate that :

[alert] Within apachelbplugin_name_trans()
[warn] lb.runtime: RNTM2019: Daemon http://mobipc:38080 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://mobipc:38181 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon http://MOBISERVER:38081 has been intialized.
[warn] lb.runtime: RNTM2019: Daemon https://MOBISERVER:38182 has been intialized.
[crit] lb.runtime: RNTM2003:After LBDaemonManager::init()

This should not happen. Initialization should happen only once and then for any request, apache should be able to lookup an existing virtual server and use it to service request.

Attach your httpd.conf, if even after suggested changes it does not work.

Also from logs I can see one instance is detected to be unhealthy. May be under loaded condition, load-balancer plugin in not able to establish connection to application-server instance and thus marking it as unhealthy. Please increase the load on system gradually and check.

Thanks,
Kshitiz

peppeme
Offline
Joined: 2007-05-30

Thanks for the help.
The problem was solved by correct httpd.conf
I had change the http port from 80 to 85, but the ServerName variable was still "localhost:80"

By setting "ServerName localhost:85" it works.

But I read(in http://docs.sun.com/app/docs/doc/819-3679/abdhg?l=en&a=view and in your reply) about prefork MPM section, StartServer and MaxChild settings in order to set sticky round robin.

I search in my httpd.conf about prefork MPM section but I didn't find it.

I attach in the post httpd.conf

I find

# WinNT MPM
# ThreadsPerChild: constant number of worker threads in the server process
# MaxRequestsPerChild: maximum number of requests a server process serves

ThreadsPerChild 250
MaxRequestsPerChild 0

But If I put inside IfModule the StartServers command it isn't recognize as a valid command.

Where I put this command?
I must write anything like

StartServers 1
MaxClients 1

What are the recommended settings in httpd.conf for using apache only in loadbalancing?

kshitiz_saxena
Offline
Joined: 2006-05-03

There are issues with prefork MPM on windows, so I believe by default windows distribution comes with WinNT MPM. Since it is single process, you don't need any other setting. For more details on it please refer to apache documentation site.

Please try session-less requests and sticky- requests. In case you face any issues, please get back to us.

Thanks,
Kshitiz