Skip to main content

Session replication in GlassFish 3.1 - Amazon EC2 Instances

6 replies [Last post]
rubycube
Offline
Joined: 2009-03-01

I have two instances(i1,i2) of cluster(c1). i1(created using "localhost" node) is running on the DAS machine(amazon ec2) and i2(created using "node1" - SSH provisioning) is a remote instance(another amazon ec2).

Will DAS be able to replicate session across these two instances(i1 & i2) (between two amazon ec2 instances)?
If yes, is there any document that describes the concept of session replication in glassfish v3.1?
Do we need to configure glassfish admin console to start the session replication?
v2.1.1 had a "clusterjsp" sample application? Do we have a similar application in v3.1?
Whats the role of "availability service" and "Group Management Service" in session replication?

If we stop the default domain "domain1", will it bring down DAS? If yes, then how is the session replication handled between the clustered instances when "domain1" is down and both the instances are running?

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
shreedhar_ganapathy
Offline
Joined: 2007-01-17

Hi Dominik
This is a good tip - if you have any samples or links it will save time - than Googling around and stumbling.

Amazon WS has been collecting feedback from users for a couple of years to enable multicast within the VPC (Virtual Private Cloud).
As recent as April 2010, it seems to be on their plans but no telling how long it will be before it sees light of day.
https://developer.amazonwebservices.com/connect/thread.jspa?threadID=379...

Several service providers have been asking for multicast support for a while now.

rubycube
Offline
Joined: 2009-03-01

Hi Shreedhar,

Thanks for the details. To sum-up, we cannot achieve session replication between two amazon ec2 instances since EC2 doesn't provide multicasting and glassfish works on this transmission method. So I guess even the "command replication" described here (http://wikis.sun.com/display/glassfish/3.1MS1ClusteringDemo) should not work between these two instances. How else can we achieve session replication here? Did you mention "Coherence"?

Shreedhar Ganapathy

The command replication referred to here is a framework for admin
commands to sent over ssh to a remote note and executed on that node.
That uses Http protocol and that would continue to work for
administration commands.

In sum, save for other solutions (which we have not planned to support
for this release due to dev and test resource constraints) such as SSH +
VPN or VLAN setups et al, you could create cluster of instances and
front them with LB and get service availability but not session
availability.

We are looking to put non multicast based support on our roadmap.

Thanks
Shreedhar

On 10/20/10 6:17 AM, glassfish@javadesktop.org wrote:
> Hi Shreedhar,
>
> Thanks for the details. To sum-up, we cannot achieve session replication between two amazon ec2 instances since EC2 doesn't provide multicasting and glassfish works on this transmission method. So I guess even the "command replication" described here (http://wikis.sun.com/display/glassfish/3.1MS1ClusteringDemo) should not work between these two instances. How else can we achieve session replication here? Did you mention "Coherence"?
> [Message sent by forum member 'rubycube']
>
> http://forums.java.net/jive/thread.jspa?messageID=485724
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Shreedhar Ganapathy

Thanks for trying this out and getting back to us with questions.
I believe on EC2, multicast is not supported and one of our requirements
in 3.1 for runtime clustering is the support for multicast messaging.
Without this, the default session replication mechanism and runtime
clustering discovery and health monitoring protocols will not work.

One alternative is to consider Coherence but more on that in a later
email .... More responses below.

On 10/20/10 12:10 AM, glassfish@javadesktop.org wrote:
> I have two instances(i1,i2) of cluster(c1). i1(created using "localhost" node) is running on the DAS machine(amazon ec2) and i2(created using "node1" - SSH provisioning) is a remote instance(another amazon ec2).
>
> Will DAS be able to replicate session across these two instances(i1& i2) (between two amazon ec2 instances)?

DAS (Domain Admin Server) is responsible for administering and managing
the configuration, deployment, synchronization and lifecycle management
of the instances in the domain. It is not responsible for session
replication.

> If yes, is there any document that describes the concept of session replication in glassfish v3.1?
We are still in development and so are documents related to it. The one
pager for session replication is on the GlassFish wiki and could be a
useful starting point
http://wikis.sun.com/display/GlassFish/GlassFish3.1HAOnePager

> Do we need to configure glassfish admin console to start the session replication?
Yes (but without multicast ..nogo ). As you deploy the application, you
need to check the availability-enabled checkbox - this enables session
persistence with default persistence type set to Full Session (each time
the full session is replicated). You can modify the session persistence
type to Modified Session or Modified Attribute in the Availability
Service screens.
> v2.1.1 had a "clusterjsp" sample application? Do we have a similar application in v3.1?
It will be - but the v2.1.1 clusterjsp sample should work for you (on a
multicast supported setup) with 3.1
> Whats the role of "availability service" and "Group Management Service" in session replication?
Availability service is the configuration that tells the session
replication (aka Shoal Cache) module to initialize support for session
persistence for a given App deployed to a cluster.

Group Management Service is a runtime clustering system (aka Shoal GMS)
in the form of a module in each clustered instance that the Session
Replication Module leverages to get notifications on cluster member
lifecycle events such as members joins, joined and ready, failed,
stopped etc. allowing the session replication module to dynamically
react to cluster shape changes and select replica nodes for each session
in a given instance. GMS module uses a combination of TCP and UDP over
multicast for messaging - for cluster member discovery and member health
monitoring it relies on UDP over multicast.

The session replication module picks a replica for each session
consistently and as a result (almost) evenly spreads sessions from a
primary instance to a different instance in the cluster.

> If we stop the default domain "[b]domain1[/b]", will it bring down DAS?
Each domain has a DAS to administer the domain. So stop domain is
synonymous with stopping the DAS. So yes you will not have an
administration server running if DAS is brought down. However, it does
not affect the running cluster within a domain, save for the ability( or
lack thereof) to run configurational actions.

> If yes, then how is the session replication handled between the clustered instances when "domain1" is down and both the instances are running?

Session replication is enabled only between clustered instances for a
given application - the DAS is not involved in session replication.

On startup, replication module instantiates a Backing Store for each HA
enabled Application.
Post instance bootstrap, the GMS module in each clustered instance
notifies replication module's BackingStore in each instance of any
cluster view changes through its discovery and health monitoring
protocols. As requests come into the containers from clients through an
LB, the sessions are persisted by the containers at the end of the
method (or optionally end of transaction in the EJB container) by
calling the save() method of the HA SPI which results in Shoal Cache ( a
Backing Store SPI implementation) picking a replica instance for that
session using a hashing algorithm. If the primary instance fails, the LB
would failover to another instance in the cluster to resume the session
- if this failed over instance is the replica instance for the session,
then the session is retrieved from the replica cache. If not the
replica instance, the Shoal Cache's Key Mapper hashing algorithm
provides the replication code the exact location of where the session is
located in a replica instance in the cluster, and a unicast load command
is sent out through GMS messaging APIs to that instance to retrieve the
session so that it can be resumed on the failed over instance.

If the replica instance does not have this request session data, then a
broadcast (multicast message) is sent out through GMS messaging to the
cluster to retrieve the session if available from the remaining instances.

Hope the above is helpful info. Feel free to ask more.
Cheers
Shreedhar

> [Message sent by forum member 'rubycube']
>
> http://forums.java.net/jive/thread.jspa?messageID=485667
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
> For additional commands, e-mail: users-help@glassfish.dev.java.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

Dominik Dorn

On Wed, Oct 20, 2010 at 9:32 AM, Shreedhar Ganapathy
wrote:
>  Thanks for trying this out and getting back to us with questions.
> I believe on EC2, multicast is not supported and one of our requirements in
> 3.1 for runtime clustering is the support for multicast messaging. Without
> this, the default session replication mechanism and runtime clustering
> discovery and health monitoring protocols will not work.

You could setup a VPN between your EC2 nodes, that way multicast should work.
You can also do that through ssh alone (no need for openvpn etc.).

just search for SSH VPN, there are some good tutorials.

All the best,
dominik
--
Dominik Dorn
http://dominikdorn.com
http://twitter.com/domdorn

Tausche Deine Lernunterlagen auf http://www.studyguru.eu !

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@glassfish.dev.java.net
For additional commands, e-mail: users-help@glassfish.dev.java.net

rubycube
Offline
Joined: 2009-03-01

I really appreciate your help. Could you please elaborate this idea? What would be the disadvantage? Will setting up the VPN between these two EC2 instances be good enough for multicasting?