Skip to main content

How I can make discovery servcie for returning comlete results?

8 replies [Last post]
sergeya
Offline
Joined: 2007-08-17

Hi all, can somebody help understanding discovery service.

Right now I have set up about 10 computers as edge peer. And one rendezvous peer.
All peer names are stated from same prefix. All of edge use only my RDV, and publish information on it.

After
discoService.getRemoteAdvertisements(null, DiscoveryService.PEER,

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
thenetworker
Offline
Joined: 2003-06-13

I revisited the discovery service and read the blog "Discovery to Exhaustion", which describes the key concept of "some results" as a design goal. But I would appreciate it more if you could elaborate on it a little bit at the source code level, especially the block starting at "// no srdi, not a rendezvous, start the walk" in the DiscoveryServiceImpl.java.

When Srdi/SrdiIndex is involved, on the contrary to what you described, the discovery service tends to conduct an "exhaustion" search within the scope of the SrdiIndex. The search result (containing addressing information of the peers that published the advertisements whose indexes are in the SrdiIndex) is used to send a query one by one to _all_ those peers, when getRemoteAdvertisements is invoked. Then the resolver service will process every single response as a result of the query. Eventually, the responses will be saved/published to the local cache as discovered advertisements, which will be used to feed the method getLocalAdvertisements.

In the previous posting, the statement was made, "the discovery service will stop propagation of queries if a single match is made". I have not found the evidence in the source code that supports that. Could someone shed some light on that in the source code?

For those who are not interested in the source code, here is how the discovery service is supposed to work, (taking a typical scenario as an example).

A peer does a local publish() of an advertisement into its own cache. The index of the advertisement will be synchronized to the rendezvous in terms of Srdi mechanism. Other peers pointing to the same rendezvous will eventually get the index of the advertisement automatically when they periodically check the rendezvous. When they want to get the actual advertisement, they do a getRemoteAdvertisements first, which, as described above, sends a query to the peer. After a while, the peer's response will come to their local cache. Then they can do a getLocalAdvertisements, which will return the advertisement. If 1000 peers are doing the same thing, each of them should eventually get all the advertisements published by the others.

bondolo
Offline
Joined: 2003-06-11

JXTA Discovery is not designed to provide complete results of the type you are looking for. I wrote an entry on my blog exactly about this topic : Discovery to Exhaustion.

If you must find all of the peers then you need to use a more direct presence mechanism such as a shared propagate pipe to enumerate all active peers.

hamada
Offline
Joined: 2003-06-12

Discovery is the wrong service to use for this purpose.

How long does take to discover all nodes of interest?
How often do you invoke discovery?

It is not efficient nor sufficient for the task.

Use a propagated pipe to announce and discover application participants, this method is probably ok for scale of 100 or so, larger if used wisely.

hamada
Offline
Joined: 2003-06-12

By design, the discovery service will stop propagation of queries if a single match is made, in addition each responding nodes imposes a response threshold, both design principals are scalability related, as well as prevention of a self inflected DoS, the most common question we get about discovery is,

how can one enumerate all node in group? The most useful outcome of that would be to tabulate the network population, and such kinds of computations, it is best done on the infrastructure nodes, rather than on the edges, imagine a 100,000 element response, or worse, a 100K responses to the edge.

As for your observation, prior to issuing the query, inspect the rendezvous' local cache, and compare that to the results you receive.

anegri
Offline
Joined: 2007-07-26

You won't be able to predict how many calls to make to the getRemoteAdvertisement to find all the peers. But what you can do is create a threaded class and run the getRemoteAdvertisement in a separate thread asynchronously. This should show all the 10 peers over time, might take a up to a minute... but if you think about it is the same case with finding peers on games such as UT or what not. It takes a bit more time with JXTA... but you don't need a central server. If you rather have the later I suggest you look into JMS.

thenetworker
Offline
Joined: 2003-06-13

There are many reasons why you don't get all advs.

But here are some things you can do to troubleshoot that.

Do you receive the same four peers every time or any four of ten randomly? If the former is true, please double check if those six are configured correctly or not.

Also, you should use getLocalAdvertisement. The peers will resync with RDV automatically.

To avoid confusion, you should test one protocol at a time. After you are fully confident with both protocols (I mean, Http and Tcp transports), you can turn both of them on if you want.

sergeya
Offline
Joined: 2007-08-17

1. I'm using tcp ip protocol.
2. for sure I'm checking local adv before and if I didn't find peer name in local, I tries to get remote adv.
3. If I'll make more than one request some time I'll got all peers, but I can say how many time I should call remote adv before I'll got all peers.

do you know how I can enforce getting adv from rdv peer and save it on local peer
(for getting their as getLocalAdvertisement).

thenetworker
Offline
Joined: 2003-06-13

Perhaps the loss of messages during the multicasting caused the no-show if you are sure all the machines were running when that happened.

By the way, you don't get advs from RDV. Instead, you only get the index from it, then based on the index information, you go to get the advs from the individual peers. So, you can not enforce "getting adv from rdv". Nor should you. In a P2P environment, you should not assume you will get all the responses from all the peers. There may be a catastrophe, a network disruption, power outage, or peers simply choosing not to reply your requests. All those factors could cause the loss of information. So, "getting everything to start working" can not be built into the platform. Your apps should take the nature of P2P into consideration.

That being said, given the fact that you have only a small number of peers, you should have received all the responses upon even only one request. I think perhaps there is a design flaw in the TCP Transport. But I think your application should be designed in a way it just makes multiple requests if it has to. Think about this. Even if the TCP Transport is improved to give you all the advs from your 10 peers, it will fail to get all advs if someone has 10 million peers.