Skip to main content

Streaming over JXTA

7 replies [Last post]
iliasr
Offline
Joined: 2009-04-09

Hello,
I was trying to implement a streaming mechanism over a JXTA network, with the platform's transport methods (pipes and sockets)... Using JxtaSocket I came to find that the performance of the streaming was much less than acceptable. I then replaced the sockets with the purer JxtaBiDiPipe, but neither this method proved efficient... Is such a thing as streaming over jxta with its own transport methods possible, or have I to implement it using the plain Socket API of Java?

I came across a couple of jxta projects that implement streaming (vop2p, mytvjxta) but could not find any source or documentation anywhere...

Any information that you might have could prove useful!
Thanks in advance!

P.S. Please excuse my english!

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
turbogeek
Offline
Joined: 2003-06-10

Part of the problem is that Jxta does no blocking. It is a queue-based system. You can't label a message with priority or somehow ensure buffering of a specific size.

Because a Relay is often involved, you also don't know how other traffic is affecting you other than direct evidence on the receiving end. You can also loose data if the queues are overloaded.

My suggestion too is that a specific service be created. I'd especially recommend your own endpoint implementation that can manage streaming traffic. Nobody has really done this yet because it is a lot of work.

People have done streaming, it just isn't very consistent. My efforts with file transfers almost always add a handshaking mechanism to throttle data based on feedback. I also boost the size of message queues to prevent loosing data. It is messy, but it works. The enhancements to threading and performance will help, but there are certain limits to what you can do with the current design that require management at the application level until we create more endpoint protocols and ways to ensure pipes can be connected to just those endpoints.

One thing to realize is that this is like a three body physics problem. It is tough to figure out the impact of three CPU and two communication paths. Any of these can be slow and cause a problem in the overall performance. What you need is a way to throttle and intelligently buffer information at the correct point of a specific configuration and load balance on top of that.

boylejohnr
Offline
Joined: 2008-10-27

It is possible, however I would think you need think careuly about the size of message and sending. Since as JXTA is message based it has a message overhead. If you only send one byte in every message that would not be good. However, as you increase the size of the buffer it has other dertrimental effects to your performance. i.e. Latency between bytes of info.

Suggest you tinker with buffering of your messages and see if you can hit the sweet spot.

Otherwise, we can think about how to extend JXTA to support Streaming more efficiently.

As a company we are generally improving JXTA performance and if you search mails from Iain McGinness you will see some of the results.

iliasr
Offline
Joined: 2009-04-09

Thank you for your answer.

I am aware of the message overhead, and i think that it comes up to 50bytes per message... I wonder, is this overhead enough to have a serious impact on the throughput of the transmission? In addition, I read that the messages that JXTA uses have a size limit of 64kbytes... From this I understand that the most suitable choice of the message length is the limit, because in this way the total messages flowing in the network will be minimized...

One last thing, you are talking about messages, so I assume that you suggest using Pipes for the streaming instead of Sockets?

boylejohnr
Offline
Joined: 2008-10-27

Serious impact depends, would need to profile. There is an overhead of doing the unpack repack handing work at each node. So more detail would help tun. Yes 64K is the total MTU (not just payload). And this is soft, ie.. if You are on direct TCP then can be larger, however relays are only obliged to handle 64K messages.

JXTA is all messages, sockets sit on top of messages/pipes.

iliasr
Offline
Joined: 2009-04-09

The thing is that when reading a file from a regular java socket/stream, throughput rises as much as 10 or 11 mbps. The chunks of the file read are immediately written to the jxta socket/pipe and passed on through the p2p. The throughput here drops at 8 or 9 mbps. This is the impact i observe. Can this be eliminated just by tuning the handling of the messages?

boylejohnr
Offline
Joined: 2008-10-27

I would think improved, but can not speak from practicle experience here.

Thinking about how JXTA works it may be possible to write a service to form streams however this would be a large effort. Where we have specific components that will form deticated connection streaming. However, goes against the grain.

we are working on performance improvements that will be ready shortly and specifically looking at some of the message handling, we have already improved thread utilisation and some of the message handling. Refer to Iain McGinness work. XML handling is excessive and not the fastest implementation. But modular enough to update without too much pain.

iliasr
Offline
Joined: 2009-04-09

I see.

boylejohnr thank you for your time and your willingness. I will take all these matters into consideration and think about how will I approach the issues. If I have tangible results, I will post them here.

Thanks again,
ilias