The Source for Java Technology Collaboration
User: Password:
Register | Login help    

Search

Online Books:
java.net on MarkMail:


Alois Cochard

AloÏs Cochard is a developer in a business software company. He started learning Java at 12 and spends his free time to develop network security tools and messaging software. During his study he explored GUI development creating an open-source alternate UI for windows (developed with an L5G language allowed for focusing on the GUI). His current project is UniNet, an open source project created to achieve a freedom decentralized area used by people to share data and CPU power.

 

Alois Cochard's blog

Networking: Ensure data integrity in a decentralized storage system

Posted by alois on January 27, 2009 at 12:57 AM PST
Introduction
I'm actually developing on my free time a decentralized computing platform in java, I'm in the early stage, defining concepts and building the foundation.
This platform is target to be available to anyone connected to internet, so data integrity must be granted to avoid corruption of data from someone else, I gonna describe the concept I created to avoid any corruption of data in a high decentralized storage system.

Definition
The decentralized platform consist of (highly simplified):
  • Network service to handle data request (from remote peer or from local clients)
  • Storage service to store data and data checksum
  • Communication service to handle request/respond between peers
The source identifier, is the peer identifier of the peer who provided the data.

Integrity checking scenario
Network service receive data request:
  • ask storage if data exist (using data id)
    • if not, request data from other peers
    • save data inside storage with source identifier
  • compute data checksum
  • request data checksum from other peers
    • if data cheksum come from same peer as data, request data cheksum from other peer
  • compare both data checksums
    • if checksums are different:
      • increment integrity failure counter
        • if integrity failure counter exceed defineed limit
          • delete data and data checksum from storage
      • exit scenario without returning data
    • if checksums are equal:
      • reset integrity failure counter
  • save data checksum with source identifier inside storage
  • return data

Conclusion
Please feel free to comment it, especially if you know better solution or any software who have implemented a such concept efficiently.

UniNet experimental project was close, you can find informations about my new project here.
Related Topics >> P2P      
Comments
Comments are listed in date ascending order (oldest first)

Have a look at Freenet. They have something that seems quite similar to what you are describing in terms of a distributed storage system.
Syndicate content