Today we continue discussing the best practice from Storage Engineering:
Storage units such as files and objects may be stored as shards on different constituents of the network. The location information in such cases becomes part of the messages between the peers.
Distributed Hash Table has gained widespread popularity in distributing load over a network. There are some well-known players in this technology. Kademlia for instance proves many of the theorems for DHT.
Messages are routed through low latency paths and use parallel asynchronous queries. Message queuing itself and its protocol is an excellent communication mechanism for a distributed network.
Integrity of the data is verified with the help of Merkle trees and proofs in such distributed framework. Others use key based encryptions.
Partial audits reduce overhead in compute and storage. For example, shards of data may be hashed many times to generate pre-leaves in a Merkle tree. And the proof may be compact but the tree may not be compact. Partial audits therefore save on compute and storage.
Data does not always need to be presented in entirety and always from the source of truth. The data may remain completely under the control of the owner while a challenge-proof may suffice
Messages may contain just the hash of the data and the challenge while the responses may contain just the proof. This enables the message exchange to be meaningful while keeping the size within a limit.
Contracts are signed and stored by both parties. Contracts are just as valuable as the sensitive data. Without a contract in place, there is no exchange possible.
Parties interested in forming a contract use a publisher-subscriber model that is topic-based and uses a bloom filter implementation which tests whether an element is a member of a set.
Storage units such as files and objects may be stored as shards on different constituents of the network. The location information in such cases becomes part of the messages between the peers.
Distributed Hash Table has gained widespread popularity in distributing load over a network. There are some well-known players in this technology. Kademlia for instance proves many of the theorems for DHT.
Messages are routed through low latency paths and use parallel asynchronous queries. Message queuing itself and its protocol is an excellent communication mechanism for a distributed network.
Integrity of the data is verified with the help of Merkle trees and proofs in such distributed framework. Others use key based encryptions.
Partial audits reduce overhead in compute and storage. For example, shards of data may be hashed many times to generate pre-leaves in a Merkle tree. And the proof may be compact but the tree may not be compact. Partial audits therefore save on compute and storage.
Data does not always need to be presented in entirety and always from the source of truth. The data may remain completely under the control of the owner while a challenge-proof may suffice
Messages may contain just the hash of the data and the challenge while the responses may contain just the proof. This enables the message exchange to be meaningful while keeping the size within a limit.
Contracts are signed and stored by both parties. Contracts are just as valuable as the sensitive data. Without a contract in place, there is no exchange possible.
Parties interested in forming a contract use a publisher-subscriber model that is topic-based and uses a bloom filter implementation which tests whether an element is a member of a set.
No comments:
Post a Comment