Today we continue discussing the best practice from storage engineering:
326) A higher layer that manages and understands abstractions, provides namespace and data operation ordering is present in almost all storage management systems
327) A lower layer comprising of distribution and replication of data without actually requiring any knowledge of abstractions maintained by the higher layer is similarly present in almost all storage products.
328) Similarly, the combination of the two layers described above is almost always separated from the front-end layer interpreting and servicing the user requests.
329) Working with streams is slightly different from fixed sized data. It is an ordered set of references to segments. All the extents are generally immutable.
330) If a shared resource can be represented as a pie, there are two ways to enhance the usage: First, make the pie bigger and the allocate the same fractions in the increment. Second, dynamically modify the fractions so that at least some of the resource usages can be guaranteed some resource.
331) In order to remove the notion of a centralized storage provider, some storage products have preferred distributed cloud storage network. A distributed hash table overlays the network.
332) Storage units such as files and objects may be stored as shards on different constituents of the network. The location information in such cases becomes part of the messages between the peers.
333) Distributed Hash Table has gained widespread popularity in distributing load over a network. There are some well-known players in this technology. Kademlia for instance proves many of the theorems for DHT.
334) Messages are routed through low latency paths and use parallel asynchronous queries. Message queuing itself and its protocol is an excellent communication mechanism for a distributed network.
335) Integrity of the data is verified with the help of Merkel trees and proofs in such distributed framework. Others use key based encryptions
326) A higher layer that manages and understands abstractions, provides namespace and data operation ordering is present in almost all storage management systems
327) A lower layer comprising of distribution and replication of data without actually requiring any knowledge of abstractions maintained by the higher layer is similarly present in almost all storage products.
328) Similarly, the combination of the two layers described above is almost always separated from the front-end layer interpreting and servicing the user requests.
329) Working with streams is slightly different from fixed sized data. It is an ordered set of references to segments. All the extents are generally immutable.
330) If a shared resource can be represented as a pie, there are two ways to enhance the usage: First, make the pie bigger and the allocate the same fractions in the increment. Second, dynamically modify the fractions so that at least some of the resource usages can be guaranteed some resource.
331) In order to remove the notion of a centralized storage provider, some storage products have preferred distributed cloud storage network. A distributed hash table overlays the network.
332) Storage units such as files and objects may be stored as shards on different constituents of the network. The location information in such cases becomes part of the messages between the peers.
333) Distributed Hash Table has gained widespread popularity in distributing load over a network. There are some well-known players in this technology. Kademlia for instance proves many of the theorems for DHT.
334) Messages are routed through low latency paths and use parallel asynchronous queries. Message queuing itself and its protocol is an excellent communication mechanism for a distributed network.
335) Integrity of the data is verified with the help of Merkel trees and proofs in such distributed framework. Others use key based encryptions
No comments:
Post a Comment