Sunday, November 18, 2018

Today we continue discussing the best practice from storage engineering:

  1. Serialization: There is nothing simpler than bytes and offsets to pack and persist any data structure. The same holds true in storage engineering. We have referred to messages as a necessity for communication between layers and components. When these messages are written out, it is irrelevant whether the destination is local or remote. Serialization comes useful in both cases. Consequently, serialization and deserialization are required for most entities. 

  1. Directories: The organization we expect from the user to maintain their storage artifacts is also the same we utilize ourselves within the storage layer so that we don’t have to mix and match different entities. Folders help us organize key values in their own collections. 

  1. Replication strategy: We have referred to replication in storage organization and replication groups earlier but there may be more than one strategy used for replication. The efficiency in replication is closely tied to the organization and the data transfer requirements. Simple file synchronization techniques include events and callbacks to indicate progress, preview the changes to be made, handle conflict resolution and have graceful error handling per unit of transfer. 

  1. Number of replications: Although replication groups are decided by the user and they correspond to sites that participate in keeping their contents similar, every data and metadata units of storage are also candidates for replication whose number does not need to be configured and has to be system defined.  A set of three copies is the norm for most such objects and their metadata. 

  1. Topology: Most storage products are deployed as single instances and usually comprising of a cluster or software defined stacks. However, the layout that the user might choose to have should remain as flexible as possible so that it can scale to their requirements. In this regard, each storage product/server/appliance must behave well with other instances in arrangements such chaining or federation. 

No comments:

Post a Comment