Tuesday, August 21, 2018

We were discussing gateway and object storage. We  wanted to create content distribution network from the object storage itself using a gateway like functionality to objects as geo-redundant copies. The storage engine layer responsible for the creation of objects would automatically take care of the replication of the objects. Storage engines generally have a notion each for a virtual data center and a replication group. An object created within a virtual data center is owned by that virtual data center. If there are more than one virtual data center within a replication group, the owning virtual data center within a group will be responsible to replicate the object in the other virtual data center and this is usually done after the object has been written. At the end of the replication, each virtual data center has a readable copy of the object.The location information is internal to the storage engine unlike the internet accessible address. The address is just another attribute for the object. Since the address has no geography specific information as per our design of the gateway, the rules for the gateway can be used to route a read request to the relevant virtual data center which will use the address to identify the object and use its location to read the object. 
Together the gateway and the storage engine provide address and copies of objects to facilitate access via a geographically close location. However, we are suggesting native gateway functionality to object storage in a way that promotes this Content Distribution Network.  Since we have copies of the object, we don’t need to give an object multiple addresses for access from different geographical regions.  
The object storage has an existing concept of replication group. Their purpose was to define a logical boundary where storage pool content is protected. These groups can actually be local or global. A local replication group protects objects within the same virtual data center. The global replication groups protect the objects against disk, node as well as site failures. The replication strategy is inherent to the object storage. The copies made for the object are within the replication group. In a multi-site content distribution network, the copies may exist outside of the local replication group. The copies of the objects are then made outside of the replication group creating new isolated objects.  In such cases, the replication strategy for content-distribution network may kick in to maintain the contents to be the same. However, in this case too, we don’t have to leverage external technologies to configure replication strategy different from that of the object storage. A multi-site virtual data center collection may be put under the same replication group and this should suffice to create enough copies across sites where sited are earmarked for different geographies. 

  

No comments:

Post a Comment