Tuesday, October 30, 2018

We were discussing the use of object storage to stash state across workers from applications and cluster nodes on a lease basis in the previous post.  The object storage acts as an aggregator across workers in this design. It brings elasticity from the virtualized storage customized to individual workers. There is no need for capacity planning for any organization or the payment for file-system, block or other forms of storage as the object storage not only represents the unification of such storage but also supports billing with the availability of detailed study on worker usages. The performance trade-off for workers is small and they are required to change their usages of conventional storage with preferred S3 access. They become facilitators for moving compute layer beyond virtual machines and disks to separate compute and storage platforms the likes of which can generate a new wave of commodity hardware suitable for both on-premise and datacenters. 
The ticketing and leasing system proposed in this post need not just be applications and clusters. It can also be other local platform providers that want to help ease the migration of workloads from local and unmanaged storage.  
The use of a leasing service facilitates object lifecycle management as much as it can be done individually by workers.  It can be offloaded to workers but the notion is that the maintenance activities move from organization owned to self-managed by this solution. Tickets may be implemented with any software as long as they map workers to resources in the object storage.  
Moreover, not all the requests need to reach the object storage. In some cases, web ticket may use temporary storage from hybrid choices. The benefits of using a web ticket including saving bandwidth, reducing server load, and improving request-response time. If a dedicated content store is required, typically the ticketing and server are encapsulated into a content server. This is quite the opposite paradigm of using object storage and replicated objects to directly serve the content from the store. The distinction here is that there are two layers of functions - The first layer is the Ticket layer that solves the life cycle management of storage resources. The second layer is the storage concerns of the actual storage which we mitigate with the help of object storage.  We will call this the storage engine and will get to it shortly.  

The Ticket would be only the ID that the worker needs to track all of its storage resources.    

No comments:

Post a Comment