Thursday, January 10, 2019

Object storage has established itself as a “standard storage” in the enterprise and cloud. As it brings many of the storage best practice to provide durability, scalability, availability and low cost to its users, it could expand its role from being a storage layer to one that facilitates data transfer. We focus on the use cases of object storage so that the onerous tasks of using object storage can become part of it. Object storage then transforms from being a passive storage layer to one actively pulling data from different data sources simply by attaching connectors. The purpose of this document is to describe such connectors that facilitate intelligent automations of data transfers from data sources with minimal or no disruption to their operations.
File-systems have long been the destination to capture traffic and while file system has evolved to stretch over clusters and not just remote servers, it remains inadequate as a blob storage. The connectors enhance the use of object storage and do not compete with the usages of elastic file stores.
We describe the connectors not by the data source they serve but by the technology behind the connectors. We enumerate synchronous send and receive over protocols, asynchronous publisher-subscriber model, compressed data transfer, deduplicated data transfer and incremental rsync based backups to name a few.  We describe their advantages and disadvantages but do not provide a prescription to the retail world which allows them to be used on a case by case basis and with flexible customizations. In this sense, the connectors can be presented in the form of a library with the object storage.
While public cloud object storage solutions offer cloud based services such as Amazon’s Simple Notification Services and Azure Notification hubs, on-premise object storage has the ability to make the leap from standalone storage offering to a veritable solution integration packaging. Public-Cloud offer robust myriad features for their general-purpose cloud services while the connectors specialize in the usages of the object storage.
The data that flows into object storage is often pushed from various senders. That usage continues to be mainstream for the object storage. However, we add additional usages where the connectors pull the data from different data sources. Previously there were in-house scripts for such data transfers. Instead we make it part of the standard storage and provide users just the ability to configure it to their purpose.
The ability to take the onerous routines of using object storage as a storage layer from the layer above across different data sources enables a thinner upper layer and more convenience to the end user. The customizations in the upper layer are reduced and the value additions bubble up the stack.
On-premise object storage is no more a standalone. The public cloud has moved towards embracing on-premise compute resources and their management via System Center integration. The same applies to on-premise object storage as well.  Consequently, the technologies behind the connectors are not only transparent but they are also setup for being monitored and reported via dashboards and charts This improves visibility into the data transfers while enabling cost accounting.
An Object Storage offers better features and cost management, as it continues to stand out against most competitors in the unstructured storage. The connectors lower the costs of usage so that the total cost of ownership is also lowered making the object storage whole lot more profitable to the end users.

No comments:

Post a Comment