Monday, September 17, 2018

We were discussing the role of Cache Service with Object Storage. The requirements for object storage need not even change while the reads and writes from the applications can be handled. There can be a middle layer as a proxy for a file system to the application while utilizing the object storage for persistence.  This alleviates performance considerations to read and write deep into the private cloud each time. That is how this Cache Layer positions itself. It offers the same performance as query plan caching does to handle the workload and while it may use its own intermediate storage, it works as a staging for the data so that the data has a chance to age and persist in object storage.
Object Storage is a limitless storage. Data from any workload is anticipated to grow over time if it is saved continuously. Consequently the backend and particularly the cloud services are better prepared for this task. While a flush to local file system with an asynchronous write may be extremely cheap compared to persistence in the cloud as an S3 object, there is no reason to keep rolling over local filesystem data by hand to object storage. 
Object storage is a zero maintenance storage. There is no planning for capacity and elastic nature of its services may be taken for granted. The automation of asynchronous writes, flush to object storage and sync of data in cache to that in object storage is now self-contained and packaged into this cache layer. 
The cloud services are elastic. Both the storage and the cache services could be deployed in the cloud which not only gives the same benefits to one application or client but also to every department, organization, application, service and workload. 
Object storage coupled with this cache layer is also suited to dynamically address the needs for client and application because the former may have been a global store but the latter is able to determine the frequency depending on the workload.  Different applications may tune the caching to its requirements. 
Performance increases dramatically when the results are returned as close to the origin of the requests instead of going deep into the technology stack. This has been one of the arguments for web cache and web proxy in general. 
Such service is hard to mimic individually within each application or client. Moreover, optimizations only happen when the compute and storage are elastic where they can be studied, cached, and replayed independent of the applications. 
The move from tertiary to secondary storage is not a straightforward shift from NAS storage to object storage without some form of chores over time. A dedicated product likes this takes the concern out of the picture. 
#linear regression for bounding box adjustment in regions of interest.

This fits a line to the data points.

No comments:

Post a Comment