Monday, August 13, 2018

We were discussing the suitability of Object Storage to various workloads
We said that the connectors for these data sources are not offered out of object storage products but they could immensely benefit data ingestion. S3 Api deals exclusively with the namespace, buckets and objects even when the Apis are made available as part of SDK but something more is needed for the connectors.
Writing the connectors for each data source is very much like an input-output model. The data is either from the external source to an object storage or from object storage to external source. In each of these directions a connector only changes for the type of external source. Otherwise the object storage facing part of the connector is already implemented in the form of S3 Apis for read and write.  The APIs varies only for the data source as available from the data source. This makes it easy to write the connector as an amalgam of source facing API for bidirectional data transfer to Object-Storage facing S3 Apis. A read from the external data source  is written to Object storage with s3 put api and a write to the external data destination has data coming from Object storage with a read using S3 get apis.  Since each connector varies by the type of external data platform, they can be written one per data platform so that it is easier to use with that data platform. Also, SDKs facilitate development by providing language based convenience. Therefore, the same connector sdk may be offered in more than one language.
The connectors are just an example of programmability convenience of data ingestion from different workloads. Specifying metadata for the objects and showing sample queries on object storage as part of sdk is another convenience for the developers using Object Storage. Well written examples in the sdk and documentation for easing search and analytics associated with Object Storage will tremendously help the advocacy of Object Storage in different software stacks and offerings. Moreover, it will be helpful to log all activities of the sdk for data and queries so that these can make its way to a log store for convenience with audit and log analysis. The usage of sdk to improve automatic tagging and logging is a powerful technique to improve usability and maintaining history.
#codingexercise
boolean isDivisibleBy22(uint n){
return isDivisibleBy2(n) && is DivisibleBy11(n);
}
boolean isDivisibleBy33(uint n) {
return isDivisibleBy3(n) && isDivisibleBy11(n);
}

No comments:

Post a Comment