We were discussing the suitability of Object Storage to various workloads. Specifically, we discussed its role in Artifactory which is used to store binary objects from CI/CD. A large number of binary objects or files gets generated with each run of the build. These files are mere build artifacts and the most common usage of these files is download. Since the hosted solution is cloud based, Artifactory users demands elasticity, durability and http access. Object Storage is best suited to meet these demands.  The emphasis here is the distinction over a file-system exposed over the web for their primary use case scenario. In fact, the entire DevOps process with its CI/CD servers can use a common Object Storage instance so that there is little if any copying of files from one staging directory to another. The Object Storage not only becomes the final single instance destination but also avoids significant inefficiencies in the DevOps processes.  Moreover, builds are repeated through development, testing and production so the same solution works very well for repetitions. This is not just a single use case but an indication that there are many cycles within the DevOps process that could benefit from a storage tier as Object Storage. Static content like binary images of executable are generally copy over write. Versioning for same named files is a feature of Object Storage. Object Storage can not only use file exports but also provide automatic versioning of content. It becomes a content library for binary artifacts of build in all the features demanded over a file system such as versioning. Previous versions may be retained for as long as the life-cycle rules allow. These rules can be specified for the objects. It can also provide time limited access to content. The URI exposed for the object can be shared with anyone. The object may be downloaded on any device anywhere. It enables multi-part upload (mpu) of large objects. This is considered a significant improvement for large binary objects since it enables binary transfer in parts. There are three steps - an mpu upload is requested, different parts are uploaded, and finally an mpu complete is requested. The object storage constructs the object from the parts and then it can be accessed just the same as any other object. Each part is identified and they can number in hundreds. The part upload request includes a part number. The object storage returns a tag header for each part. The header and part number must be included in subsequent requests. The parts can be sent to object storage in any order.  A complete request or an abort request must be sent to finalize the parts and permit Object storage to start reconstruction of the object and removing the parts. Parts uploaded so far can be listed.  If the listed parts are more than 1000, a series of such list requests need to be sent. Multi-part uploads can be concurrent.If a part is sent again, it will update the already uploaded part. All the parts are used for reconstruction of the original object only after the complete request is received.

We were discussing the suitability of Object Storage to various workloads and the programmability convenience that enables migration of old and new workloads. We discussed the use of UI as well as SDK for ingesting data.

Let us now consider the usage of object storage for powering web applications. Static resources and files for web application can be served directly out of object storage. There are many web applications that require to serve a portion of the file system over the web due to a large number of artifacts. These are ideal for Object Storage. Consider Artifactory which is a leading hosted solution for all things binary. It is a perfect match for code repositories and aids CI/CD. A large number of binary objects or files gets generated with each run of the build. These files are mere build artifacts and the most common usage of these files is download. Since the hosted solution is cloud based, Artifactory demands elasticity, durability and http access. These are just some of the things that Object Storage provides a suitable platform for.  The emphasis here is the suitability of Object Storage over a filesystem for the primary use case scenario. In fact, the entire DevOps process with its CI/CD servers can use a common Object Storage instance so that there is little if any copying of files from one staging directory to another. The Object Storage not only becomes the final single instance destination but also avoids significant inefficiencies in the DevOps processes.  Moreover, builds are repeated through development, testing and production so the same solution works very well for repetitions. This is not just a single use case but an indication that there are many cycles within the DevOps process that could benefit from a storage tier as Object Storage.
boolean isDivisibleBy35(uint n)
{
return isDivisbleBy5(n) && isDivisibleBy7(n);
}

We were discussing the suitability of Object Storage to various workloads and the programmability convenience that enables migration of old and new workloads. In  particular, we discussed connectors for various data sources and their bidirectional data transfer. Duplicity is a command line tool that is an example of a connector tool but we were discussing availability of an SDK with the object storage. Writing the connectors for each data source is very much like an input-output model. The data is either from the external source to an object storage or from object storage to external source. In each of these directions a connector only changes for the type of external source. Otherwise the object storage facing part of the connector is already implemented in the form of S3 Apis for read and write.  The APIs varies only for the data source as available from the data source. This makes it easy to write the connector as an amalgam of source facing API for bidirectional data transfer to Object-Storage facing S3 Apis. A read from the external data source  is written to Object storage with s3 put api and a write to the external data destination has data coming from Object storage with a read using S3 get apis.  Since each connector varies by the type of external data platform, they can be written one per data platform so that it is easier to use with that data platform. Also, SDKs facilitate development by providing language based convenience. Therefore, the same connector sdk may be offered in more than one language.

SDKs may be offered in any language for the convenience of writing data transfer in any environment. It just does not stop there. UI widens the audience for the same purposes and brings in administrators and systems engineering without the need for writing scripts or code. ETL for example is a very popular usage of designer tools with drag and drop logic facilitating wiring and transfer of data. SDK may power the UI as well and both can be adapted to the data source, environment and tasks.

#codingexercise

bool isDivisibleBy55(uint n)

{

return isDivisibleBy5(n) &&isDivisibleBy11(n);

}

bool isDivisibleBy77(uint n)

{

return isDivisibleBy7(n) &&isDivisibleBy11(n);

}

We were discussing the suitability of Object Storage to various workloads
We said that the connectors for these data sources are not offered out of object storage products but they could immensely benefit data ingestion. S3 Api deals exclusively with the namespace, buckets and objects even when the Apis are made available as part of SDK but something more is needed for the connectors.
Writing the connectors for each data source is very much like an input-output model. The data is either from the external source to an object storage or from object storage to external source. In each of these directions a connector only changes for the type of external source. Otherwise the object storage facing part of the connector is already implemented in the form of S3 Apis for read and write.  The APIs varies only for the data source as available from the data source. This makes it easy to write the connector as an amalgam of source facing API for bidirectional data transfer to Object-Storage facing S3 Apis. A read from the external data source  is written to Object storage with s3 put api and a write to the external data destination has data coming from Object storage with a read using S3 get apis.  Since each connector varies by the type of external data platform, they can be written one per data platform so that it is easier to use with that data platform. Also, SDKs facilitate development by providing language based convenience. Therefore, the same connector sdk may be offered in more than one language.
The connectors are just an example of programmability convenience of data ingestion from different workloads. Specifying metadata for the objects and showing sample queries on object storage as part of sdk is another convenience for the developers using Object Storage. Well written examples in the sdk and documentation for easing search and analytics associated with Object Storage will tremendously help the advocacy of Object Storage in different software stacks and offerings. Moreover, it will be helpful to log all activities of the sdk for data and queries so that these can make its way to a log store for convenience with audit and log analysis. The usage of sdk to improve automatic tagging and logging is a powerful technique to improve usability and maintaining history.
#codingexercise
boolean isDivisibleBy22(uint n){
return isDivisibleBy2(n) && is DivisibleBy11(n);
}
boolean isDivisibleBy33(uint n) {
return isDivisibleBy3(n) && isDivisibleBy11(n);
}

We were discussing the suitability of Object Storage to various workloads after having discussed its advantages and its position as a perfect storage tier:
The data sources can include:
Backup and restore workflows
Data warehouse ETL loads
Log stores and indexes
Multimedia libraries
Other file systems
Relational database connections
NoSQL databases
Graph databases
All upstream storage appliances excluding aging tiers.
Notice that the connectors for these data sources are not offered out of object storage. In reality, S3 Api deals exclusively with the namespace, buckets and objects even when the Apis are made available as part of SDK.
Writing the connectors for each data source is very much like an input-output model. The data is either from the external source to an object storage or from object storage to external source. In each of these directions a connector only changes for the type of external source. Otherwise the object storage facing part of the connector is already implemented in the form of S3 Apis for read and write.  The APIs varies only for the data source as available from the data source. This makes it easy to write the connector as an amalgam of source facing API for bidirectional data transfer to Object-Storage facing S3 Apis. A read from the external data source  is written to Object storage with s3 put api and a write to the external data destination has data coming from Object storage with a read using S3 get apis.  Since each connector varies by the type of external data platform, they can be written one per data platform so that it is easier to use with that data platform. Also, SDKs facilitate development by providing language based convenience. Therefore, the same connector sdk may be offered in more than one language.



#codingexercise
bool isDivisibleBy14 (n) {
return isDivisibleBy (2) && isDivisibleBy(7);
}

Object Storage is very popular with certain content. Files directly map to objects. Multimedia content are also helpful to be served from object storage  Large files such as from Artifactory are also suitable for Object Storage. An entire cluster based file system may also be exported and this may be used with Object Store. Deduplication appliance may also provide benefit benefits in conjunction with an Object Storage.
Object Storage is usually viewed as a storage appliance in itself. Therefore it provides a form of raw storage suitable for what can be viewed as objects. However a suite of connectors may be made available in the form of sdk, that enables data to move into object storage from well-known platforms. For example, data in a content-library can be moved into object storage with the help of a connector in the sdk. This is just one of the examples, there are several more.

The data sources can include:
Backup and restore workflows
Data warehouse ETL loads
Log stores and indexes
Multimedia libraries
Other file systems
Relational database connections
NoSQL databases
Graph databases
All upstream storage appliances excluding aging tiers

#codingexercise
bool isDivisibleBy21 (n) {
return isDivisibleBy (3) && isDivisibleBy(7);
}

We were discussing application virtualization and the migration of workloads:
We brought up how both application as well as storage tier benefit from virtualization and the automation of workload migration using tools. Object storage itself may be on a container facilitating easy migration across hosts. Since object storage virtualizes datacenters and storage arrays, it is itself at once a storage application as well as  a representation of unbounded storage space. Once the workloads have been migrated to object storage, both can then be moved around the cloud much more nimbly than they were if they used raw storage volumes. 
One of the challenges associated with migration is that Application Server - Storage Tier model has evolved to a lot more complex paradigms. There is no more just an application server and a database. In fact servers are replaced by clusters and nodes. Applications are replaced by modules and modules run on containers. Platform as a service has evolved to using Mesos and Marathon where even the storage volumes are moved around if they are not  a shared volume.  Data usually resides in the form of files and the database connectivity is re-established because the connection string does not change as the nodes are rotated. Marathon monitors the health of the nodes as the application and storage is moved around  In the object storage, the location of the object is arbitrary once the underlying storage is virtualized. Object storage itself may use a container that may make it portable but it is generally not the norm to move Object Storage around in a Marathon framework. If anything Object Storage is akin to a five hundred pound gorilla in the room.
Object Storage is very popular with certain content. Files directly map to objects. Multimedia content are also helpful to be served from object storage  Large files such as from Artifactory are also suitable for Object Storage. An entire cluster based file system may also be exported and this may be used with Object Store. Deduplication appliance may also provide benefit benefits in conjunction with an Object Storage. 
Object Storage is usually viewed as a storage appliance in itself. Therefore it provides a form of raw storage suitable for what can be viewed as objects. However a suite of connectors may be made available in the form of sdk, that enables data to move into object storage from well-known platforms. For example, data in a content-library can be moved into object storage with the help of a connector in the sdk. This is just one of the examples, there are several more.

#codingexercise
bool isDivisibleBy12(uint n)
{
return isDivisibleBy3(n) && isDivisbleBy4(n);
}