Thursday, October 4, 2018

We were discussing the queue layer over Object Storage.

The reserving of certain queues in the queue layer and the reserving of certain object storage namespaces alleviates the need to have special purpose stacks. In addition, it reuses the existing to infrastructure for concerns such as monitoring, reporting, and background processes. There are no additional artifacts here other than the reservation so that they don’t mix with user data.

Securing system artifacts is equally necessary just as it is important to not allow data corruption. This can be achieved by isolation and encryption. Audit logs for example should not be tampered with. Therefore, they may be encrypted so that the data may not be modified even if they are leaked. System artifacts can use specific prefixes so that they are differentiated from user namespaces. In addition, these namespaces may not be allowed to be created by the user. Finally, data in transit must be secured just as much as it is necessary for the data at rest to be secured.

On the other hand, user queues and object storage must be made as programmatically available collectively as they are made easy individually. Many message queuing stacks provide APIs to manage the queues and they follow similar patterns. These APIs are made available over the web so they can be invoked remotely via direct calls. In addition, if SDKs are made available, then they can improve programmability. SDKs make working with queues and objects easier in the programming language of the user. They don’t need any extra setup and facilitate the calls made to the APIs.

Unlike other SDKs, the queue layer does not merely provide the APIs associated with the queue. It consolidates functionality through the queue layer on the object storage that would have otherwise been directly accessible. Such automation of common tasks associated with the object storage for the purpose of queue management is a useful addition to the SDK.

No comments:

Post a Comment