We were discussing the queue layer over Object Storage.
The reserving of certain queues in the queue layer and the reserving of certain object storage namespaces alleviates the need to have special purpose stacks. In addition, it reuses the existing to infrastructure for concerns such as monitoring, reporting, and background processes. There are no additional artifacts here other than the reservation so that they don’t mix with user data.
User queues and object storage could be made as programmatically available collectively as they are made available individually. Many message queuing stacks provide APIs to manage the queues and they follow similar patterns. Unlike other SDKs, the queue layer does not merely provide the APIs associated with the queue. It consolidates functionality through the queue layer on the object storage that would have otherwise been directly accessible. Such automation of common tasks associated with the object storage for the purpose of queue management is a useful addition to the SDK.
While Queues serve a variety of purpose, they can come in especially useful to capture traffic. From network packets, api messages to just about anything that has a continuous flow, a queue will help record the traffic regardless of the flow. There are no upper limits to the number of messages in a queue and there is no limit to what can be stored in the object storage, therefore these will serve almost any traffic.
Queues do not need to actively participate in standard query operations on the captured traffic. They merely transfer the data to the object storage which can then serve the queries with the help a query execution engine like log parser that uses the available data as an enumerable. The query operations could include selection, aggregation, grouping and ordering.
The reserving of certain queues in the queue layer and the reserving of certain object storage namespaces alleviates the need to have special purpose stacks. In addition, it reuses the existing to infrastructure for concerns such as monitoring, reporting, and background processes. There are no additional artifacts here other than the reservation so that they don’t mix with user data.
User queues and object storage could be made as programmatically available collectively as they are made available individually. Many message queuing stacks provide APIs to manage the queues and they follow similar patterns. Unlike other SDKs, the queue layer does not merely provide the APIs associated with the queue. It consolidates functionality through the queue layer on the object storage that would have otherwise been directly accessible. Such automation of common tasks associated with the object storage for the purpose of queue management is a useful addition to the SDK.
Queues do not need to actively participate in standard query operations on the captured traffic. They merely transfer the data to the object storage which can then serve the queries with the help a query execution engine like log parser that uses the available data as an enumerable. The query operations could include selection, aggregation, grouping and ordering.
No comments:
Post a Comment