We were discussing the use case of stream storage with message broker. We continue the discussion today.
A message broker can roll over all data eventually to persistence, so the choice of storage does not hamper the core functionality of the message broker.
A stream storage can be hosted on any Tier2 storage whether it is files or blobs. The choice of tier 2 storage does not hamper the functionality of the stream store. In fact, the append only unbounded data nature of messages in the queue is exactly what makes the stream store more appealing to these message brokers
While message broker lookup queues by address, a gateway service can help with translation of address. In this way, a gateway can significantly expand the distribution of queues.
First, the address mapping is not at site level. It is at queue level.
Second, the address of the queue – both universal as well as site specific are maintained along with the queue as part of its location information
Third, instead of internalizing a table of rules from the external gateway, a lookup service can translate universal queue address to the address of the nearest queue. This service is part of the message broker as a read only query. Since queue name and address is already an existing functionality, we only add the ability to translate universal address to site specific address at the queue level.
Fourth, the gateway functionality exists as a microservice. It can do more than static lookup of physical location of an queue given a universal address instead of the site-specific address. It has the ability to generate tiny urls for the queues based on hashing. This adds aliases to the address as opposed to the conventional domain-based address. The hashing is at the queue level and since we can store billions of queues in the queue storage, a url shortening feature is a significant offering from the gateway service within the queue storage. It has the potential to morph into other services than a mere translator of queue addresses. Design of a url hashing service was covered earlier as follows.
Fifth, the conventional gateway functionality of load balancing can also be handled with an elastic scale-out of just the gateway service within the message broker.
Sixth, this gateway can also improve access to the queue by making more copies of the queue elsewhere and adding the superfluous mapping for the duration of the traffic. It need not even interpret the originating ip addresses to determine the volume as long as it can keep track of the number of read requests against existing address of the same queue.
These advantages can improve the usability of the message brokers and their queues by providing as many as needed along with a scalable service that can translate incoming universal address of queues to site specific location information.
A message broker can roll over all data eventually to persistence, so the choice of storage does not hamper the core functionality of the message broker.
A stream storage can be hosted on any Tier2 storage whether it is files or blobs. The choice of tier 2 storage does not hamper the functionality of the stream store. In fact, the append only unbounded data nature of messages in the queue is exactly what makes the stream store more appealing to these message brokers
While message broker lookup queues by address, a gateway service can help with translation of address. In this way, a gateway can significantly expand the distribution of queues.
First, the address mapping is not at site level. It is at queue level.
Second, the address of the queue – both universal as well as site specific are maintained along with the queue as part of its location information
Third, instead of internalizing a table of rules from the external gateway, a lookup service can translate universal queue address to the address of the nearest queue. This service is part of the message broker as a read only query. Since queue name and address is already an existing functionality, we only add the ability to translate universal address to site specific address at the queue level.
Fourth, the gateway functionality exists as a microservice. It can do more than static lookup of physical location of an queue given a universal address instead of the site-specific address. It has the ability to generate tiny urls for the queues based on hashing. This adds aliases to the address as opposed to the conventional domain-based address. The hashing is at the queue level and since we can store billions of queues in the queue storage, a url shortening feature is a significant offering from the gateway service within the queue storage. It has the potential to morph into other services than a mere translator of queue addresses. Design of a url hashing service was covered earlier as follows.
Fifth, the conventional gateway functionality of load balancing can also be handled with an elastic scale-out of just the gateway service within the message broker.
Sixth, this gateway can also improve access to the queue by making more copies of the queue elsewhere and adding the superfluous mapping for the duration of the traffic. It need not even interpret the originating ip addresses to determine the volume as long as it can keep track of the number of read requests against existing address of the same queue.
These advantages can improve the usability of the message brokers and their queues by providing as many as needed along with a scalable service that can translate incoming universal address of queues to site specific location information.
No comments:
Post a Comment