Tuesday, February 18, 2020

We were discussing the use case of stream storage with message broker. We continue the discussion today.



A message broker can roll over all data eventually to persistence, so the choice of storage does not hamper the core functionality of the message broker.



A stream storage can be hosted on any Tier2 storage whether it is files or blobs. The choice of tier 2 storage does not hamper the functionality of the stream store. In fact, the append only unbounded data nature of messages in the queue is exactly what makes the stream store more appealing to these message brokers

A message broker is supposed to assign the messages If it sends it to the same single point of contention, it is not very useful When message brokers are used with cache, the performance generally improves over what might have been incurred in going to the backend to store and retrieve That is why different processors behind a message broker could maintain their own cache.  A dedicated cache service like AppFabric may also be sufficient to distribute incoming messages. In this case, we are consolidating processor caches with a dedicated cache. This does not necessarily mean a single point of contention. Shared cache may offer at par service level agreement as an individual cache for messages. Since a message broker will not see a performance degradation when sending to a processor or a shared dedicated cache, it works in both these cases. Replacing a shared dedicated cache with a shared dedicated storage such as  a stream store is therefore also a practical option.

No comments:

Post a Comment