The previous article continued the elaboration on the usage of the public cloud services for provisioning queue, document store and compute. It talked a bit about the messaging platform required to support this social-engineering application. The problems encountered with social engineering are well-defined and have precedence in various commercial applications. They are primarily about the feed for each user and the propagation of solicitations to the crowd. The previous article described selective fan out. When the clients wake up, they can request their state to be refreshed. This perfects the write update because the data does not need to be sent out. If the queue sends messages back to the clients, it is a fan-out process. The devices can choose to check-in at selective times and the server can be selective about which clients to update. Both methods work well in certain situations. The fan-out happens in both writing as well as loading. It can be made selective as well. The fan-out can be limited during both pull and push. Disabling the writes to all devices can significantly reduce the cost. Other devices can load these updates only when reading. It is also helpful to keep track of which clients are active over a period so that only those clients get preference.    

Historical user activity processing, query processing on warehouse data, and workflows involving report generations for campaign collection do not have to be user interactive. They can be scheduled to run periodically or on-demand and may work on the data in batches or in stream mode. Such workflow automation can easily be serviced with a Master data management architecture that leverages microservices against the store. With this separation of transactional and analytical storage, individual parts may easily employ one or the other vendor while supporting old or new functionalities. 

The Campaign processing system has the option to be streamlined to being only transactional in nature where the users can use their campaign responses and there is no analytics or warehouse required. Then there is the option for the business to gather more information for the campaigns and then use it to better serve the end users.

Some of the advantages for the gathering of information and their patterns in the campaign storage include the grouping and ranking of the top contributors, determining the most usages of the campaigns, the usefulness of certain options in the campaign response features used by the same user, the data mining techniques to perform market basket analysis on the campaign responses and so on. 

Any storage pertaining to operations such as logs, metrics and events constitute machine data, and their storage can not only be dedicated but also support their own indexes, queries and reporting stacks. This is also left out of this discussion except with the mention that each kind of mentioned machine data has well-known solution for their storage and queries. Some of these stacks operated independently from the business and can even be non-intrusive.