This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice:
240) Build a product where the communication to explain how the product works consumes bulk of the time that could have otherwise been spent on test and development.
241) Build a product where the manual for the product is bulky and can become a headrest
242) Build a product where the user interface real estate is all cluttered up and there is no room to display anything new.
243) Build a product where users register through a third-party identity provider and cannot login to the product if there is an outage
244) Build a product where the home page is a mashup of different websites and the user has to squint at a small portion of the screen
245) Build a product where the forms are so big and large they are never completed in one sitting
246) Build a product that seems to have one form after another to fill on every page.
247) Build a product that shows ads or campaigns that distract the user from their workflows
248) Build a product that makes recommendations even when you want it to stop.
249) Build a product that chronicles every order and makes it creepy to anticipate your next order
250) Build a product that looks simple but when you click the advanced options, it becomes overwhelming.
251) Build a product that makes you think how to query the results
252) Build a product that gives no option to query other than scroll through the results.
253) Build a product that makes you go and forth between pages of search results
254) Build a product that does not let you order the results
255) Build a product that requires you to know the jargon or the domain with little or no warning for consequences.
256) Build a product that has clunky form to create a search criteria
257) Build a product where the forms don’t let you see the entire entry you type
258) Build a product where the forms don’t let you correct your mistakes
259) Build a product that does not let you rename your resources and you have to create more
260) Build a product that does not send out reminders. 

This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice :
230) Build a product where the defects flow to the core of the product from any surface area because it is easier to consult the expert than to do what it takes to resolve the defect upfront
231) Build a product where the core is shielded by layers upon layers and defects in the core lead to workarounds and complexity in outer layers
232) Build a product where the clients can’t keep up with the breaking changes in the server both before and after release
233) Build a product where there are proprietary protocols between components and find that the protocol grows in complexity as opposed to following open, published and popular protocols
234) Build a product where the components fail silently or worse lie when they fail.
235) Build a product where the callers have no way to figure out whether the callee did the task
236) Build a product where the caller does not allow differentiation of callees and find that there is no way to put an expedient fix specific to certain callee.
237) Build a product where there is no protection of boundary such as between layers or external endpoints and find all kinds of security vulnerabilities afterwards
238) Build a product that is well thought through and find the solution to be too elegant and simple to describe.
239) Build a product that provides no mechanism for callers to inject tracers and find that the callers have to rely on communications with the experts
240) Build a product where the communication to explain how the product works consumes bulk of the time that could have otherwise been spent on test and development.
241) Build a product where the manual for the product is bulky and can become a headrest
242) Build a product where the user interface real estate is all cluttered up and there is no room to display anything new.
243) Build a product where users register through a third-party identity provider and cannot login to the product if there is an outage
244) Build a product where the home page is a mashup of different websites and the user has to squint at a small portion of the screen
245) Build a product where the forms are so big and large they are never completed in one sitting
246) Build a product that seems to have one form after another to fill on every page.
247) Build a product that shows ads or campaigns that distract the user from their workflows
248) Build a product that makes recommendations even when you want it to stop.
249) Build a product that chronicles every order and makes it creepy to anticipate your next order
250) Build a product that looks simple but when you click the advanced options, it becomes overwhelming.

This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice :

220) Build a product that specializes monitoring and alerting features and find that customers are not utilizing because they cannot plug it into their existing infrastructure.
221) Build a product where the features are built independently but find that the customer has turned on all of them
222) Build a product as a culmination of development efforts but find a mismatch between customer expectations and delivery
223) Build a product with internal vetting and sometimes friction and find the customer advocacy voice waning.
224) Build a product with the notion of getting a feature into the hands of customer without being explained fully why the customers want it
225) Build a product as a part of a portfolio and find that the entire value proposition of the product subdued by the portfolio perception.
226) Build a product as a part of portfolio that prevents it from changing its price, licensing and distribution and all the innovation in the product averages to that of the portfolio
227) Build a product that cannot be sold separately and find that the product gains attention with the plus and minus of the other.
228) Build a product where the distribution is in flavors that does not work as well as the default and have all the negative feedback be attributed to the product as a whole
229) Build a product that has client-server technology and find that all the client issues are attributed to the server.
230) Build a product where the defects flow to the core of the product from any surface area because it is easier to consult the expert than to do what it takes to resolve the defect upfront
231) Build a product where the core is shielded by layers upon layers and defects in the core lead to workarounds and complexity in outer layers
232) Build a product where the clients can’t keep up with the breaking changes in the server both before and after release
233) Build a product where there are proprietary protocols between components and find that the protocol grows in complexity as opposed to following open, published and popular protocols
234) Build a product where the components fail silently or worse lie when they fail.
235) Build a product where the callers have no way to figure out whether the callee did the task
236) Build a product where the caller does not allow differentiation of callees and find that there is no way to put an expedient fix specific to certain callee.
237) Build a product where there is no protection of boundary such as between layers or external endpoints and find all kinds of security vulnerabilities afterwards
238) Build a product that is well thought through and find the solution to be too elegant and simple to describe.
239) Build a product that provides no mechanism for callers to inject tracers and find that the callers have to rely on communications with the experts
240) Build a product where the communication to explain how the product works consumes bulk of the time that could have otherwise been spent on test and development.

This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice :

205) Build a product with rich mobile enhanced experience across a variety of devices and find that the customers are able to use the product even before going to sleep
206) Build a product with mobile integration of payment methods such as personal wallet and the chore of paying becomes smooth.
207) Build a product that can use digital cards and refill them at will and the product becomes usable at all participating stores increasing adoption via partner networks
208) Build a product with partner sponsored incentives to customers and see the appeal grow as partner network endears itself to the customer more than it would have individually
209) Build a product with little or no ecosystem and the product fails to strike popularity in conferences, exhibitions and symposiums
210) Build a product with little user education and find that the sales booths at an exhibition are not visited.
211) Build a product with very little wording on consequences and see the amazement when users click a button.
212) Build a product that does not warn on consequences and find that the user has just deleted his data.
213) Build a product that does not take any notes the user wants to leave and find that the user does not remember why he took some actions
214) Build a product that does lets users erase their activities and find no record to come to their aid
215) Build a product that works with some users and not with others and find that they have to scratch their heads to find out why.
216) Build a product that has intermittent outages which it hides from the user
217) Build a product that has little or no documentation on escalation policy and find that genuine problems can be described well but not brought up for mitigation.
218) Build a product that tries to tackle the hard 10% scientific problems at the cost of 90% commercial problems.
219) Build a product that mitigates troubleshooting via published documents and find that the support team is building supportability features rather than support case handling
220) Build a product that specializes monitoring and alerting features and find that customers are not utilizing because they cannot plug it into their existing infrastructure.
221) Build a product where the features are built independently but find that the customer has turned on all of them
222) Build a product as a culmination of development efforts but find a mismatch between customer expectations and delivery
223) Build a product with internal vetting and sometimes friction and find the customer advocacy voice waning.
224) Build a product with the notion of getting a feature into the hands of customer without being explained fully why the customers want it
225) Build a product as a part of a portfolio and find that the entire value proposition of the product subdued by the portfolio perception.

The benefits of Async stream readers in cache for stream storage:
The cache essentially projects portions of the stream so that the entire stream does not have to scanned from begin to end repeatedly. This mode of operation is different from finding popular stream segments across stream readers.
The cache may use a queue to hold contiguous segments from a stream.
The choice of queue is extremely important for peak throughput. The use of a lock free data structure as opposed to an ArrayBlockingQueue can do away with lock contention. The queues work satisfactorily until they are full. Garbage free Async stream readers have the best response time.
The benefits here are similar to LogAppenders except that they are on write path whole these are in read path.
Asynchronous read together with lock free access and skip level access boost performance. The cache may encounter significant size for each segment and a writer that transfers a segment over the network may take time in the order of hundreds of milliseconds. Instead, having a continuous background import for adjustable window of steam segments tremendously improves the load on the stream store while responding to seek requests on segments
Another approach is to have a continuous reader read the stream from beginning to end in a loop and have the cache decide which of the segments it needs to read. This approach requires the cache to discard segments that are not necessary for the clients.  The cache can fork as many readers as necessary to replenish the window of segments it needs. The cache can decide which windows if segments it needs based on overlapping intervals of segments requested by clients. These segments are no longer bound to clients and they are managed exclusively by the cache.
The skip level access continues to serve the readers in this case although it doesn’t need to read the streams directly. The readers present the segments and the cache then uses its statistics and algorithm to prefetch the segments that will serve the clients best. These cache entries can be evicted by any algorithm that suits the cache. Storage is not a concern for the cache because we have distributed ring cache with consistency check points
The maximum number of segments in a window is directly dependent on the cache. The segment size may even be a constant in which case the number of segments depends on how many streams and how many windows per stream the cache wants to accommodate. This window size is not the same as a continuously sliding window used to analyze the stream which is usually one per stream.
When there is only one reader to replenish the segments for the cache, the clients are freed from performing the same task over and over again on the actual stream. The segments that the client need will most likely be found in the cache. If the segments are not found, the cache can fetch it from the stream on demand. Otherwise the scheduled reading from the stream by the cache will replenish it. The cache itself alleviates the load from the stream store.
The cache can also fork multiple readers at fixed duration offsets from the start of the first. Multiple readers at fixed periodic intervals from the start of the first will continue to read the steam segments and reduce the latency in replenishing the segments that the cache needs. When the cache advertises it needs segment number n, the readers that have not yet read n will read and make it available. With multiple readers, the chance that one of them is closest to segment n is higher.
There are ways in which the cache can become smart about loading segments using these readers. For example, instead of waiting for the nearest reader to the segment to read it into the cache, the cache can have a dedicated reader fetch the segment or a set of segments via skip-level access.
There are also techniques to make read and writes to go together. Typically reads and writes are not in the same stream and the cache behaves as a write-through, hence the clients can interact directly with the cache rather than the store. When a segment is not found in the cache, it is also efficiently handled with skip level access.

When readers read a stream from beginning to end, they become easy to scale.  This is simple strategy and works well for parallelizing computations. However, it introduces latency as streams can be quite long.

A cache served to provide the stream segments do that clients don’t have to go all the way to the store. It could employ a group of readers that could replenish the segments that are in most demand. There is always the option for clients to reach the store if the cache does not have what it needs. Generally s cache will bring in the segment on behalf of the client if it goes not have it.

The techniques for providing stream segments do not matter to the client and the cache can use any algorithm. The cache also provides the benefits of alleviating load from the stream store without any additional constraints. In fact the cache will also use the same stream reader as the client and with the only difference that there will be fewer stream readers on the stream store than before.
We have not compared this cache layer with a message queue server but there are interesting problems common to both. For example, we have a multiple consumer single producer pattern in the periodic reads from the stream storage. The message queue server or broker enables this kind of publisher-subscriber pattern with retries and dead letter queue. In addition, it journals the messages for review later. Messaging protocols are taken up a notch in performance with the use of a message queue broker and their reliance on sockets with steroids.  This leaves the interaction between the caches and the storage to be handled elegantly with well-known messaging framework. The message broker inherently comes with a scheduler to perform repeated tasks across publishers. Hence it is easy for the message queue server to perform as an orchestrator between the cache and the storage, leaving the cache to focus exclusively on the cache strategy suitable to the workloads.  Journaling of messages also helps with diagnosis and replay and probably there is no better store for these messages than the object storage itself. Since the broker operates in a cluster mode, it can scale to as many caches as available. Moreover, the journaling is not necessarily available with all the messaging protocols which counts as one of the advantages of using a message broker.  Aside from the queues dedicated to handle the backup of objects from cache to storage, the message broker is also uniquely positioned to provide differentiated treatment to the queues.  This introduction of quality of service levels expands the ability of the solution to meet varying and extreme workloads The message queue server is not only a nice to have feature but also a necessity and a convenience when we have a distributed cache to work with the stream storage.
The number of workers for the cache or the store does not matter and they can scale.
Please refer to the discussion on caching context for stream managers
https://1drv.ms/w/s!Ashlm-Nw-wnWvBXkU9jz_Z2EXWLp

and sample implementation with: https://github.com/ravibeta/JavaSamples/tree/master/logging

This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice :

195) Build a product that does not keep history of all actions, assets and activities only to find significant effort in rebuilding it.

196) Build a product that does not maintain registry for all users and find that there is no blacklist or whitelist capability

197) Build a product that assumes secure communication via tunnels and find that the tunnels need not be continuous through a proxy.

198) Build a product that does not gain statistics of its usage and find that the pain grows among users

199) Build a product that has no telemetry and not have any insight into how customers are doing with the product.

200) Build a product that has built in dial home capabilities and find that privacy gurus will find it suspect.

201) Build a product that has little or no test tools shipped with its product and find that customers are knocking

202) Build a product that leverages APIs for all its workflows and find that the product becomes part of more and more workflows.

203) Build a product with enough bread crumbs and sitemap to let the user navigate the pages and see that the usability goes up and so does the customer appeal.

204) Build a product that ships on mobile devices and find the audience skyrocket

205) Build a product with rich mobile enhanced experience across a variety of devices and find that the customers are able to use the product even before going to sleep

206) Build a product with mobile integration of payment methods such as personal wallet and the chore of paying becomes smooth.

207) Build a product that can use digital cards and refill them at will and the product becomes usable at all participating stores increasing adoption via partner networks

208) Build a product with partner sponsored incentives to customers and see the appeal grow as partner network endears itself to the customer more than it would have individually

209) Build a product with little or no ecosystem and the product fails to strike popularity in conferences, exhibitions and symposiums

210) Build a product with little user education and find that the sales booths at an exhibition are not visited.
211) Build a product with very little wording on consequences and see the amazement when users click a button.
212) Build a product that does not warn on consequences and find that the user has just deleted his data.
213) Build a product that does not take any notes the user wants to leave and find that the user does not remember why he took some actions
214) Build a product that does lets users erase their activities and find no record to come to their aid
215) Build a product that works with some users and not with others and find that they have to scratch their heads to find out why.

This is a continuation of the earlier posts to enumerate funny aspects of software engineering practice :

190) Build a product with ambitious charter and find the release timelines reducing the plan.
191) Build a product that does not preserve data between restarts, retries or disaster recovery
192) Build a product that does not retain user settings only to have the user apply them again
193) Build a product by putting together functionality via dependencies and find that the massive surface area attracts all kinds of vulnerabilities.
194) Build a product that creates sdks, command-line clients and other artifacts and find that the versioning for each becomes a maintenance issue.
195) Build a product that does not keep history of all actions, assets and activities only to find significant effort in rebuilding it.
196) Build a product that does not maintain registry for all users and find that there is no blacklist or whitelist capability
197) Build a product that assumes secure communication via tunnels and find that the tunnels need not be continuous through a proxy.
198) Build a product that does not gain statistics of its usage and find that the pain grows among users
199) Build a product that has no telemetry and not have any insight into how customers are doing with the product.
200) Build a product that has built in dial home capabilities and find that privacy gurus will find it suspect.
201) Build a product that has little or no test tools shipped with its product and find that customers are knocking
202) Build a product that leverages APIs for all its workflows and find that the product becomes part of more and more workflows.
203) Build a product with enough bread crumbs and sitemap to let the user navigate the pages and see that the usability goes up and so does the customer appeal.
204) Build a product that ships on mobile devices and find the audience skyrocket
205) Build a product with rich mobile enhanced experience across a variety of devices and find that the customers are able to use the product even before going to sleep
206) Build a product with mobile integration of payment methods such as personal wallet and the chore of paying becomes smooth.
207) Build a product that can use digital cards and refill them at will and the product becomes usable at all participating stores increasing adoption via partner networks
208) Build a product with partner sponsored incentives to customers and see the appeal grow as partner network endears itself to the customer more than it would have individually
209) Build a product with little or no ecosystem and the product fails to strike popularity in conferences, exhibitions and symposiums
210) Build a product with little user education and find that the sales booths at an exhibition are not visited.