Thursday, July 1, 2021

 A revisit to event-driven systems: 

Introduction: Software applications often make use of event-driven systems in addition to synchronous calls between components and services. This article revisits some of these discussions. 

Description: First, let us define the event-driven system. It is state based. When the state changes, the system is expected to take certain actions following a finite automaton. This kind of system knows only how to proceed from one state to another and the state transitions must be determined beforehand and validated to not run into endless loops or no activity. The states hold meaning for the application calling the system. 

There are several advantages to an event-driven system. Protocol handlers are a notable example of such a system. They do not have to remember the caller or maintain a session and can be entirely event-based. Message and requests arriving over the wire can be a representation of control plane state transition requests. This form of system is highly scalable. 

Events must be well-defined. Their scope and granularity determine the kind of actions taken. Event-driven systems can perform stateful processing. They can persist the states to allow the processing to pick up where it left off. The states also help with fault tolerance. This persistence of state protects against failures including data loss. The consistency of the states can also be independently validated with a checkpointing mechanism available from Flink. The checkpointing can persist from the local state to a remote store. Stream processing applications often take in the incoming events from an event log. Therefore, this event log stores and distributes event streams written to durable append-only log on tier 2 storage where they remain sequential by time. Flink can recover a stateful streaming application by restoring its state from a previous checkpoint. It will adjust the read position on the event log to match the state from the checkpoint. Stateful stream processing is therefore not only suited for fault tolerance but also reentrant processing and improved robustness with the ability to make corrections. Stateful stream processing has become the norm for event-driven applications, data pipeline applications, and data analytics applications. 


No comments:

Post a Comment