Wednesday, September 18, 2019

There are a few caveats to observe when processing events as opposed to other resources with webhooks. 

Events can come as a cascade, we cannot cause undue delay in the processing of any one of them. Events can cause the generation of other events. This would require stopping additional events from generating if the webhook is responsible for transforming and creating new events that cause those additional events. The same event can occur again, if the transformed event triggers the original event or if the transformed events ends up again in the webhook because the transformed event is also of the same kind as the original event. The webhook has to skip the processing of the event it has already handled. 

The core resources of Kubernetes all are subject to the same verbs of Get, create, delete, etc. However, events are far more numerous and occur at a fast clip than some of the other resources. This calls for handling and robustness equivalent to the considerations in a networking protocol or a messaging queue. The webhook is not designed to handle such rate or latency even though it might skip a lot of the events. Consequently, selector labels and processing criteria become all the more important. 

The performance consideration of the event processing by webhooks aside, a transformation of an event by making an http request to an external service also suffers from outbound side considerations such as the timeouts and retries on making the http request. Since performance of the webhooks of events has been called out above as significantly different from the few and far in between processing for other resources, the webhook might have to make do with lossy transformation as opposed to a persistence-based buffering.  

None of the above criteria should matter for the usage of this kind of webhook for the purpose of diagnostics which is most relevant to troubleshooting analysis. Diagnostics have no requirement on performance from webhook. When the events are already filtered to audit events, and when the conversion of those events is even more selective, the impact on diagnostics is little or none from changes in the rate and delay of incoming events. 

No comments:

Post a Comment