The native k8s events can also be transformed to custom events to suit the need of any other event processing engine. Typically, organizations have their own event gateway and event stores for making them proprietary such as for the use of dial home, network operations center and remote diagnostic sessions. This ability to transform events then let us do without reserving large storage as long as there is some buffering possible from the source.
It is this notion that can be extended to Extract-Transform-Load operations suitable to different downstream systems.
The difference between log forwarding and event forwarding becomes clear when the use of command line options for kube-apiserver is considered. For example, the audit-log-path option dumps the ausit events to a log file that cannot be accessed from within the Kubernetes runtime environment within the cluster. Therefore this option cannot be used with FluentD because that us a containerized workload. On the other hand, the audit-web-hook option allows the service to listen for callbacks from the Kubernetes control plane to the arrival of audur events. These service listening from Falco for this web hook endpoint is running in its own container as a Kubernetes service. The control plane makes only web request per audit event and since the events are forwarded over the http, the Falco service can efficiently handle the rate and latency of traffic.
The performance consideration between the two options is also notable. The log forwarding is the equivalent of running the rail command on the log file and forwarding it over TCP as netcat command. This utilizes the sand amount of data in transfers and uses a TCP connection although it does not traverse as many layers as the web hook. It is also suitable for Syslog drain that enables further performance improvements
The Webhook command is a push command and requires packing and unpacking of data as it traverses up and down the network layers. There is no buffering involved on the service side so there is a chance that some data will be lost as service goes down. The connectivity is also subject to faults more than the Syslog drain. However the http us best suited for message broker intake which facilitates filtering and processing that can significantly improve performance.
It is this notion that can be extended to Extract-Transform-Load operations suitable to different downstream systems.
The difference between log forwarding and event forwarding becomes clear when the use of command line options for kube-apiserver is considered. For example, the audit-log-path option dumps the ausit events to a log file that cannot be accessed from within the Kubernetes runtime environment within the cluster. Therefore this option cannot be used with FluentD because that us a containerized workload. On the other hand, the audit-web-hook option allows the service to listen for callbacks from the Kubernetes control plane to the arrival of audur events. These service listening from Falco for this web hook endpoint is running in its own container as a Kubernetes service. The control plane makes only web request per audit event and since the events are forwarded over the http, the Falco service can efficiently handle the rate and latency of traffic.
The performance consideration between the two options is also notable. The log forwarding is the equivalent of running the rail command on the log file and forwarding it over TCP as netcat command. This utilizes the sand amount of data in transfers and uses a TCP connection although it does not traverse as many layers as the web hook. It is also suitable for Syslog drain that enables further performance improvements
The Webhook command is a push command and requires packing and unpacking of data as it traverses up and down the network layers. There is no buffering involved on the service side so there is a chance that some data will be lost as service goes down. The connectivity is also subject to faults more than the Syslog drain. However the http us best suited for message broker intake which facilitates filtering and processing that can significantly improve performance.
No comments:
Post a Comment