Configuring a PKS cluster for auditing:
In a previous post, we described that the auditing framework for any Kubernetes cluster can be enabled by specifying the audit flags in the command line to the kube-apiserver. Most systems facilitate that by ssh to the host running the concerned server.
In this document, we take a closer look at the PKS cluster. PKS facilitates this with the help of BOSH Command line interface which can be run from any PKS development environment. We gather the credential and IP address information of the BOSH director and SSH into the Ops Manager VM.
We create the BOSH alias for the PKS environment with the ip address and the credentials so it is easy to use with the commands. The “pks login” is one such command which lets us login to the BOSH director.
The “PKS deployments” is another command we run to get the name and the hash of the deployment. We can use this to get the VMs name. To SSH into the PKS VM, we run the “ssh pks-vm” command from BOSH. The “cloud-check” command in particular is very helpful to detect the differences between the VM state database maintained by the BOSH director and the actual state VMs. It also helps reboot, recreate or delete references.
The PKS database stores information on the pods which includes watermark and consumption. Watermark is the number of pods that can run at a single time. Consumption is the memory and CPU usage of the pods. This database can also be accessed from the PKS VM. The billing database can be accessed with pre-defined credentials which can then be used with SQL queries.
PKS can also be monitored with sinks. RFC 5424 describes log ingress over TCP and introduces the notion of a sink. These sink resources help PKS to send the logs to that destination. Logs as well as events can use a shared format. The Kubernetes API events are denoted by the string “k8s.event” and with their “APP-NAME” field. A typical Kubernetes API event includes the host ID of the BOSH VM, the namespace and the Pod-ID as well. Failure to retrieve containers from Registry is specified with an identifying string of “Error: ErrImagePull”. Malfunctioning containers are denoted with “Back-off restarting failed container” in their events. Successful scheduling of containers have “Started container” in their events.
The logs for any cluster can also be downloaded from the PKS VM using the BOSH CLI command such as “logs pks/0”
Kubernetes master node VMs also have etcd an open source distributed key value store which it uses for service discovery and configuration sharing. The etcd also has metrics which help cluster health monitoring.
In a previous post, we described that the auditing framework for any Kubernetes cluster can be enabled by specifying the audit flags in the command line to the kube-apiserver. Most systems facilitate that by ssh to the host running the concerned server.
In this document, we take a closer look at the PKS cluster. PKS facilitates this with the help of BOSH Command line interface which can be run from any PKS development environment. We gather the credential and IP address information of the BOSH director and SSH into the Ops Manager VM.
We create the BOSH alias for the PKS environment with the ip address and the credentials so it is easy to use with the commands. The “pks login” is one such command which lets us login to the BOSH director.
The “PKS deployments” is another command we run to get the name and the hash of the deployment. We can use this to get the VMs name. To SSH into the PKS VM, we run the “ssh pks-vm” command from BOSH. The “cloud-check” command in particular is very helpful to detect the differences between the VM state database maintained by the BOSH director and the actual state VMs. It also helps reboot, recreate or delete references.
The PKS database stores information on the pods which includes watermark and consumption. Watermark is the number of pods that can run at a single time. Consumption is the memory and CPU usage of the pods. This database can also be accessed from the PKS VM. The billing database can be accessed with pre-defined credentials which can then be used with SQL queries.
PKS can also be monitored with sinks. RFC 5424 describes log ingress over TCP and introduces the notion of a sink. These sink resources help PKS to send the logs to that destination. Logs as well as events can use a shared format. The Kubernetes API events are denoted by the string “k8s.event” and with their “APP-NAME” field. A typical Kubernetes API event includes the host ID of the BOSH VM, the namespace and the Pod-ID as well. Failure to retrieve containers from Registry is specified with an identifying string of “Error: ErrImagePull”. Malfunctioning containers are denoted with “Back-off restarting failed container” in their events. Successful scheduling of containers have “Started container” in their events.
The logs for any cluster can also be downloaded from the PKS VM using the BOSH CLI command such as “logs pks/0”
Kubernetes master node VMs also have etcd an open source distributed key value store which it uses for service discovery and configuration sharing. The etcd also has metrics which help cluster health monitoring.
No comments:
Post a Comment