Saturday, July 25, 2020

Support for small to large footprint introspection database and query.



Introspection is runtime operational states, metrics, notifications and smartness saved from the ongoing activities of the system. Since the system can be deployed in different modes and with different configurations, this dedicated container and query can take different shapes and forms. The smallest footprint may merely be similar to log files or json data while the large one can even open up the querying to different programming languages and tools. Existing hosting infrastructure such as container orchestration framework and etcd database may provide excellent existing channels of publications that can do away with this store and query altogether but they rely highly on network connectivity while we took the opportunity to discuss that Introspection datastore does not compete but only enhances offline production support. 
The depth of introspection is also unparalleled with dynamic management views that is simply not possible with third party infrastructure. The way to get the information is probably best known only to the storage system and improves the offering from the storage product. 
There are ways to implement the introspection suitable to a size of deployment of the overall system. Mostly these are incremental add-ons with the growth of the system except for the extremes of the system deployments of tiny and Uber. For example, while introspection store may save runtime only periodically for small deployments, it can be continuous for large deployments. The runtime information gathered could also be more expansive as the size of the deployment of the system grows. The gathering of runtime information could also expand to more data sources as available from a given mode of deployment from the same size. The Kubernetes mode of deployment usually has many deployments and statefulsets and the information on those may be available from custom resources as queried from kube-apiserver and etcd database. The introspection store is a container within the storage product so it is elastic and can accommodate the data from various deployment modes and sizes. Only for the tiniest deployments where repurposing a container for introspection cannot be accomodated, a change of format for the introspection store becomes necessary. On the other end of the extreme, the introspection store can include not just the dedicated store container but also snapshots of other runtime data stores as available. A cloud scale service to virtualize these hybrid data stores for introspection query could then be provided. These illustrate a sliding scale of options available for different deployment modes and sizes. 
Another innovation for introspection store is dynamic instantiation on administrator request. When the deployment size of the overall system is small, the size of the introspection store might be highly restricted. It might benefit to instantiate the store only as and when needed. If the store were considered to be a file, this is the equivalent of writing the diagnostics operation information out to the file. Higher end deployments can repurpose a stream to store the periodically or continuously collected operation information. Smaller deployments can materialize the store for the administrator on demand.
 for small to large footprint introspection database and query. 
Introspection is runtime operational states, metrics, notifications and smartness saved from the ongoing activities of the system. Since the system can be deployed in different modes and with different configurations, this dedicated container and query can take different shapes and forms. The smallest footprint may merely be similar to log files or json data while the large one can even open up the querying to different programming languages and tools. Existing hosting infrastructure such as container orchestration framework and etcd database may provide excellent existing channels of publications that can do away with this store and query altogether but they rely highly on network connectivity while we took the opportunity to discuss that Introspection datastore does not compete but only enhances offline production support. 
The depth of introspection is also unparalleled with dynamic management views that is simply not possible with third party infrastructure. The way to get the information is probably best known only to the storage system and improves the offering from the storage product. 
There are ways to implement the introspection suitable to a size of deployment of the overall system. Mostly these are incremental add-ons with the growth of the system except for the extremes of the system deployments of tiny and Uber. For example, while introspection store may save runtime only periodically for small deployments, it can be continuous for large deployments. The runtime information gathered could also be more expansive as the size of the deployment of the system grows. The gathering of runtime information could also expand to more data sources as available from a given mode of deployment from the same size.  The Kubernetes mode of deployment usually has many deployments and statefulsets and the information on those may be available from custom resources as queried from kube-apiserver and etcd database. The introspection store is a container within the storage product so it is elastic and can accommodate the data from various deployment modes and sizes. Only for the tiniest deployments where repurposing a container for introspection cannot be accomodated, a change of format for the introspection store becomes necessary. On the other end of the extreme, the introspection store can include not just the dedicated store container but also snapshots of other runtime data stores as available. A cloud scale service to virtualize these hybrid data stores for introspection query could then be provided. These illustrate a sliding scale of options available for different deployment modes and sizes. 
Another innovation for introspection store is dynamic instantiation on administrator request. When the deployment size of the overall system is small, the size of the introspection store might be highly restricted. It might benefit to instantiate the store only as and when needed. If the store were considered to be a file, this is the equivalent of writing the diagnostics operation information out to the file. Higher end deployments can repurpose a stream to store the periodically or continuously collected operation information. Smaller deployments can materialize the store for the administrator on demand.

No comments:

Post a Comment