Support for small to large footprint introspection database and query

Another innovation for introspection store is dynamic instantiation on administrator request. When the deployment size of the overall system is small, the size of the introspection store might be highly restricted. It might benefit to instantiate the store only as and when needed. If the store were considered to be a file, this is the equivalent of writing the diagnostics operation information out to the file. Higher end deployments can repurpose a stream to store the periodically or continuously collected operation information. Smaller deployments can materialize the store for the administrator on demand.

Similarly, transformation and preparation of system collected information can happen within the introspection store for the high-end systems and offloaded to external stacks at the lower end. The exposure of introspection datastore in external data pipelines is made possible with direct access or copying. 

These results are also useful towards statistical processing from historical trends that may require special purpose logic that would not necessarily need to be rewritten outside the store by consumers. If the logic involves external analytical packages, it can be made available via the analysis platform that the regular data is used with.

The introspection store could also benefit with its own importers and exporters for high-end systems so that it remains the source of truth for a variety of audience. Administrators are not the only ones to benefit from introspection store. Users and applications of the system can access the information in the store via programmatic or tool-based means so long as this shared container cannot be polluted by their access. Existing health reporting, monitoring and metrics may derive secondary information providing methods from this store for users to continue getting the benefit when they don’t need to know that such a store exists but are interested in querying specific runtime information of the system.

Appenders and relayers are patterns that allow simultaneous and piped conversion of data to different formats. A Log appender writes data to console, file and other destinations simultaneously. Text data can be formatted through a json relayer to form json which can then be converted to xml in staged progression. These mechanisms also apply to data in introspection store and can be performed by services operating on the store.

Full-Service stores come with the benefit that the data is not only prepared, saved and sent from the store but it can also be extracted, transformed and loaded in ways suitable to subscribers. The preparation of data in different data stores including the Introspection data store via common publishers from the system itself is a feature that enables consumers to get more from the same data so they have to do little, if any work. The ability to take on services on behalf of the audience by the product/solution is referred to as full-service. The introspection store can certainly participate in full-service solutions.