This is a continuation of a series of articles on Microsoft Azure from an operational point of view that surveys the different services from the service portfolio of the Azure public cloud. The most recent article on Service Fabric discussed the Dataverse and solution layers. This document talks about merging the layers.

Microsoft Dataverse is a data storage and management system for the various Power Applications so that they are easy to use with Power Query. The data is organized in tables some of which are built-in and standard across applications, but others can be added on a case-by-case basis for applications. These tables enable applications to focus on their business needs while providing a world class, secure and cloud-based storage option for the data that are 1. Easy to manage, 2. Easy to secure, 3. Accessible via Dynamics 365, has rich metadata, logic and validation, and come with productivity tools. Dynamics 365 applications are well-known for enabling businesses to quickly meet their business goals and customer scenarios and Dataverse makes it easy to use the same data across different applications. It supports incremental and bulk loads of data both on a scheduled and on-demand basis.

Solutions are used to transport applications and components from one environment to another or to add customizations to an existing application. It can comprise applications, site maps, tables, processes, resources, choices, and flows. It implements Application Lifecycle management and powers Power Automate. There are two types of solutions (managed and unmanaged) and the lifecycle of a solution involves create, updates, upgrade and patch.

Managed and unmanaged solutions can co-exist at different levels within a Microsoft Dataverse environment. They form two distinct layer levels. What the user sees as runtime behavior, comes from the active customizations of an unmanaged layer which in turn might be supported by a stack of one or more user-defined managed solutions and system solutions in the managed layer.  Managed solutions can also be merged. The solution layers feature enables one to see all the solution layers for a component.

Merge behavior is important because it affects the same component when a solution maker should understand the merge behavior when a solution is updated or when multiple solutions are installed that affect the same component. Merging involves only mobile-driven applications, forms and sitemap component types. All other components uses the “top level wins” behavior. The “top wins” behavior is one where the layers that reside at the top determine how the component works at app runtime. A top layer can be introduced by a staged pending upgrade. This behavior can be demonstrated with an example where the current top layer has a property say maximum length=100 and a solution upgrade is performed which adds a layer with the maximum length of 150.  In this case, the affected target will permit a maximum length of 150.

Update and Upgrade merge behavior are applied after they are stacked on top of the base solution. These can be merged by selecting “Apply upgrade” from the Solutions areas in the Power Application, which flattens the layers and creates a new base solution.  When the solution is prepared for distribution, the recipients may already have multiple solutions. Using segmented solutions to achieve isolated solutions is recommended.

Solution segmentation involves exporting solution updates with selected entity assets rather than entity fields, forms and views rather than entire entities with all the assets. A segmented solution is created by selecting one or more options when adding an existing entity to the solution. These include the option to add no components or metadata so that only the minimal entity solution information is added to the solution. Another option is to select components associated with the entity such as fields so that only those that have been added or changed are include in the solution update. Similarly, there is an option to include only the metadata associated with the entity but not the components. Lastly, there is an option to include all components and metadata associated with an entity.

 

Continuous root cause analysis via analysis of time-series events:  

 

Problem statement: Given a method to collect many data points for errors in logs, can there be prediction on the resolution time of the next root-cause   

 

Solution: There are two stages to solving this problem:

1.       Stage 1 – discover root cause and create a summary to capture it

2.       Stage 2 – use a time-series algorithm to predict the relief time.

 

Stage 1:

We start with the hidden weighted matrix that the neural network layer generates and then use that hidden layer to determine the salience using the gradient descent method.     

 

All values are within [0,1] co-occurrence probability range.    

 

The solution to the quadratic form representing the embeddings is found by arriving at the minima represented by Ax = b using conjugate gradient method.  

We are given input matrix A, b, a starting value x, a number of iterations i-max and an error tolerance  epsilon < 1  

 

This method proceeds this way:   

 

set I to 0   

 

set residual to b - Ax   

 

set search-direction to residual.  

 

And delta-new to the dot-product of residual-transposed.residual.  

 

Initialize delta-0 to delta-new  

 

while I < I-max and delta > epsilon^2 delta-0 do:   

 

    q = dot-product(A, search-direction)  

 

    alpha = delta-new / (search-direction-transposed. q)   

 

    x = x + alpha.search-direction  

 

    If I is divisible by 50   

 

        r = b - Ax   

 

    else   

 

        r = r - alpha.q   

 

    delta-old = delta-new  

 

    delta-new = dot-product(residual-transposed,residual)  

 

     Beta = delta-new/delta-old  

 

     Search-direction = residual + Beta. Search-direction  

 

     I = I + 1   

 

Root cause capture – Exception stack traces that are captured from various sources and appear in the logs can be stack hashed. The root cause can be described by  a specific stacktrace, its associated point of time, the duration over which it appears and the time of fix introduced, if known. 

 

Stage 2: A time-series algorithm does not need any attributes other than the historical collection of relief times to be able to predict the next relief time. It only looks at scalar value regardless of the type or factors playing into the relief time of an individual incident or its root cause attributes. The historical data is utilized to predict an estimation on the incoming event as if the relief were a scatter plot along the timeline. Unlike other data mining algorithms that involve additional attributes of the event, this approach uses a single auto-regressive method on the continuous data to make a short-term prediction. The regression is automatically trained as the data accrues

 Gut Feelings: The Microbiome and our health. 

This is a book written by Susie Flaherty and Our Health and is written by MIT press, 2022 

The recent epidemic notwithstanding, the next generations seem more prone to epidemics of chronic, non-infectious, inflammatory diseases.  There is a radical change required in the health care model as suggested by the authors to counter that by centering on the emerging body of research into the human microbiome. Rather than treat the microorganisms that populate the people’s guts as enemies, there must be better understanding of the complex ecosystem each human carries within, and work to create more holistic, preventative health interventions.  

Microorganisms have tremendous potential. This has caused medicine and science to take note.  When people view microorganisms through the lens of germ theory and infectious diseases, they treat it as a conflict between microbial ecosystems. The treatment is usually to prescribe antibiotics without discriminating between the microorganisms. Even if microorganisms contribute to diseases, they live in symbiosis with human hosts and can be instrumental in treating diseases ranging from cancer to neurological conditions. The authors content that microorganisms live in complex sophisticated civilizations that we did not appreciate yet. 

The Human Genome project made significant strides over three decades in addressing chronic inflammatory illnesses via better understanding genetics’ role in those diseases. Yet only 2% of the potential therapeutic targets have been addressed. They call the coexistence of humans and microorganisms a “coevolutionary destiny” 

Our lifestyle and the environment affect the microbiome’s composition which plays a significant role in the immune system function.  The microbiome extends beyond our body and connects us to a global ecosystem of microbes that live everywhere from soil to other creatures. The implications on our health and disease can be understood only when we consider the whole of the earth’s ecosystem as a continuous circle of life. 

Environmental factors that deplete the diversity of gut microbiota trigger poor health. It has been seen in some cases that when people change their dietary habits, overuse antibiotics, and overly sanitize human environments, they lose ancient microorganisms. This forms the hypothesis for “missing microbes” by Martin Blaser. For example, the East African hunter-gatherer tribe – whose diets and lifestyles resemble that of our predecessors, host several beneficial microorganisms that are found absent in other demographics. 

This forms the basis for the suggestion that the prevalence of chronic inflammatory disease in the West with a loss of microbial diversity which can then be addressed by fortifying the probiotics.  But there is no study that can differentiate the probiotics and map the human microbiome as the scientists did with the genome.  

The Human Microbiome project has had five primary aims since its inception which include: 

1. Isolate and identify microbial genome sequences, much like the Human Genome Project. 

2. Establish whether a “core microbiome” exists  

3. Find the relationship between alterations in the microbiome and disease. 

4. Develop new technologies and tools  

5. Reflect on the legal, ethical and social dimensions of microbiome sequencing. 

Good health outcomes entail balancing a complex ecosystem of microorganisms. This is evident in the conditions of homeostasis where there is a balance between states of health and disease.  This approach is paralleled and employed by holistic healing modalities – such as traditional Chinese medicine.  

The microbiome includes microorganisms such as bacteria, fungi, viruses, parasites, protozoa, archaea and yeast. So, bacteria is not the only kind that needs to be studied. Viruses or “virome” are also a key contributor to disease and its effects on health.  

The authors suggest a framework to consider much more complex and multidimensional explanations of the drivers of human health and disease with the notion of “five pillars” which are: genetic predisposition; exposure to certain environmental factors such as pollution; depleted mucosal barrier; immune system dysregulation; and a lack of balance in the microbiome. The easiest to manipulate are gut permeability, immune system function, and the microbiome.  

The gut microbiome influences the development or severity of many disease symptoms including the Gut inflammatory disorders, Obesity, Autoimmunity, Neurological and behavioral disorders, Cancer. A preventative approach to health care will employ predictive computational models.  

The authors argue that we have the power to stop these epidemics of non-infectious, chronic inflammatory diseases, if our scientific discoveries could be put in the microbiome domain at the service of the public health policies.  

The future holds promise with: 

Prebiotics trigger the activity of the colon bacteria. 

Probiotics ingest bacteria such as lactobacillus that people can use to balance the microbiome and boost immune function.  

Correlations are being mapped between strains of probiotics and the treatment of specific diseases. 

Synbiotics – the synergistic combination of prebiotics and postbiotics which can ensure desirable strains of bacteria colonize and survive in the gut. 

Postbiotics – include organic acids, peptides and enzymes. One way of consuming postbiotics is through fermenting foods.  

Psychobiotics – Microbiome research could result in the creation of a new class of probiotics to treat nervous system diseases such as mental health disorders and neurodegenerative diseases. 

Emerging health strategies include everything from prebiotics to psychobiotics. 

The authors summarize that these emerging fields of science benefit demand our investments in our time, talent, and resources to develop a roadmap for translating current and future scientific information into implementable clinical interventions that could change our collective destiny for the better.