Wednesday, July 30, 2014

As we describe the inputs and the forwarder of SplunkLite.Net, its interesting to note that the events diffused in the input file follow a Markov chain. Consider a Markov chain and decompose it into communicating classes. For each positive recurrent communicating class C, we have a unique stationary distribution pi-c which assigns positive probability to each state in C. Let a be a state belonging to C and let pi-c be pi-a. An arbitrary stationary distribution pi must be a linear combination of these pi-C.
To define this more formally (Konstantopoulous) : To each positive recurrent communicating class C there corresponds a stationary distribution pi-C. Any stationary distribution pi is necessarily of the form pi = Sigma - over - C  ( alpha-C. pi - C) where alpha-C >= 0, such that Sum-c(alpha-C) = 1 
We say that the distribution is normalized to unity because the probabilities acts as weights and the sums of all these probabilities is one.
Here's how we prove that an arbitrary stationary distribution is a weighted combination of the unique stationary distributions.
Let pi is an arbitrary stationary distribution, If pi(x) is >  0 then x belongs to some positive recurrent class C. The conditional distribution of x over C is then given by pi(x | C) = pi(x) | pi (C) where the denominator is an aggregation of all x in C. To make x unique we have pi-(x|C) as equivalent to pi-C(x) where pi-x  aggregated over all C is alpha-C - the state we picked.  So we can write pi = alpha-c(pi-C) is an arbitrary stationary distribution pi is a linear combination of these unique stationary distributions.
Now we return to our steps to port the Splunk forwarder

No comments:

Post a Comment