Wednesday, November 23, 2016

We started discussing the paper "Nested sampling  for general Bayesian computation" by Skilling. Nested sampling estimates directly how the likelihood function relates to previous value of the discrete random variable. It can be used to compare two data models for the same set by comparing the conditional probability that is assigned after the relevant evidence is taken into account.
This method directly computes the evidence.The calculation of the evidence allows different model assumptions to be compared through the ratios of evidence values known as Bayes factors.
This is an improvement over Markov Chain Monte Carlo methods discussed in earlier posts because they yielded a set of samples representing the normalized posterior and the evidence was of secondary importance. Here the evidence is the direct result and  calculated with far greater ease than MCMC methods. The parameters are sorted by their likelihood values and then summed up to give the evidence. Since there are many data points, this method performs nested sampling to simulate the operation statistically. The evidence is then associated with corresponding numerical uncertainity. Skilling states that this nested sampling is Bayesian in nature. Furthermore, with the nested samples, we can estimate the density of states, to obtain samples from the posterior and to quantify arbitrary properties of the parameter.
This technique simplifies the evidence calculation by not summing over the parameters directly but instead performing it on the cumulated prior mass  that cover likelihood values greater than say lambda. As Lambda increases, the enclosed mass decreases from one to zero.  Therefore the summation simplifies to a one dimensional integral over unit range.
This is the primary intuition behind this paper. We will consider to make it online instead of batch.

This method is implemented in Python here and is not part of the standard numpy package yet.
http://www.inference.phy.cam.ac.uk/bayesys/python/mininest.py

#codingexercise
Find the number of BSTs possible for numbers ranging from 1 to n.
For example, n = 2, # = 2
n= 3, #=5
        static List<Node> makeBSTs(int start, int end)
        {
            var items = new List<Node>();
            if (start > end)
            {
                items.Add(null);
                return items;
            }
            for (int i = start; i <= end; i++)
            {
                var left = makeBSTs(start, i - 1);
                var right = makeBSTs(i + 1, end);
                if (left.Count == 0 && right.Count == 0)
                {
                    var node = new Node();
                    node.data = i;
                    node.left = null;
                    node.right = null;
                    items.Add(node);
                    continue;
                }
                if (left.Count == 0)
                {
                    for (int k = 0; k < right.Count; k++)
                    {
                        var rightnode = right[k];
                        var node = new Node();
                        node.data = i;
                        node.left = null;
                        node.right = rightnode;
                        items.Add(node);
                    }
                    continue;
                }
                if (right.Count == 0)
                {
                    for (int k = 0; k < left.Count; k++)
                    {
                        var leftnode = left[k];
                        var node = new Node();
                        node.data = i;
                        node.left = leftnode;
                        node.right = null;
                        items.Add(node);
                    }
                    continue;
                }
                for (int j = 0; j < left.Count; j++)
                {
                    var leftnode = left[j];
                    for (int k = 0; k < right.Count; k++)
                    {
                        var rightnode = right[k];
                        var node = new Node();
                        node.data = i;
                        node.left = leftnode;
                        node.right = rightnode;
                        items.Add(node);
                    }
                    continue;
                }

            }
            return items;
        }

we can verify the BST formed with
foreach( var root in items){
        printBST(root);
        Console.WriteLine();
}
which should print in ascending order
where the printBST is inorder traversal as follows:
void printBST(Node root)
{
if (root)
{
printBST(root.left);
Console.Write("{0} ", root.data);
print.BST(root.right);
}
}
which in this case verifies the above code.

No comments:

Post a Comment