We continue discussing the paper "Nested sampling for general Bayesian computation" by Skilling
With this technique, nested sampling estimates directly how the likelihood function relates to previous value of the discrete random variable. And it can be used to compare two data models for the same set by comparing the conditional probability that is assigned after the relevant evidence is taken into account.
This method directly computes the evidence. This technique simplifies the evidence calculation by not summing over the parameters directly but instead performing it on the cumulated prior mass that cover likelihood values greater than say lambda. As Lambda increases, the enclosed mass X decreases from one to zero. We were looking at the transformation of computing the evidence based on the prior mass instead of the parameters. We looked at the integration performed. This paper essentially says don't navigate the parameter space. It is sufficient to explore a likelihood weighted space
The method replaces the point of the lowest likelihood by new one drawn from within the likelihood
It increments the evidence by filling in the missing band with weight w = Xj/N for each surviving point. The random values in the range 0,1 suffice.to draw the samples within the constrained likelihood contour. This method also calculates the information H during the iterations as a by product.
There are j iteartive steps. At each step, the procedure has points theta1, theta2, ... thetaN, with corresponding likelihoods L(theta1), L(theta2) .. L(thetaN). The lowest such value Li is the likelihood L associated with step i. Initially the evidence is zero and prior mass is one.
In each iteration, we set the prior mass as exp(-i/N) crude and weight Xi-1 - Xi
The evidence is then incremented by the quantity Li.wi
The point of lowest likelihood is then replaced by the new one drawn from within L(theta) > Li, in proportion to the prior pi(theta)
After all the iterations, we increment the evidence by the sum of the corresponding likelihoods L(theta1), L(theta2) .. L(thetaN) with the weight N^-1 Xj which fills in the missing band of the integral with the chosen weight for each surviving point now that the domain is between 0 and Xj. The weighted summation for evidence is therefore performed for j iterations and N points (j+N).
The paper also shows an example with j = 5 and N = 3 points. Initially the three points are taken from the entire parameter space which is equivalent to taking randomly over X from 0 to 1. Their labels are not known. The worst of these is taken and replaced so that all three are then uniform in a reduced range X1. In step 2, the new point happened to be inner most and is identified as number 5. After five such iterations there are five such discarded points and three survivors. It is over these points that the weighted sum is then evaluated.
Coding exercises updated
: https://1drv.ms/w/s!Ashlm-Nw-wnWljZZqmcHRUZNx9PF
Node GetParent(Node root, Node child)
{
If (child == null) return null;
Node parent = null;
While ( root )
{
If (child.val == root.val) return parent;
parent = root;
If (child.val < root.val)
Root = root.left;
Else
Root = root.right;
}
Return null;
}
With this the GetPredecessor changes to
Get predecessor of binary node
Node GetPredecessor(Node root)
{
if (root == NULL) return root;
if (root.left )
{
Node current = root.left;
While (current && current.right)
Current = current.right;
Return current;
}
Node parent = GetParent(root);
While (parent && parent.left == root)
{
root = parent;
parent = GetParent(root, parent);
}
return parent;
}
