Friday, August 18, 2017

We continue discussing the ZooKeeper. It is a co-ordination service with elements from group messaging, shared registers and distributed lock services. It provides a interface that guarantees wait-free property and FIFO execution of requests from each client.  Requests across all clients are also linearized.
We saw a few examples of primitives that are possible with ZooKeeper. These primitives exist only on the client side. These included implementing dynamic configuration in a distributed application, specifying group membership and to implement locks and double barriers. These have been put to use with applications such as Yahoo web crawler which uses this for its page fetching service. Katta, a distributed indexer uses this for coordination. Even high traffic Yahoo message broker which is a distributed publish-subscribe system uses this for managing configuration, failure detection and group membership.
The fetching service is part of the Yahoo crawler and involves a master that commands page fetching processes. The masters provide the fetchers with configuration, and the fetchers keep the master posted with status and health. ZooKeeper gives the advantage to recover from failures of masters, guaranteeing availability despite failures and decoupling  the client from the servers.
Similarly Katta divides the work into shards and a master assigns shards to slaves Katta uses this to track the status of slave servers and the master to handle reelecting a master. Katta uses the ZooKeeper to track and propagate the assignment of shards to slaves.
The servers of a Yahoo message broker use a shared-nothing distributed architecture which makes co-ordination essential for correct operation and ZooKeeper is used to manage the distribution of topics, deal with failures and control system processes.
#codingexercise
Given an index of an element in a sorted array with duplicates, count the number of duplicates
int GetCount(String input, char val)
{
int current = binary_search(input, 0, input.Length - 1, val);
if (current == -1) return 0;
int mid = current;
while (current >= 0 && input[current] == val) current--;
current++;
int start = current;
current = mid;
while (current <= input.Length-1 && input[current] == val)
          current++;
current--;
int end = current;
return end-start+1;
}

No comments:

Post a Comment