We continue discussing the ZooKeeper. It is a co-ordination service with elements from group messaging, shared registers and distributed lock services. It provides a interface that guarantees wait-free property and FIFO execution of requests from each client. Requests across all clients are also linearized.
We were discussing the throughput of ZooKeeper when the system is saturated and with various injected failure.It was seen that the throughput tanked for failure and recovery of a follower, failure and recovery of a different follower, failure of the leader, failure of the two followers in the first two marks and recovery at the third mark, and recovery of the leader. The most dip in throughput occurred with the failures of the leader. On the other hand, failure of the followers is tolerated with a quorum and therefore the throughput falls only as much as the failing read requests Also the leader election algorithm helps mitigate this further. Third even if the followers take more time to recover, ZooKeeper is able to raise the throughput with the distribution of the load after recovery.
The latency of requests was also measured. A worker process creates and deletes new node except that the deletes are asynchronous. The number of worker process is varied but a large number of nodes are attempted. The requests processed per second seemed to increase with the number of the workers but decrease with the number of servers. The average request latency was found to be between 1.2ms - 1.4 ms.
A number of barriers was also executed sequentially to evaluate the behavior of primitives with ZooKeeper. For each barrier like a mutex, a client waits for all other clients before moving on to the instruction succeeding it. This is true for both entry and exit. The number of barriers were executed one after the other. The time to process the barriers increased linearly with the number of barriers which indicated that the concurrent access to the data tree does not hamper the execution. Also, the latency increased proportionally to the number of clients.
#codingexercise
Find the number of elements who have the same maximum number of duplicates in a contiguous sorted sequence
Solution:
1. For each element in a contiguous sequence
2. Insert the element, count of repetitions in a dictionary
3. Find the max count from the values in a dictionary
4. for each key-value pair in the dictionary
if the value == max count
print the key
We were discussing the throughput of ZooKeeper when the system is saturated and with various injected failure.It was seen that the throughput tanked for failure and recovery of a follower, failure and recovery of a different follower, failure of the leader, failure of the two followers in the first two marks and recovery at the third mark, and recovery of the leader. The most dip in throughput occurred with the failures of the leader. On the other hand, failure of the followers is tolerated with a quorum and therefore the throughput falls only as much as the failing read requests Also the leader election algorithm helps mitigate this further. Third even if the followers take more time to recover, ZooKeeper is able to raise the throughput with the distribution of the load after recovery.
The latency of requests was also measured. A worker process creates and deletes new node except that the deletes are asynchronous. The number of worker process is varied but a large number of nodes are attempted. The requests processed per second seemed to increase with the number of the workers but decrease with the number of servers. The average request latency was found to be between 1.2ms - 1.4 ms.
A number of barriers was also executed sequentially to evaluate the behavior of primitives with ZooKeeper. For each barrier like a mutex, a client waits for all other clients before moving on to the instruction succeeding it. This is true for both entry and exit. The number of barriers were executed one after the other. The time to process the barriers increased linearly with the number of barriers which indicated that the concurrent access to the data tree does not hamper the execution. Also, the latency increased proportionally to the number of clients.
#codingexercise
Find the number of elements who have the same maximum number of duplicates in a contiguous sorted sequence
Solution:
1. For each element in a contiguous sequence
2. Insert the element, count of repetitions in a dictionary
3. Find the max count from the values in a dictionary
4. for each key-value pair in the dictionary
if the value == max count
print the key
No comments:
Post a Comment