We continue discussing the ZooKeeper. It is a co-ordination service with elements from group messaging, shared registers and distributed lock services. It provides a interface that guarantees wait-free property and FIFO execution of requests from each client. Requests across all clients are also linearized.
We were discussing the throughput of ZooKeeper when the system is saturated and with various injected failure.The most dip in throughput occurred with the failures of the leader. On the other hand, failure of the followers is tolerated with a quorum and the leader election algorithm helps mitigate this further.
The latency of requests was also measured. The requests processed per second seemed to increase with the number of the workers but decrease with the number of servers. The average request latency was found to be between 1.2ms - 1.4 ms.
We conclude with discussion of related work as cited by the authors. They mention Chubby which also uses a file system interface and an agreement protocol to guarantee the replicas but it is a lock service. Clients using ZooKeeper can choose to implement locks. Also Chubby only allows clients to connect to the leader and not with any other server. ZooKeeper has better performance and a more relaxed consistency model
Some systems focus on fault-tolerance such as ISIS which transforms abstract type specifications into fault tolerant distributed objects thus making fault tolerance mechanism transparent to users. Other systems like Totem guarantee order of messages in an architecture that exploits hardware broadcasts of local area networks. ZooKeeper also implements the notion of synchronization on a virtual timeline and ordering of requests. ZooKeeper also supports a variety of network topology.
Some systems utilize a state machine replication as for example, Paxos that combines transaction logging for consensus with write-ahead logging for data recovery Some replicated state machines are fully Byzantine tolerant. ZooKeeper is not so but it can be made one without modifying the server code. Boxwood uses Paxos to form a distributed lock service but it is a higher level primitive while ZooKeeper does not restrict clients from having different primitives. Sinfonia introduced mini-transactions . a new paradigm for building scalable distributed systems. Sinfonia has been designed to store application data but ZooKeeper stores application metadata.Moreover ZooKeeper can add watches where as Sinfonia cannot. Dynamo allows clients to put data in a distributed key - value store. The key space in Dynamo is not hierarchical unlike ZooKeeper which also provides better consistency and durability guarantees.
#codingexercise
Find the number of elements who have the same minimum number of duplicates in a contiguous sorted sequence
Solution:
1. For each element in a contiguous sequence
2. Insert the element, count of repetitions in a dictionary
3. Find the min count from the values in a dictionary
4. for each key-value pair in the dictionary
if the value == min count
print the key
This can be improved without use of a hash table by retaining only a single key value pair that is updated when the value is lower than the previous. A count is maintained for every match with the key value pair and reset when the key value pair changes.
#Fraud detection service introduction: https://1drv.ms/w/s!Ashlm- Nw-wnWsEv9woJ7ynzJAPpv
We were discussing the throughput of ZooKeeper when the system is saturated and with various injected failure.The most dip in throughput occurred with the failures of the leader. On the other hand, failure of the followers is tolerated with a quorum and the leader election algorithm helps mitigate this further.
The latency of requests was also measured. The requests processed per second seemed to increase with the number of the workers but decrease with the number of servers. The average request latency was found to be between 1.2ms - 1.4 ms.
We conclude with discussion of related work as cited by the authors. They mention Chubby which also uses a file system interface and an agreement protocol to guarantee the replicas but it is a lock service. Clients using ZooKeeper can choose to implement locks. Also Chubby only allows clients to connect to the leader and not with any other server. ZooKeeper has better performance and a more relaxed consistency model
Some systems focus on fault-tolerance such as ISIS which transforms abstract type specifications into fault tolerant distributed objects thus making fault tolerance mechanism transparent to users. Other systems like Totem guarantee order of messages in an architecture that exploits hardware broadcasts of local area networks. ZooKeeper also implements the notion of synchronization on a virtual timeline and ordering of requests. ZooKeeper also supports a variety of network topology.
Some systems utilize a state machine replication as for example, Paxos that combines transaction logging for consensus with write-ahead logging for data recovery Some replicated state machines are fully Byzantine tolerant. ZooKeeper is not so but it can be made one without modifying the server code. Boxwood uses Paxos to form a distributed lock service but it is a higher level primitive while ZooKeeper does not restrict clients from having different primitives. Sinfonia introduced mini-transactions . a new paradigm for building scalable distributed systems. Sinfonia has been designed to store application data but ZooKeeper stores application metadata.Moreover ZooKeeper can add watches where as Sinfonia cannot. Dynamo allows clients to put data in a distributed key - value store. The key space in Dynamo is not hierarchical unlike ZooKeeper which also provides better consistency and durability guarantees.
#codingexercise
Find the number of elements who have the same minimum number of duplicates in a contiguous sorted sequence
Solution:
1. For each element in a contiguous sequence
2. Insert the element, count of repetitions in a dictionary
3. Find the min count from the values in a dictionary
4. for each key-value pair in the dictionary
if the value == min count
print the key
This can be improved without use of a hash table by retaining only a single key value pair that is updated when the value is lower than the previous. A count is maintained for every match with the key value pair and reset when the key value pair changes.
#Fraud detection service introduction: https://1drv.ms/w/s!Ashlm-
No comments:
Post a Comment