We continue with a detailed study of Microsoft Azure stack as inferred from an introduction of Azure by Microsoft. We reviewed some more features of Azure storage. We saw how erasure coding works in Windows Azure Storage aka WAS and how its different from Pelican. We saw what steps WAS takes to improve durability. We also saw the benefits of journaling. We reviewed the partition layer which translates objects such as blob, table or queue to storage. We saw how the partition manager maintains a massive Object Table that is sharded on different partition servers which maintain their consistency across concurrent transactions using a lock service. We read the partition layer data model and the supported data types and operations, the partition layer architecture and the range partition data structures. we reviewed the load balancing, split and merge operations. Then we reviewed the intrastamp and interstamp replication. We also took stock of TCO, performance, application profiles and their comparision study
Today we conclude reviewing the list of design designs taken for the Windows Azure Service. This is primarily an append-only system which simplifies snapshot and replication because snapshots of previous states can now be kept at little or no cost. To keep the space overhead low, a Garbage Collection System is crucial.
To maintain integrity of user data, WAS needed to use end to end checksums. Checksum is padded onto the data from the moment it is received in the system.
Another design decision was to spread servers evenly across fault and upgrade domains. This way if the fault domain goes down, only a fraction of the servers is lost. Upgrade domains may be a different fraction.
Yet another decision was to support multiple data abstractions from a single stack. Blobs, Tables and Queues are all supported on the same stack.
We will recall that blobs, tables and queues were all tracked with the same master Object Table. This system defined abstraction reduces management by the system.
Storage is handled in buckets of 100 TB. WAS limis the amount of storage for an account to be no more than 100TB which allows all storage data pertaining to an account to fit within a given storage stamp. To obtain more storage capacity within a single data center, customers use more than one account within that location. This was considered alright for large customers because they already had multiple accounts.
WAS mentions that it seems to defy CAP theorem WAS provides high availability with strong consistency guarantees while the theorem states that high availability, consistency and partition tolerance may not be achieved at the same time.
Lastly, WAS uses extensive debug logging infrastructure and the logs are maintained on local disks.
The WAS System has also been stress tested using pressure points.
#codingexercise
To count total number of N digit numbers whose sum of even and odd digits are both even, we modify it as follows:
long GetCountEqual(int digits, int even, int odd, int n, bool isOdd)
{
if (digits == n) return (even % 2 == 0 && odd % 2 == 0);
long count = 0;
if (isOdd){
for (int i = 0; i <= 9; i++)
count += GetCountEqual(digits+1, even, odd+i, n, false);
}else{
for (int i = 0; i <= 9; i++)
count+= GetCountEqual(digits+1, even+i, odd, n, true);
}
return count;
}
Today we conclude reviewing the list of design designs taken for the Windows Azure Service. This is primarily an append-only system which simplifies snapshot and replication because snapshots of previous states can now be kept at little or no cost. To keep the space overhead low, a Garbage Collection System is crucial.
To maintain integrity of user data, WAS needed to use end to end checksums. Checksum is padded onto the data from the moment it is received in the system.
Another design decision was to spread servers evenly across fault and upgrade domains. This way if the fault domain goes down, only a fraction of the servers is lost. Upgrade domains may be a different fraction.
Yet another decision was to support multiple data abstractions from a single stack. Blobs, Tables and Queues are all supported on the same stack.
We will recall that blobs, tables and queues were all tracked with the same master Object Table. This system defined abstraction reduces management by the system.
Storage is handled in buckets of 100 TB. WAS limis the amount of storage for an account to be no more than 100TB which allows all storage data pertaining to an account to fit within a given storage stamp. To obtain more storage capacity within a single data center, customers use more than one account within that location. This was considered alright for large customers because they already had multiple accounts.
WAS mentions that it seems to defy CAP theorem WAS provides high availability with strong consistency guarantees while the theorem states that high availability, consistency and partition tolerance may not be achieved at the same time.
Lastly, WAS uses extensive debug logging infrastructure and the logs are maintained on local disks.
The WAS System has also been stress tested using pressure points.
#codingexercise
To count total number of N digit numbers whose sum of even and odd digits are both even, we modify it as follows:
long GetCountEqual(int digits, int even, int odd, int n, bool isOdd)
{
if (digits == n) return (even % 2 == 0 && odd % 2 == 0);
long count = 0;
if (isOdd){
for (int i = 0; i <= 9; i++)
count += GetCountEqual(digits+1, even, odd+i, n, false);
}else{
for (int i = 0; i <= 9; i++)
count+= GetCountEqual(digits+1, even+i, odd, n, true);
}
return count;
}
No comments:
Post a Comment