We continue reading the paper "Erasure coding in Windows Azure storage" by Cheng Huang, Huseyin Simitci, Yikang Xu, Aaron Ogus, Brad Calder, Parikshit Gopalan, Jin Li, and Sergey Yekhanin.
Erasure coding is a data protection method where data is broken into fragments, expanded and encoded with redundant data pieces and stored across a different set of locations or media. This paper talks about Erasure coding in windows Azure storage WAS. It introduces new codes called the Local Reconstruction Codes.A (k,l,r) LRC divides k data fragments into l local groups. It encodes l local parities, one for each local group and r global parities. We were reviewing the performance of LRC in WAS with small I/Os and large I/Os. Finally we read up on the latency comparision for decoding fragments between LRC and Reed Solomon in this performance study. The latency is expected to be improved in LRC due to the reduced number of fragments. However this is much smaller by orders of magnitude than the overall latency to transfer the fragments to perform the reconstruction. This makes the latency comparable between LRC and Reed Solomon
#coding exercise
Implement Galois field polynomial subtraction
Polynomial is represented as coefficient and power pairs in ascending order
They are grouped and operated.
List <Term> SubtractPolynomial ( List <Term> first, List <Term> Second)
{
int ret = new List <Term>();
first.sort();
second.sort();
int i = 0;
int j = 0;
while (i < first.Count && j < second.count)
{
if (first[i].power == second[j].power){
ret.Add(new Term(first[i].coefficient - second[j].coefficient, first[i].power));
i++;
j++;
}
else if (first[i].power < second[j].power)
{
ret.Add(new Term(first[i].coefficient , first[i].power));
i++;
}
else
{
ret.Add(new Term(second[j].coefficient , second[j].power));
j++;
}
}
while ( i < first.Count)
{
ret.Add(new Term(first[i].coefficient , first[i].power));
i++;
}
while (j < second.Count)
{
ret.Add(new Term(second[j].coefficient , second[j].power));
j++;
}
Return ret;
}
Erasure coding is a data protection method where data is broken into fragments, expanded and encoded with redundant data pieces and stored across a different set of locations or media. This paper talks about Erasure coding in windows Azure storage WAS. It introduces new codes called the Local Reconstruction Codes.A (k,l,r) LRC divides k data fragments into l local groups. It encodes l local parities, one for each local group and r global parities. We were reviewing the performance of LRC in WAS with small I/Os and large I/Os. Finally we read up on the latency comparision for decoding fragments between LRC and Reed Solomon in this performance study. The latency is expected to be improved in LRC due to the reduced number of fragments. However this is much smaller by orders of magnitude than the overall latency to transfer the fragments to perform the reconstruction. This makes the latency comparable between LRC and Reed Solomon
#coding exercise
Implement Galois field polynomial subtraction
Polynomial is represented as coefficient and power pairs in ascending order
They are grouped and operated.
List <Term> SubtractPolynomial ( List <Term> first, List <Term> Second)
{
int ret = new List <Term>();
first.sort();
second.sort();
int i = 0;
int j = 0;
while (i < first.Count && j < second.count)
{
if (first[i].power == second[j].power){
ret.Add(new Term(first[i].coefficient - second[j].coefficient, first[i].power));
i++;
j++;
}
else if (first[i].power < second[j].power)
{
ret.Add(new Term(first[i].coefficient , first[i].power));
i++;
}
else
{
ret.Add(new Term(second[j].coefficient , second[j].power));
j++;
}
}
while ( i < first.Count)
{
ret.Add(new Term(first[i].coefficient , first[i].power));
i++;
}
while (j < second.Count)
{
ret.Add(new Term(second[j].coefficient , second[j].power));
j++;
}
Return ret;
}
No comments:
Post a Comment