Saturday, June 23, 2018

We were reviewing Storj network. This was initiated to address scalability and increase decentralization.  It is a distributed cloud storage network and removes the notion of a centralized third party storage provider.  It provides client side encryption which improves data security. It maintains data integrity with a proof of retrievability. It introduces a network first design where peers are autonomous agents and there is a protocol to enable them to negotiate contracts, transfer data, verify the integrity and availability of remote data and to reward with payments. It provides tools to enable all these interactions.
It distributes the storage of a file as shards on this network and these shards are stored using a distributed hash table. The shards are themselves not stored in this hash table, rather a distributed network and messaging facilitates it with location information. Storj is built on Kademlia which provides the distributed hash table. Kademlia routes messages  through low latency paths and uses parallel asynchronous queries to avoid timeouts. It also reduces the number of configuration messages peers need to learn. The information spreads automatically  so the number of nodes can scale. Many of the important properties of Kademlia can be formally proven. Keys are opaque hash of some larger data and peers have node IDs. The key-value pairs are stored on the nodes with the ID close to the key where closeness is some notion. With the use of Kademlia, Storj focuses on the data instead. Whether the file is in tact is informed via challenge response interaction which is dubbed an audit. Merkle trees and proofs are used to verify integrity. This scheme may introduce some overhead so an extension is used that utilizes subsets of the data.
Today we will look at contracts and negotiations. Contracts is between the data owner and the farmer. The latter is a term used to describe the one who houses the data.  The term farmer or banker is interchangeable because he provides a storage for one of the distributed ledgers and hopes to make money from mining it.  The banker or farmer does not need to know the contents of the data.  The data owner retains complete control over the encryption key which determines access to the data.  The data owner keeps a set of challenges, selects one of them and sends it to the farmer.  The farmer uses the challenge and the data to generate the Merkle pre-leaf and uses it with the other leaves to generate a Merkle proof. This proof is sent back to the data owner. The data owner uses the root and tree depth  to verify the proof.
#codingexercise
Count the number of decreasing paths in a matrix if the traversal is permitted only on adjacent horizontal and vertical cells
Solution: we can sum the number of paths with each cell as the starting location if we know how to do it for  a given starting position. We can do this backtracking as well as dynamic programming.  Backtracking does not allow us to make a wrong choice but we repeat the same calculations over and over again. Dynamic programming utilizes the calculations of previously computed start locations. Ideally a good starting location will be the center of the matrix.
initialize the sum to one for the number of paths at the center where the center forms a standalone single element  path
foreach of the four adjacent positions around a center:
          if the value is less than the center:
                recurse to find the count at the adjacent cell
                add the count through that adjacent cell to the sum at the center.
return the sum of paths for this position

No comments:

Post a Comment