Wednesday, January 17, 2018

File descriptors on steroids continued
design

While the file descriptor was inherently local to the process, the DFS allowed it to point to a file on a remote computer. Likewise, file system protocols such as CIFS allowed remote servers to be connected through Active Directory and access was granted to users registered there. Deduplication worked on segments that were identical so that space could be conserved. RSync protocol helped replicate between source and destination regardless of whether the destination was a file system or a S3 endpoint. In all these tasks, much of the operations were asynchronous and involved a source and destination. This library utilizes ZeroMQ messaging library for file system operations.
Performance:
ZeroMQ has demonstrated performance for communications. The stacking of file operations from a storage solution perspective over this library can also meet the stringent requirements for cloud level operations. The implementation might vary on purpose, scope, scale and management as we add plugins for a client but the assumption that asynchronous operations on a remote file will not be hampered by ZeroMQ remains sound.

Security:
Enhanced File Descriptors are inherently as secure as sockets. However file system utilities to secure the files work because these behave the same as regular ones to the layers above.

Testing:
The implementation for this Storage Framework must be able to process hundred thousand requests per second with message sizes of a mix from 0 to 64kb for duration of 1 hour with little or no degradation in write latency for the writes to a million files. Checksum may be used to see that the files are correct. Testing might require supportability features in addition to random file writes. The statistics, audit log, history and other management aspects of the queue should be made available for pull via web APIs.

Conclusion:
With smart operating system primitives we can enhance each process to give more power to individual business.
#codingexercise

trace requests and responses from logs and stores.
#codingexercise
Get Fibonacci number
 we compared the following
uint GetTailRecursiveFibonacci(uint n, uint a = 0, uint b = 1)
{
    if (n == 0)
        return a;
    if (n == 1)
        return b;
    return GetTailRecursiveFibonacci(n-1, b, a+b);
}
with the conventional:
uint GetFibonacci (uint n)
{
    if (n == 0)
        return 0;
    if (n == 1)
        return 1;
    return GetFibonacci(n-1) + GetFibonacci(n-2);
}
0 1 1 2 3 5 8 13 21 34



No comments:

Post a Comment