Tuesday, June 30, 2020

Stream operations continued

The stream manager is also the right interface to add these capabilities required at the stream store level from these automation workflows. Most of the existing methods on that interface target streams and the operations taken on the stream as a whole and these automation workflows are served by equivalent commands on the same stream. The new capabilities can be added cleanly without any issues with the earlier releases.
One of the advantages of providing this capability is that the applications can close the stream manager when they are done with its use allowing proper finalization in all cases. The stream manager interface already comes with the close method to facilitate this. 
The stream Manager can also facilitate copying segmentRanges by utilizing head and tail streamcuts This works just like whole streams except that the reader/iterator is initialized with the segment ranges. The segmentRange has to be contiguous because only a head and tail stream cut is specified. This is not a problem if multiple segmentRanges have to be copied. In such a case, the copy operation is repeated in every contiguous range 
Scopes can also be copied by iterating over all the streams and making copies of each stream. This is generally a long running operation as each stream could be massive by itself. The operation to perform scope copy is the similar to the copy of a folder containing streams that were written to files
The utility of stream copy is only enhanced by the use cases of scope copy, segment range copy and archival. In place editing of streams is avoided by performing sealing of segments written. There is no overwrite and all write operations are append only 
The streamManager could also move a stream from one scope to another by using copy operation although metadata only operation would be significantly more efficient in this case. 

No comments:

Post a Comment