Monday, January 14, 2019

Today we continue discussing the best practice from storage engineering :

311) The SSD could be treated as a pool of fast storage that is common to all the processes.  since it is pluggable and external from all hard drives, it can be dynamically used as long as there is any availability.
312) In this sense it is very similar to L3 cache, however it is not meant for dynamic partitions, balancing access speed, power consumption and storage capacity. It is not as fast as cache but it is more flexible than conventional storage and plays a vital role in managing inter-process communication. This is a simplified storage.
313) SSDs can make use of different storage including flash storage. The two most common are NOR and NAND. NOR was the first of the two to be developed. It is very fast for reads but not as fast for writes, so it is used most often in places where code will be written once and read a lot. NAND is faster for writes and takes up significantly less space than NOR, which also makes it less expensive. Most flash used in SSDs is the NAND variety.
314) One of the easiest way to perform diagnostics on storage devices is to enable diagnostic  API which do not need any credentials and inform resource statistics
315) These diagnostic queries can show even Btree information as long as they are gathered correctly.
316) Blobs, tables and queues are three primary forms of storage. While storage products excel in one or the other forms of storage, only a public cloud provider is best suited to offer all three from the same storage.
317) Ingestion engine is usually built separate from the storage engine. Eventually the resources may result in the form of unstructured storage such as user files or blobs and structured storage such as tables.
318) Ingestion engines have a part to play in the larger data processing pipeline that users search with the help of a search engine. The data storage has to be searchable. Therefore, the ingestion engine also annotates the data, classifies the content, classifies for language and tags. The search engine crawls and expands the links in the data. The results are stored back as blobs. These blobs then become publically searchable. The worflows over the artifacts may be implemented in queues and the overall timing of the tasks may be tightened to make the results available within reasonable time to the end user in their search results.
319) The increase in the data size after annotations and search engine suitability is usually less than double the size of original data.
#codingexercise
List <String> getSameSuffixLength ( List <String> input, String tail){
Int length = getSuffixLength  (tail);
Return input.stream ()
              .select ( x -> getSuffixLength (x) == length)
              .collect (Collectors.toList ());
}


No comments:

Post a Comment