Today we continue discussing the best practice from storage engineering :
175) Storage can be used with any analysis package. As a storage tier, a product only serves the basic Create-update-delete-list operations on the resources. Anything above and beyond that is in the compute layer. Consequently, storage products do well when they integrate nicely with popular analysis packages.
176) When it comes to integration, programmability is important. How well a machine learning SDK can work with the data in the storage, is not only important for the analysis side but also for the storage side. The workloads from such analysis are significantly more different that others because they require GPUs for heavy iterations in their algorithms.
177) There are many data sources that can feed the analysis side but the storage tier is uniquely positioned as a staging for most in and out data transfers. Consequently, the easier it is to to trace the data through the storage tier, the more popular it becomes for the analysis side
178) Although we mentioned documentation, we have not elaborated on the plug and play kind of architecture. If the storage tier can act as a man in the middle with very little disruption to ongoing data I/O, it can become useful in many indirect ways that differ from the usual direct storage of data.
179) A key aspect of storage tier is the recognition of a variety of data formats or in a specific case file types. If the file is an image versus a text file, then it does not help with certain storage chores such as dedeplication and compression. Therefore, if the storage product provides ability to differentiate data natively, it will help in its acceptance and popularity
180) File types are equally important for applications. Workloads become segregated by file types. If a storage container has all kinds of files, the ability to work with them as if the file types were independent would immensely benefit in separating workloads and providin differential treatment.
175) Storage can be used with any analysis package. As a storage tier, a product only serves the basic Create-update-delete-list operations on the resources. Anything above and beyond that is in the compute layer. Consequently, storage products do well when they integrate nicely with popular analysis packages.
176) When it comes to integration, programmability is important. How well a machine learning SDK can work with the data in the storage, is not only important for the analysis side but also for the storage side. The workloads from such analysis are significantly more different that others because they require GPUs for heavy iterations in their algorithms.
177) There are many data sources that can feed the analysis side but the storage tier is uniquely positioned as a staging for most in and out data transfers. Consequently, the easier it is to to trace the data through the storage tier, the more popular it becomes for the analysis side
178) Although we mentioned documentation, we have not elaborated on the plug and play kind of architecture. If the storage tier can act as a man in the middle with very little disruption to ongoing data I/O, it can become useful in many indirect ways that differ from the usual direct storage of data.
179) A key aspect of storage tier is the recognition of a variety of data formats or in a specific case file types. If the file is an image versus a text file, then it does not help with certain storage chores such as dedeplication and compression. Therefore, if the storage product provides ability to differentiate data natively, it will help in its acceptance and popularity
180) File types are equally important for applications. Workloads become segregated by file types. If a storage container has all kinds of files, the ability to work with them as if the file types were independent would immensely benefit in separating workloads and providin differential treatment.
No comments:
Post a Comment