Network engineering continued

This is a continuation of the earlier posts starting with this one:  http://ravinote.blogspot.com/2020/09/best-practice-from-networking.html

 Shared-memory systems have been popular. They include SMPs, multi-core systems, and a combination of both. The simplest way to use it is to create threads in the same process. Shared-memory parallelism is widely used with big data.

The Shared-Nothing model supports shared-nothing parallelism.  When each node is independent and self-sufficient, there is no single point of contention. None of the nodes share memory or disk storage. Generally, these compete with any model that has a single point of contention in the form of memory or disk space.

Shared-Disk: This model is supported where a large space is needed. Some products implement shared-disk and some implement shared-nothing. Shared-nothing and shared-disk do not go together in the same code base. 

The implementation of a content-distribution network such as for images or videos generally translates to random disk reads which means caching may not always help. Therefore, the disks that are RAIDed are tuned. It used to be a monolithic RAID 10 when it is served from a single master with multiple slaves.  Instead, nowadays a sharded approach is taken and preferably served from Object Storage.

Image and video libraries will constantly run into cache misses especially with slow replication. It is better to separate traffic to different cluster pools. The replication and caching into the picture to handle the load.  With a distribution to different cluster pools, we can distribute the load and avoid them.

File Systems may implement byte-range locking to enable concurrent access. Typically, they are not supported by the File mapping operation. Poor use of file locks can result in performance issues or deadlock.