Network engineering continued ...

This is a continuation of the earlier posts starting with this one: http://ravinote.blogspot.com/2020/09/best-practice-from-networking.html

564. 571. Sequences can be stored efficiently with bitmap index when they are rather sparse. Bitmaps also help with conjunctive filters and this is useful in sequences with repeating members 

     572. 
          

     572. The sequences can be more efficiently queried than standard query operators if the predicates are pushed down closer to the storage.  

     573. 
          

     573. Sequences work well with bloom filters which test whether a member is part of the sequence or not. Sometimes it is enough to rule out that a member is not part of the set 

     574. 
          

     574. If the range of sequences can be limited to a window, the user and application can take on much of the processing relieving the compute requirements from storage. Such intensive scripts can run anywhere the user wants if the data is available.  

     575. 
          

     575. If logic pertains specifically to some data and applicable only to that data, it is possible to register logic and load a runtime to execute that logic specific to data just as it is possible to externalize query processing over an iterative data set. 

     576. 
          

     576. There are several containers for logic usually packaged as modules and they can be invoked by a common runtime. However, at its simplest form, this logic is merely a set of rules. 

Network engineering continued ...

 This is a continuation of the earlier posts starting with this one: http://ravinote.blogspot.com/2020/09/best-practice-from-networking.html

564. Sales and support need to be armed with the latest information and remain current on all features from the customers. They need to have those features work exactly as they say it would. 

565. 
     

565. The focus on business value does not remain confined to the people on the border with the customers. It comes from deep within product development and engineering. 

566. 
     

566. A network has high reusability and can be saved for all subsequent usages. This works well for data transfers that do not change over time. 

567. 
     

567. When the data changes frequently, it helps to organize it in a way such that those that don’t change are on one side and those that do are on the other side. This helps with incremental results from the data. 

568. 
     

568. Data will inevitably have patterns with reuse. We can call them sequences. While most data might be stored with general-purpose B-Tree, the sequences call for more efficient data structures such as the radix tree. These help insert and lookup of sequences to be easier. 

569. 
     

569. Sequences are more efficiently stored if they remain sorted. This canonicalizes them. It also makes lookup use binary search. 

570. 
     

570. The number of sequences might become very large. In such a case, it might be better to not make it part of the same tree and use more than one tree with shards and an index.  

Network engineering continued ...

This is a continuation of the earlier posts starting with this one: http://ravinote.blogspot.com/2020/09/best-practice-from-networking.html

543. 555. The statistics may be stored in a container belonging to the system. Since the system containers are treated differently from the user, they will need to be serviced separately. 

     556. 
          

     557. Application and system both publish statistics. They may both need to be the source of truth for their data  

     558. 
          

     558. When the same container is replicated in different zones, there is a notion of local and remote. Only one of them is designated as primary. The remote is usually secondary 

     559.  

     560. The number of applications using the same network is usually not a concern. The ability to serve them with different quality of service makes it special.

     561.  

     561. When an application wants to change the workload on the network, architects to prefer to swap a networking product with something more suitable. However, a performance engineer can circumvent the approach with optimizations that leverage the existing product. It is always a good practice to give this a try. 

     562. 
          

     562. System architecture holds in favor of changing business needs from the smallest components to the overall topology. However, it is rather centralized and sometimes using another instance of the topology with customizations can mitigate the urgency while giving ample time for consolidation. 

     563. 
          

     563. The use of networking products also depends on the developer community. Many products such as software-defined stacks and container orchestration framework have generated greater acceptance by endearing the product to developers. 

Network engineering continued ...

 This is a continuation of the earlier posts starting with this one: http://ravinote.blogspot.com/2020/09/best-practice-from-networking.html

543. Book-keeping is essential for both scheduler and allocator not only to keep track of grants but also for analysis and diagnostics. 

544. 
     

544. A scheduler and allocator can each have their own manager that separates the concerns of management from their work 

545. 
     

545. The more general purpose the scheduler and allocator become, the easier it is to use them in different components. Commodity implementations win hands down against specialized ones because they scale. 

546. The requests for remote resources are expected to perform longer than local operations. If they incur timeouts, the quantum grants may need to stretch over.  

547. 
     

547.  Timeout must expand to include timeouts from nested operations. 

548. 
     

548. Some event notification schemes are helpful to handle them at the appropriate scope. 

549. 
     

549. A recovery state machine can help with global event handling for outages and recovery. 

550. The number of steps taken to recover from outages can be reduced by dropping scoped containers in favor of standby