Part 3: The refactoring of old code to new microservices

 Part 2 of this article earlier, described microservices versus monolithic architecture. With the introduction of microservices, it became easy to host not only a dedicated database but also a dedicated database server instance and separate the concerns for each functionality that the user interface comprised of. When we use microservices with Mesos-based clusters and shared volumes, we can even have many copies of the server for high availability and failover. This is possibly great for small and segregated data but larger companies often require massive investments in their data, often standardizing tools, processes, and workflows to better manage their data. In such cases, consumers of the data don't talk to the database directly but via a service that sits behind say even a message bus. If the consumers proliferate, they end up creating and sharing many different instances of services for the same data each with its own view rather than the actual table.  APIs for these services are more domain-based rather than implementing a query-friendly interface that lets you directly work with the data. As services are organized, data may get translated or massaged as it makes its way from one to another. It is possible to have a ring of microservices that can take care of most data processing for business requirements. Data may even be at most one or two fields of an entity along with its identifier for such services. This works very well to alleviate the onus and rigidity that comes with organization, the interactions between the components, and the various chores that need to be taken to keep it flexible to suit changing business needs. The microservices are independent so they stand by themselves as if spreading out from data for their respective functionalities. This is already business-friendly because each service can now be modified and tested independently of others.

The transition to microservices from legacy monolithic code is not straightforward. The functionalities must be separated beyond components. And in the process of doing so, we cannot risk regression. Tests become a way to scope out behavior at boundaries such as interface and class interactions.  Adequate coverage of tests will guarantee backward compatibility for the system as it is refactored. The microservices are independently testable both in terms of unit tests as well as end-to-end tests. Services usually have a REST interface which makes it easy to invoke them from clients and comes with the benefits of using browser-based developer tools. The data store does not need to be divided between services. In some cases, only a data access service is required which other microservices can call. The choice and design of microservices stem from the minimal functionalities that need to be separated and articulated. If the services don’t need to be refactored at a finer level, they can remain encapsulated in a singleton.

The rule of thumb for the refactoring of the code is the follow-up of the Don’t Repeat Yourself or (DRY) principle which is defined as “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system”. This calls for every algorithm or logic that is cut and pasted for different usages to be consolidated at a single point of maintenance.  This improves flexibility because enhancements such as the use of a new data structure can be replaced in one place and it also reduces the bugs that come by when similar changes must be made in several places. This principle also reduces the code when it is refactored especially if the old code had several duplications. It provides a way to view the minimal skeleton of the microservices when aimed at the appropriate scope and breadth. Even inter-service calls can be reduced with this principle.

 

Good microservices are not only easy to discover from their APIs but also easy to read from their documentation which can be autogenerated from the code with markdowns. Different tools are available for this purpose and both the approach of using microservices as well as the enhanced comments describing the APIs provide sufficient information for the documentation.

 

Part 1 of this article describes application modernization. This section deals with microservices architecture that suits application modernization very well. Microservices break away from the monolithic architecture which has been the norm in legacy systems for a while. Monolithic applications tend to grow indefinitely which also increases the complexity. Finding bugs and creating new features take a long time. If a part of the application needs to be updated, the whole application must be restarted which can mean a considerable down time for large systems. Monolithic applications are harder to deploy since some parts of the application might have different requirements. Some parts are computationally heavy, and others are memory heavy. The one-size-fits-all environment to satisfy all requirements of the application is usually expensive and suboptimal. They are not scalable.  A peak in traffic can lead to failures from various components. If the number of instances of the entire application is increased, it wastes resources. These systems do not evolve fast because they are locked in technology. The same programming language and framework must be used from the first to the last module.

Microservices are self-sufficient processes that can interact with other microservices to form a distributed application.  Generally, a ring of microservices is developed that are small independent services that have their own isolated environment with operating systems, databases and other support software.  Each microservice might be dedicated to a distinct set of resources that it supports with create, update and delete operations. They often use message passing via web requests to communicate with one another. Each microservice can be built with different programming languages and different environments depending on the requirements.

Microservices facilitate cross-functional team organization and based on business capabilities. This leads to faster delivery and higher quality due to testability and focused effort. This avoids the immense cross-team interactions from component-based software development. It also avoids developers from writing logic in the layer that is closest to them be it user-interface, service, or database.

Cloud service platforms have made operating and deploying microservices based applications easier and cheaper. It allows teams to build microservices using continuous integration and continuous delivery. The pipeline automates testing, building, developing, deploying and delivering the microservices. Updates to one microservice does not affect the others. But when a single microservice goes down, it can have a cascading effect on other services because they have high fault density. This is true also for components that grow in size. This is generally overcome by keeping microservices focused and small.

The reliability of microservices is dependent on the reliability of the communication between them. Http and protocol buffers are the communication protocols of choice. Development and deployment are also owned by the same team. This idea is inspired by Amazon’s “you build it, you run it” philosophy. The transition to microservices from legacy monolithic code is not straightforward. The functionalities must be separated beyond components. And in the process of doing so, we cannot risk regression. Tests become a way to scope out behavior at boundaries such as interface and class interactions.  Adequate coverage of tests will guarantee backward compatibility for the system as it is refactored. The microservices are independently testable both in terms of unit tests as well as end-to-end tests. The choice and design of microservices stem from the minimal functionalities that need to be separated and articulated. If the services don’t need to be refactored at a finer level, they can remain encapsulated in a singleton. 

The rule of thumb for the refactoring of the code is the follow up of the Don’t Repeat Yourself or (DRY) principle which is defined as “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system”. This calls for every algorithm or logic that is cut and pasted for different usages to be consolidated at a single point of maintenance.  This improves flexibility because enhancements such as the use of a new data structure can be replaced in one place and it also reduces the bugs that come by when similar changes must be made in several places. This principle also reduces the code when it is refactored especially if the old code had several duplications. It provides a way to view the minimal skeleton of the microservices when aimed at the appropriate scope and breadth. Even inter-service calls can be reduced with this principle. 

- courtesy Kristian Tuusjärvi

Application Modernization:

 Software used by companies is critical to their business and will continue to provide return on investment. Companies will try to maximize this for as long as possible. Some maintenance is required to these software systems which satisfy business and customer needs and address technical debt that accrues over time. Maintenance works well for short term needs but as time progresses, the systems become increasingly complex and out of date. Eventually maintenance will no longer be efficient or cost-effective. At this point, modernization is required to improve the system’s maintainability, performance, and business value. It takes much more effort to accomplish compared to maintenance. If a software can no longer be maintained or modernized, it will need to be replaced. 

The risks of modernizing legacy systems primarily come from missing documentation. Legacy systems seldom have a complete documentation specifying the whole system with all its functions and use cases.  In most cases, the documentation is badly missing which makes it hard to rewrite a system that would function identically to the previous one. Companies usually couple their legacy software with their business processes. Changing legacy software can cause unpredictable consequences to the business processes that rely on it. The nature of replacing legacy systems with new ones is risky, since the new system can be more expensive on a total cost of ownership basis and there can be problems with its schedule of delivery. 

There are at least three strategies for dealing with legacy systems: scrap the legacy system, keep maintaining the system or replace the whole system. Companies generally have limited budgets on the legacy systems, so they want to get the best return on the investment. Scrapping the system can be an option if the value has diminished sufficiently. Maintenance can be opted into when it is cost-effective. Some improvement is possible by adding new interfaces to make the system easier to maintain. Replacement can be attempted when the support has gone, the maintenance is too expensive, and the cost of the new system is not too high. 

Both technical and business perspectives are involved. If a legacy system has low quality and low business value, the system should be removed. Those with low quality but high business value must be maintained or modernized depending on the expense.  Systems with high quality can be left running. 

Modernization is a more extensive process than maintenance because modernization often incorporates restructuring, functional changes, and new software attributes. Modernization can be either white-box or black-box depending on the level of abstraction. White box modernization requires a lot of information about the internals of the legacy system. Contrary to that, the black box modernization only requires external interfaces and compatibility. Replacement is an option when neither approach works. 

Software modernization is also an evolution of systems. White box systems are more popular than black box systems which might be counter-intuitive to the notion that black-box modernization is easier than white-box modernization. The tool for whitebox methods could have become better to help with the shift. Legacy systems are harder to integrate. Software integration allows companies to better control their resources, remove duplicate business rules, re-use existing software, and reduce cost of development. The effort needed to keep legacy systems running often takes resources away from other projects. Legacy systems also suffer from diminishing ownership and knowledge base which makes changes difficult to make. On the other hand, their business value makes them appear like rare diamonds even when they cost a lot. 

Data Modernization:

 

Data technologies in recent years has popularized both structured and unstructured storage. This is fueled by applications that are embracing cloud resources. The two trends are happening simultaneously and are reinforcing each other.

Data modernization means moving data from legacy databases to modern databases. It comes at a time when many databases are doubling their digital footprint. Unstructured data is the biggest contributor to this growth and includes images, audio, video, social media comments, clinical notes, and such others. Organizations have shifted from a data architecture based on relational enterprise-based data warehouses to data lakes based on big data. If the survey from IT spends is to be believed, a great majority of organizations are already on their way towards data modernization with those in the Finance service firms leading the way. These organizations reported data security planning as part of their data modernization activities. They consider the tools and technology that are available in the marketplace as the third most important reason in their decision making.

Drivers for one-time data modernization plan include security and governance, strategy and plan, tools and technology, and talent. Data modernization is a key component of, or reason for, migrating to the cloud. The rate of adoption of external services in the data planning and implementation is about 44% for these organizations.

The perceived obstacles to implementing data modernization include budget/cost constraints, lack of understanding of technology, lack of consensus among decision-makers, absence of clarity on success metrics, and such other causes. Cloud is already a dominant storage location for nine out of ten of these organizations and it is both a means and an important consequence. A majority of these organizations have all their important applications and data in the cloud. Application and data can be moved independently but many organizations are putting it on modernized platforms at the same time and moving them from on-premises to the cloud. Traditional IT architectures and on-premises data centers often come with their own cost concerns which makes cost a key driver of cloud migration. Those organizations that have combined cloud migration and data modernization could deliver on their strategic goals.

This leads to the assertion that almost all data management approaches will likely eventually be modernized and almost all data and applications will be in the cloud. Cloud migration and data modernization will continue to mutually reinforce each other. Since these two trends support and overlap each other, most companies will do well with both trends.

Collaborative caching for multitenant solutions:

 With the case study for developing a multitenant solution for the delivery of user generated content to designated audience, different algorithms are required to tune the system for higher performance. These include collaborative caching, context aware streaming, user redirection, distribution tree etc. This section discusses one of them.

Content caches are strategically located through the network which support services with optimally distributing the content from its source to its consumers. Collaborative caching allows software to use hints to influence cache management. When the hints are about the inclusion property, optimal caching can be achieved if the access sequence and cache size are known beforehand. An example of such a hint could be a binary choice between Least Recently Used Algorithm or Most Recently Used Algorithm. Another way a hint could be provided is with a number encoding a property. The result is a new algorithm that can be described as priority LRU and captures the full range between MRU and LRU.

Hints are added by annotating each memory access with a numerical priority. Data accessed with a higher priority takes precedence than data accessed with a lower priority. Collaborative caches enable the caller to specify the importance and the system to determine the priority to manage the data. LRU/MRU used to infer this importance but with a sliding scale priority specified by the caller, a new type of inclusion – non-uniform inclusion becomes possible.

The word inclusion derives from the hierarchy of caches at the machine level namely L1, L2 and so on. The property states that the larger cache will always contain the content of the smaller cache. Generally, a stack data structure is used to visualize hits and misses in play. The stack distance is a useful metric derived from the stack simulations such as for LRU and denotes the amount of data accessed between consecutive re-uses of the same entry and suggests system locality. Data elements at the top c stack positions are the ones in a cache of size c. The stack position defines the priority of the stored data. All accessed data are ordered by their priority in a priority list. The stack distance gives the minimal cache size to make an access, a cache hit and is calculated by simulating such a cache of an infinite size. In the simple LRU case, the data is prioritized by the most recent access time.  The data in a MRU cache is also prioritized by the most recent access time but unlike LRU, the lowest priority is the data element with the most recent access time.  The LRU-MRU can also be mixed where a hint indicates whether the access is LRU or MRU.  Its stack distance would be computed by the stack-based algorithm by assigning the current access time as priority for LRU and corresponding negation for the MRU. The priority hint changes the default scheme.

The effects of the priority hints on cache management include 4 cases for cache hits and 2 cases for cache miss. Consider the access to w with a priority i, (w,i) arriving in a cache of size m, and the current stack position j. The change in priority leads to the item moved up, not at all or moved down in the stack of m items and the move is just conceptual pertaining only to the organization in the stack.

These cases include:

1.       1 <= i < j <= m and w is found in the cache which leads to hit up move where item moves up to position i and the entries from i to j-1 move one position lower.

2.       1 <= j  = i <= m and w is found in the cache which leads to no movement

3.       1 <= j < i <= m and w is found in the cache which leads to hit down move where the item moves down to position I and the entries j+1 to I move one position higher.

4.       1 <= j <= m < I and w is moved out of the cache and the entries from j+1 to m are moved one position higher. This is a voluntary eviction.

5.       j = infinity and 1 <=i and <= m and the accessed data element w is missed from the cache and moved into the position at I and all entries move one position lower. The lowest priority entry is evicted.

6.       J = infinity and i > m which results in a miss bypass and the entries in the cache are unaffected.

This way the priorities stated in the hints and those implied in the access sequence is reconciled.

The case for disk quotas:

 Oracle Solaris ZFS demonstrated the use of resources and quotas. With the evolution towards cluster-based computing, Network Accessible Storage significantly widened the horizon for unlimited disk space because capacity could now be added in the form of additional nodes and their disks. This represented a virtualized storage but there were no reservations. While ZFS demonstrated effective resource management with isolations for workloads, the cluster would not keep up with the same practice without some form of roles in the form of control and data nodes. In addition, the reservations need not be governed exclusively by the user. They can be decided by system as quality-of-service levels. To achieve service levels, on the same shared disks, we can create disk groups.

There will be times when node disk groups become overloaded by I/O requests. At such time, it will be difficult to identify where the I/O requests are predominantly originating from so that those accounts could be throttled while well-behaved accounts are not affected. Each node disk group keeps track of accounts that issue the I/O requests. The system can then use a Sample-Hold algorithm to track the request rate history of the top N busiest accounts. This information can then be used to determine whether an account is well-behaved or not. If the traffic reduces when the account is throttled, it becomes well-behaved. If a node disk group is getting overloaded, it can use this information to selectively limit the incoming traffic, targeting accounts that are causing the issue. For an example of a metric to serve this purpose, a node disk group can compute a throttling probability of the incoming requests for each account by determining the request rate history for the account. If the request rate is high, it will have a higher probability of being throttled. The opposite also holds. When the metric builds up a history of measurements, it will be easier to tell if the account is well-behaved.  

Load balancing will continue to keep the servers loaded within an acceptable limit. If the access patterns cannot be load balanced, then there is probably high traffic. In such cases, the accounts will be limited, and they will be well-behaved again.  

A node may have more than one disk group so it can form logical partitions in its existing disk space while not requiring cluster level changes in providing service levels from its disks. This is the intelligent node model. If the cluster can perform more effectively by grouping data nodes or their disks and with the help of dedicated control nodes, then the data nodes are freed up to focus on the data path and merely read-write from disk groups. In either case the group id is merely a part of the metadata.
The word group is generally used with client-facing artifacts such as requests and the system resources such as disks are referred to as pools. By that definition, I’m sorry for using the word group and the above argument could be read as for disk pools.
 

 

This article picks up the previous discussion on the requirements from a multi-tenant solution for the delivery of user-generated content to designated consumers and delves into the architecture.

The architecture comprises a cloud infrastructure layer, a content service virtualization layer, and an Identity Access Management that provides authentication, authorization, and auditing. These can be integrated through an inter-cloud messaging bus that facilitates a distributed protocol for resource management, account management, content management and service orchestration.

The cloud infrastructure layer spans the public, private and personal cloud. The purpose of stretching a cloud infrastructure layer over hybrid IT resources is to form a dynamic content delivery service overlay. This overlay consists of a content distribution tree which provides 1) the required Quality-of-Service and 2) has a minimum cost. In addition to the standard storage and bandwidth required by the overlay, some computing resources are reserved for content processing, rendering and management functionalities. Each of these functionalities requires a dedicated application engine. The key module to pool all the required IT resources together is a distributed algorithm for cloud resource management. Any message-based consensus protocol could be used here such as Paxos or Raft*.

The content service virtualization layer is the one providing a suite of middleware components to support different application engines. These include:

Content distribution: which provides services for optimally distributing the content from its source to its consumers. These services are supported by content caches strategically located through the network. The distribution component. The distribution component also includes the actual mechanism and the protocols used for transmitting data over the underlying network and serving the content to the consumers.

Content processing: which provides services for sourcing and adapting content based on available bandwidth, user preferences and device capabilities. Modification and conversion also includes those for wireless devices.

Content storage: which includes modules that provision directories and artifacts for storing user generated content with security specified by the providers. The implementation of an S3 storage is a good analogy here. Functionalities include distributed secure storage, content naming, and resolution, and content replication.

Request routing: which provides services for navigating both clients’ requests to upload their contents and consumers requests to retrieve the content from the nearest and available location. The selection of this location depends on both the proximity and the availability of the system resources.

Service orchestration: which provides services for integrating different media services for internal and external service entry points. For example, a social TV application would draw a lot of individual media services such as a buddy service from social networking and a media streaming service.

The IAM and Service Bus are left out to be completed as appropriate for the above application engines.