Tuesday, January 26, 2021

The API layer for use by mobile applications...

  This is a continuation from the previous post:

  • It is easier to add workflows to existing software by reading data and transforming it for analysis and reporting given that the online processing needs to be as efficient and streamlined as possible, so it remains available to the customers.  And finally, the organization may find it easier make value propositions with vertical silos instead of horizontal modifications. While databases used to form the shared data infrastructure and services were differentiated for the value propositions, today organizations prefer to move from infrastructure view to microservice model with the introduction of new services and the retiring of old services. 

  • Regardless of the intention, scale, scope and impact of the changes to the code base, most improvements suffer from the malaise that there are not enough tests and that it is too expensive and even prohibitive to have the adequate surface coverage for acceptance. It is widely believed that most of the tests are investments for feature launches as well as customer satisfaction. 

  • Tests are just as much currency as any other artifact. The presence of a well written test can enforce expectations from the software as early as design time and all the way through the software development cycle. In this regard, one of the improvements we could consider is to increase API based testing for the service. Generally, an n-tier service has tests at all levels starting from the database unit tests at the back end, api level tests at the service layer, and front-end tests for the front end. Out of these, the design of the APIs and their tests poses the maximum benefit in the tradeoffs for value of tests and the technical debts. At the same time all non-functional aspects such as security and performance can still be covered. Yet APIs are hard to keep tidy in the face of ever-increasing improvements and business changes to the software. 

  • A web application will rely heavily on backend services that may work exclusively for specific purposes such as phone contact information lookup, address information lookup, authentication and auditing, utilities such as issuing tokens for logged in users and many others. Each of these services may also be hosted in different environments that are used to prepare and test the code prior to its release to production. Consequently, these services could very well be counted as a dependency for the software. In order that we keep track of the dependencies and for troubleshooting issues with the software, we could consider command line tools that enable Application Programming Interface (API) calls to the service alone. These tools help use the service in isolation while also provide a command line convenience to gain information on individual resources such as an account or a number lookup. While curl – a popular command line tool can be used to call services via APIs hosted over the HTTPS, most enterprise services are secured and some of these APIs often require a pre-amble where one API called before another.  These tools can come in helpful for this purpose. These tools come in helpful for triage and to work in isolation from the tests and code. Moreover, these tools are for the convenience of getting data or setting state and not for operations or monitoring which generally have their own infrastructure based on impact radius. 

Monday, January 25, 2021

The API Layer for use by mobile applications.

 This is a continuation from the previous post:

·        Organization also comes with access routines and controls. Row level security and Role based access control continue to dominate this purpose.  Label based controls and organization spanning resource tree access security are increasingly being taken on at the cloud level, so we have less to content within the business implementations.

·        Identity and access management including credentials, keys, ciphers, messages and rotations are increasingly tackled by dedicated and almost ubiquitous platforms and products. Neither the membership provider nor the mechanism or protocol is specific to the business.

·        Plugins are easy ways to bring in functionality into a platform. They also allow isolation and working with the applications, but they need to conform to the plugin architecture. This is like separation of concerns, but it is tighter with consistency requirements. When the customizations by way of plugins have been around for a while, the platform can consolidate some of the functionalities and offer them as usable by the plugins.

·        The way the mobile applications evolve depends on the backend services just as much as the business requirements from the point of sales. Therefore, the mobile applications merely act as a facilitator.

·        APIs like most customer facing software interactions suffer from the same liabilities over time. Fortunately, the versioning and retirement policies can be enforced for all clients with the added advantage that more and more clients accept HTTPS based APIs over other forms of interface. As devices, platforms and ecosystems change, the APIs present an evergreen investment that can stand the test of time better than others. Moreover, they naturally lend themselves to automation, diagnostics and monitoring than most other software.

·        As software matures, many engineers choose to be dainty about the improvements they make often requiring very little change to existing data paths or processes. They do this for several reasons. First, because code reuse and original design decisions work favorably. Second altering existing code paths often runs the risk of regression. Third the cost of churn generally rises higher than the benefits. Also, much of the changes are usually in the form of technical debt – a term used to refer to the corrections needed as features are released by taking shortcuts which create instant debt.


Sunday, January 24, 2021

The API Layer for use by mobile applications.

 This is a continuation from the previous post:

  • Less chatty traffic between mobile applications and servers: Internet is extremely popular to allow any sized load and traffic for businesses anywhere. At first, this seems counter-intuitive towards highly responsive mobile applications but the migration of business applications from SOAP to REST to GraphQL has shown that mobile applications can do more with less.  GraphQL makes the frontend easier to develop, API versioning is no longer a problem, works with less data fetching, works well with serverless computing and driving down costs with being dynamic and flexible. 


  • Operational insights: Almost all IT businesses are concerned about ITOM such as with alerts and events, ITSM such as with incidents and service requests, and intelligence in operations. For example, the use of a web proxy helps differentiate between the traffic to the backend service, gather statistics and troubleshooting information on the clients – both mobile and desktop and to improve the security and offerings of the backend services. Similarly, IT data analytics software allow search and reporting on any kind of logs, metrics and events.  Together these empower the business to move fast on the demand areas for improvements. 


  • Allowing mobile applications and browsers to be quirky – Customers will never be happy with one stack or application. They are going to be fickle as choices explode but what will draw them back will be those applications that are mindful of their convenience and user experience.  Investing in a platform agnostic framework with security best practices such as from OWASP allows this convenience to them with no vulnerabilities or liabilities to the business. 


  • Organizing the APIs was only needed for stateful services while the stateless services allowed a namespace and nesting enabled, folder-like resource enumeration. Some techniques allow APIs to be discoverable from requests and responses. In all these cases, it is better to tailor it to the demand rather than the infrastructure or storage. Customers do not notice the difference between a data pipeline and a data storage but if we get the synchronization right, they will like their data, purchases and experiences to be carried across services. Performance is also improved when the focus on customer requirements is streamlined via dedicated services allow storage layer to take care of making it available globally and in near real time. 

Saturday, January 23, 2021

The API Layer for use by mobile applications.

Problem statement: Business all over the world have found mobile computing contributing to the topline and driving sales as customers appreciate the ubiquity and convenience of services accessed via handheld devices. Regardless of the business model, purpose and value offerings, most mobile applications require certain minimum requirements from backend services. This article attempts to enumerate some such requirements. 


Solution: The primary language of communication between mobile applications and business services are web requests made over HTTP. Fortunately, web applications have been around for over several decades leveraging clean separation of concerns and adoption of standardized application blocks, protocols, and deployment strategies. Their evolution from onerous object relational mapping in monolithic application logic and storage has shown a shift towards decentralized application modules hosted as microservices in a deeply separated hardware and software stacks hosted in the cloud. They continue to evolve with improvements made in specific pockets, but we are writing this article to pay attention to the considerations that will significantly improve the most common mode of implementation and one that will support a growing business. 


  • Cloud-empowered mobile applications: Even though large corporations are heavily invested in hybrid cloud computing; we recommend a public cloud infrastructure for attracting traffic to your business. It is a big sponge for the data that flows from customers and even with multi-tenant model on a highly efficient and cost-effective hybrid cloud computing framework, the future would be brighter with public cloud computing. Public cloud is here to stay and not just because Information technology was deprioritized to commodity computing in favor of new business processes and efficient business models. Build it right from the start. Sure, there might be some noise about cost estimates, changing landscapes and adoptions, and the insecurity about handing off your infrastructure to the cloud, but these diminish in favor of the overall value of the public cloud. 

Friday, January 22, 2021

When-to-use-what data mining algorithms ( continued )...

 

Collaborative filtering 

Recommendations include suggestions for knowledge base, or to find model service requests. In order to make a recommendation, first a group sharing similar taste is found and then the preferences of the group is used to make a ranked list of suggestions. This technique is called collaborative filtering. A common data structure that helps with keep tracking of people and their preferences is a nested dictionary. This dictionary could use a quantitative ranking say on a scale of 1 to 5 to denote the preferences of the people in the selected group.  To find similar people to form a group, we use some form of a similarity score. One way to calculate this score is to plot the items that the people have ranked in common and use them as axes in a chart. Then the people who are close together on the chart can form a group. 

Several approaches mentioned earlier provide a perspective to solving a problem. This is different from those in that opinions from multiple participants in a group is taken to determine the best set of articles or service requests to recommend. 


Thursday, January 21, 2021

When-to-use-what data mining algorithms:

 

Plugin Algorithms 

Several algorithms get customized to the domain they are applied to resulting in unconventional or new algorithms. For example, a hybrid approach on association clustering can benefit determining relevant associations when the matrix is quite large and has a large tail of irrelevant associations from the cartesian product. In such cases, some from clustering could be done prior to association to determine the key items prior to this market-basket analysis. 

IT service requests are notoriously susceptible to being opened with variations even when pertaining to the same category. These service requests do not have pre-populated fields from a template, and everyone enters values for inputs that differ from one to another. Using a hybrid approach, it is possible to preprocess these requests with clustering before analyzing such as with association clustering.  

Simultaneous classifiers and regions-of-interest regressors 

Neural nets algorithms typically involve a classifier for use with the tensors or vectors. But regions-of-interest regressors provide bounding-box localizations. This form of layering allows incremental semantic improvements to the underlying raw data. 

IT Service requests are time-series data and as more and more are opened, specific time ranges become as important as the semantic classification of the requests. Using this technique, underlying issues can be discovered as tied to outages. The determination of root cause behind a handful of service requests is valuable information. 

Algorithm Implementations: