Saturday, December 16, 2017



We were discussing the benefits for client-server separation of computing so that client specific improvements can happen independent of the server. While server side user controls, grids and views have been tremendously popular and continue to evolve with Model-View-Controller, Model-View-ViewModel techniques along with templates and rendering engines, they are only for the convenience of organization and the developers because all the code and the resources are available all at once for mix and match in any page to be rendered. On the other hand, client side javascripting can take care of most of these request and also have their own libraries while working in a platform independent and browser independent way. If we give the resources to the client side library and ask the client to display the page, the server doesn't need to do them while facilitating developments on the client side. While HTML5 and client side scripts have preferred brevity in their source, the mix and match processing when offloaded to the client can still be brief as long as all the resources are available as a bundle for the duration of the page or session.

One of the advantages of client side scripting is the significant amount of support for development that we get from browser and plugins. Some security is better enforced from the server side but allowing computations on public resources can easily be dedicated to the client side.

Let us look at few of the security concerns in the client side technologies. For example, most of the scripts on the client side is done with javascript. One of the most common concerns with Javascript is that it can be injected by third parties. Another concern is that it is often used to append or prepare html with string concatenations. Even server side html injection has been shown to be vulnerable but client side is notorious. Second browsers have to enforce cross origin resource sharing which previously allowed cross domain Ajax requests. Third the processing is utterly visible to all. The XHR API shows not only the data that is exchange but also the calls made. A view page source is enough to show how cluttered the client side can get
One of the most common vulnerability seem with client side scripting is that input and injection attacks are not thwarted by way of mitigation. Although HTTP headers are used to provide some amount of security, they rely on the best practice from both origination and destination  Even SQL can be injected directly through the user interface. Locale and geographical customizations often make it harder to enforce a rule much less to talk about user agents, browsers,  apps and devices. While HTML5,  JSON and CSS have each addressed this with their own separation of concerns and brevity, much of the script is still left to the programmer to organize. Technologies on in this end tried to innovate on functionalities with flash and ActiveX but users had to disable some of these for the sake of security. Instead of we could come up with newer organization, more brevity and separation of concerns within resources and scripts so that we have far more security and less surface.

Friday, December 15, 2017

If a web page is viewed as an html, text usually appears as span within a div. These div only indicate the position but their content is decided by the rendering engine such as Razr. The literals in the html are looked up based on a identifier that acts as its alias. The same alias may be used for the translations of the string in different languages. Since the (alias, locale) pair is unique, we know that there will be a unique translation for the string. When the view is prepared, these literals are pulled from the database. Usually this preparation happens on the server side. If it were assets such as css, javascript or static html, then they could be served by the content delivery network which ensures that they are as geographically close as possible to the user. String are resources too but they don't appear as a resource on the CDN because there is a computation involved. But technically each string may be placed in its own static file and the sections of the html that display the content may instead serve the literal via loading a file. Round trips from a CDN are generally not considered expensive but given that there may be many literals on a page, this could become expensive. Each CDN may also need to keep its own copy of the literals. This means we don't have the single point maintenance on the server side as we did earlier. Moreover, updates and deletes of the string involves making changes to all CDN. But it is not impossible to offload the literals as a resource so that html can load it on the client side without having the server to incur the work.
The above discussion merely marks a benefit for client-server separation of computing so that client specific improvements can happen independent of the server. While server side user controls, grids and views have been tremendously popular and continue to evolve with Model-View-Controller, Model-View-ViewModel techniques along with templates and rendering engines, they are only for the convenience of organization and the developers because all the code and the resources are available all at once for mix and match in any page to be rendered. On the other hand, client side javascripting can take care of most of these request and also have their own libraries while working in a platform independent and browser independent way. If we give the resources to the client side library and ask the client to display the page, the server doesn't need to do them while facilitating developments on the client side. While HTML5 and client side scripts have preferred brevity in their source, the mix and match processing when offloaded to the client can still be brief as long as all the resources are available as a bundle for the duration of the page or session.

Thursday, December 14, 2017

We were discussing  SDK and command line interface CLI options in addition to the APIs which improve the customer base.
Let us now talk about locale. Every user belongs to a geography and hence has a preferred language for the login page. When the user is displayed a web page, most of the strings on the web page are translated to a language of his or her choice. These translations can be made available with the user interface as language packs so that the corresponding translations may show up on the page. Care must be taken that all translations exist in every locale supported by the user interface. Also, when a language specified is invalid or a translation could not be found by an identifier, the corresponding default strings from english language can be displayed.
If a web page is viewed as an html, text usually appears as span within a div. These div only indicate the position but their content is decided by the rendering engine such as Razr. The literals in the html are looked up based on a identifier that acts as its alias. The same alias may be used for the translations of the string in different languages. Since the (alias, locale) pair is unique, we know that there will be a unique translation for the string. When the view is prepared, these literals are pulled from the database. Usually this preparation happens on the server side. If it were assets such as css, javascript or static html, then they could be served by the content delivery network which ensures that they are as geographically close as possible to the user. String are resources too but they don't appear as a resource on the CDN because there is a computation involved. But technically each string may be placed in its own static file and the sections of the html that display the content may instead serve the literal via loading a file. Round trips from a CDN are generally not considered expensive but given that there may be many literals on a page, this could become expensive. Each CDN may also need to keep its own copy of the literals. This means we don't have the single point maintenance on the server side as we did earlier. Moreover, updates and deletes of the string involves making changes to all CDN. But it is not impossible to offload the literals as a resource so that html can load it on the client side without having the server to incur the work.

Wednesday, December 13, 2017

We were discussing  SDK and command line interface CLI options in addition to the APIs which improve the customer base. Tests and automations can make use of commands and scripts to drive the service. This is a very powerful technique for administrators as well as for day to day usage by end users. Command line usage has one additional benefit - it can be used for troubleshooting and diagnostics. While SDKs provide an additional layer and are hosted on the client side, command line interface may be available on the server side and provide fewer variables to go wrong. With detailed logging and request-response capture CLI and SDK help ease the calls made to the services. 
Let us now consider applying the SDK and CLI to something that Is inherently UI specific – say the login page. The login page has a few more restrictions that other user interface probably don't have. First, it is central to the company where the login happens. People login because they trust the page they are on. This means the login page is inherently tied to the domain the web application hosted in. Even if a third party application wants to allow its user to login, it will redirect the users to login on the company page. Since a user interface is involved in specifying username and password, it might not seem likely to delegate it to a command line or sdk because that would involve sending the credentials in the clear. However transmitting the credentials can be done over encrypted channel as it is routed from one service to another. Besides we have never intended the user to provide the credentials anywhere other than the login page. The suggestion is merely to decouple the UI from being in the response body to being in a standalone web view that calls a service Since both the service and the UI need to available and hosted on the same domain, it doesn't matter whether the credentials get passed from the UI to the service via a form submit or via a relay from one service to another. The UI and service for the signin are merely shifting boundaries between the app and the service.  
Second users are accustomed to not disclosing their credentials. These work differently from key-value pairs. Many cloud providers use key-value pair in their SDKs and CLI. They know that their services such as compute and storage will be used in applications and services and therefore they find it easier to ship SDKs and CLIs. A user interface is a frontend only to a user and not an application or a digital asset. Therefore, providing an SDK or CLI does not hold as much appeal as a login page. At the same time, separating out the view for the user and focusing on the internal architecture with the help of SDK and CLI helps bring the best practice from the cloud to the retail. 
Lastly users are comforted knowing that they are signing in to a look and feel that they are familiar with. Therefore a login page is a frontend that users expect not to change. By the very same argument, if we pull the user interface out of the API response and let the UI be part of an MVC that makes internal service call under-neath, then we are less likely to make changes to the login page as the service gets revisioned. Moreover, the customizations that we need to make for third party, clients and end-users now don't need to run as deep as the service. 


#codingexercise


We were discussing finding closest numbers between the elements of three sorted integer arrays. 

Another usage of finding the closest numbers in these arrays to a given input value is to discover overlapping range of sorted numbers. In order to do this on number line, we could choose max(start1,start2,start3) and min(end1,end2,end3) for (start,end) pair for every array. And by finding the closest of the min and max values, we can determine the overlapping range projection on any one of three arrays. Then we can choose the projection with the smallest number of elements 

Determining the overlapping region projections helps shrink the search space for any operations involving any or all three arrays. It may not be useful for sum, average, mode or median but it is helpful for partitioning the number line to determine the presence of a number in any array. If the number is not in the overlapping range then we exclude one of the arrays and binary search for it outside the projections in other two arrays.


  1. One such usage is goes back to determine the closest numbers of a given value. If the number is not included in the projections, we know that we can search in a smaller space 

Tuesday, December 12, 2017

We continue discussing software improvements from earlier posts. We were talking about Services available via APIs. Their versioning and organization have an industry wide recognized best practice and are therefore, quite staple. At the same time presence of user interface in the API responses is also a viable option for improving interactions with end users
As services morph from old to new they may also change their dependencies both in the version and in the kind of service. While user interface may become part of the responses as the response body, it may be better to separate the UI as a layer from the responses so that the service behind the API may be consolidated while the UI may be customized on different clients. This keeps customization logic away from the services while enabling the clients to vary on plugins, browsers, applications and devices. Some request responses may continue to keep their UI because they may not accept any other form of data transfer but the XMLHttpRequest that transfers the data from the browser to the server is still a client based technology and doesn't need to be part of the server response body. Another reason why servers choose to keep their responses to include user facing forms and controls is that they want to enforce same origin and strict client registrations and their redirections. By requiring that some of their APIs to be internal, they also restrict others from making similar calls. APIs do have a natural address, binding and contract properties that allow their endpoints to be secured but client technologies on the other hand do not require such hard and fast rules. Moreover, with services relaying data via calls, strict origin registration and control may still be feasible.
Another way to work with services is to write a SDK so that clients don’t have to make the REST calls directly. This is a best practice favored by many cloud providers who know their cloud computing capability will be called from a variety of clients and devices. By exposing the resources directly as objects and methods they enable far easier integration rather than with individually crafting the API calls. Moreover a variety of programming languages can be used to provide the SDK which widens the client base. The SDKs are provided as a mere convenience so this could be something that the developer community might initiate and maintain.SDK. Another benefit to using the SDK is that now the API versioning and service maintenance onus is somewhat reduced as more customers use the SDK rather than the wire calls. SDKs also support platform independent and interoperable programming which helps test and automation.
Similarly command line interface CLI also improve the customer base. Tests and automations can make use of commands and scripts to drive the service. This is a very powerful technique for administrators as well as for day to day usage by end users. Command line usage has one additional benefit - it can be used for troubleshooting and diagnostics. While SDKs provide an additional layer and are hosted on the client side, command line interface may be available on the server side and provide fewer variables to go wrong. With detailed logging and request-response capture CLI and SDK help ease the calls made to the services.

Monday, December 11, 2017

We continue discussing software improvements from earlier posts.
API versioning and organization have an industry wide recognized best practice and are therefore, quite staple. At the same time presence of user interface in the API responses is also a viable option for improving interactions with end users
API versioning is mentioned as a best practice for clients and consumers to upgrade And there is one very useful property of the HTTPS protocol where a request can be redirected so that the users may be kept informed at the very least or their calls translated to newer versions in full service solutions. It is true however that versioning is probably the only way for providing an upgrade path to users. In order that we don't take on the onus of backward compatibility, the choice to offer both old and new becomes clearer. It is also true that not all customers or consumers can move quickly to adopt new APIs and the web service then takes the onus of maintaining the earlier behavior in some fashion. As web services mature and they become increasingly hard and unwieldy, they do not have any other way to maintain all the earlier behavior without at least relieving the older code base from newer features and providing them in a new offering.
As services morph from old to new they may also change their dependencies both in the version and in the kind of service. While user interface may become part of the responses as the response body, it may be better to separate the UI as a layer from the responses so that the service behind the API may be consolidated while the UI may be customized on different clients. This keeps customization logic away from the services while enabling the clients to vary on plugins, browsers, applications and devices. Some request responses may continue to keep their UI because they may not accept any other form of data transfer but the XMLHttpRequest that transfers the data from the browser to the server is still a client based technology and doesn't need to be part of the server response body. Another reason why servers choose to keep their responses to include user facing forms and controls is that they want to enforce same origin and strict client registrations and their redirections. By requiring that some of their APIs to be internal, they also restrict others from making similar calls. APIs do have a natural address, binding and contract properties that allow their endpoints to be secured but client technologies on the other hand do not require such hard and fast rules. Moreover, with services relaying data via calls, strict origin registration and control may still be feasible.

Sunday, December 10, 2017

We reflected on a few ideas of improvement for software related to identity management here.  We continue with a few more ideas. As software matures, many engineers choose to be dainty about the improvements they make often requiring very little change to existing data paths or processes. They do this for several reasons. First, because code reuse and original design decisions work favorably. Second altering existing code paths often runs the risk of regression. Third the cost of churn generally rises higher than the benefits. Also, much of the changes are usually in the form of technical debt – a term used to refer to the corrections needed as features are released by taking shortcuts which create instant debt. It is assumed that this debt is paid as customers pay for the usage of the new features. Moreover, it is easier to add workflows to existing software by reading data and transforming it for analysis and reporting given that the online processing needs to be as efficient and streamlined as possible so it remains available to the customers.  And finally, the organization may find it easier make value propositions with vertical silos instead of horizontal modifications. While databases used to form the shared data infrastructure and services were differentiated for the value propositions, today organizations prefer to move from infrastructure view to microservice model with the introduction of new services and the retiring of old services.
Regardless of the intention, scale, scope and impact of the changes to the code base, most improvements suffer from the malaise that there are not enough tests and that it is too expensive and even prohibitive to have the adequate surface coverage for acceptance. It is widely believed that most of the tests are investments for feature launches as well as customer satisfaction. Therefore, tests are just as much currency as any other artifact. The presence of a well written test can enforce expectations from the software as early as design time and all the way through the software development cycle. In this regard, one of the improvements we could consider is to increase API based testing for the service. Generally, an n-tier service has tests at all levels starting from the database unit tests at the back-end, api level tests at the service layer, and front-end tests for the front end. Out of these, the design of the APIs and their tests poses the maximum benefit in the tradeoffs for value of tests and the technical debts. At the same time all non-functional aspects such as security and performance can still be covered. Yet APis are hard to keep tidy in the face of ever increasing improvements and business changes to the software. APIs like most customer facing software interactions suffer from the same liabilities over time. Fortunately, the versioning and retirement policies can be enforced for all clients with the added advantage that more and more clients accept HTTPS based APIs over other forms of interface. As devices, platforms and ecosystems change, the APIs present an evergreen investment that can stand the test of time better than others. Moreover, they naturally lend themselves to automation, diagnostics and monitoring than most other software.
API versioning and organization have an industry wide recognized best practice and are therefore, quite staple. At the same time presence of user interface in the API responses is also a viable option for improving interactions with end users