We discussed techniques for request chaining here:  https://www.blogger.com/blogger.g?blogID=1985795500472842279#editor/target=post;postID=859017183586117344;onPublishedMenu=allposts;onClosedMenu=allposts;postNum=0;src=link
We were looking at some of the search queries that are typical of searching the logs of identity provider: 
Some other interesting events for identity include

5)  – failed attempt to login to a disabled account 

Here we look for all failed login events and filter out those that have been due to a disabled account. We do this for a given time range and then we count the events based on attributes such as those belonging to a specific account 

6) Failed versus successful logon attempts – While there may be counters for failed versus successful logon attempts globally, it requires to scan the logs for individual accounts. Here we find all the requests associated with the customer and then we filter out the ones that are logon attempts and for each logon attemps we find the corresponding success or failure response by following the sequence of requests from each such starting request. Then we split the failure versus success counts 

7) Getting a list of concurrent users – Generally this requires a time-frame where the associated  events are collected and then they are processed for detecting active distinct sessions or customers – either of which may serve the purpose. Again we filter the requests to see who have not signed out and then count it. 

8) Success rate of workflows – An identity provider may have some common use cases such as register account, sign in, change account details and forgot password. The number of failures encountered in each workflow on a customer by customer count may also indicate a form of success rate because the total number of customers calling in a 24 hour period is an easy aggregation. 

We discussed techniques for request chaining here:  https://www.blogger.com/blogger.g?blogID=1985795500472842279#editor/target=post;postID=859017183586117344;onPublishedMenu=allposts;onClosedMenu=allposts;postNum=0;src=link

Another common practice for searching the logs is activity across services. Fortunately here too the customerId can be used to propagate across the services to filter out the requests associated with the customer.

Some of the other search queries include:
1) duration of user logon time between logon/logoff events - here the requests for the user may be selectively filtered for specific security events and then the timestamps for the corresponding pair of events may be put in a table.

2) potential suspicious activity detection - Here the requests made by the user are compared in their routing paths with the known set for anomalies specifically that don't fall in known workflow sequences and then raised as suspicious

3) detecting callers - clientIds and clients identified by the programs they use can help mitigate denial of service attacks mounted by specific clients that don't behave well with others. The number of request made from the client is compared with the others in this case to see if they are repeatedly trying something that they should not.

4) Find trends in patterns - often specific failures trigger specific failure path api calls. These calls can be hashed and the counts of the hashes may indicate the most number of mitigations taken by the user. This is slightly different from directly counting the number of exceptions.

We discussed techniques for request chaining here:  https://www.blogger.com/blogger.g?blogID=1985795500472842279#editor/target=post;postID=859017183586117344;onPublishedMenu=allposts;onClosedMenu=allposts;postNum=0;src=link

The best way to establish chaining is with a common attribute specific only to the chaining. 

one such attribute is the customer id. For example, this untested but illustration only example for searching indicates the chaining is automatically established as part of selection:

#! /usr/bin/bash
#
if [ $# -ne 1 ]
then
 echo "Usage: chainer $0 customerId"
 exit $E_BADARGS
fi
listfiles=$(ls -d -1 $PWD/manaus*.*)
unset files
while IFS= read -r line
do
    files+=("$line")
done < <(echo "$listfiles")
echo $files
for file in ${files[@]}
do
  echo "file="; echo $file;
  search="zgrep -n \"^customerID=$1\" $file | cut -d\":\" -f1"
  echo $search; echo
  command_test=$(whatis "$search" | grep 'nothing appropriate');
  echo "search=";echo $search;
  unset linesno
  linesno=()
  if [[ -z "$command_test" ]]
  then
    echo "zgrep available"
    while IFS= read -r line
    do
        echo $line; echo
        linesno+=("$line")
    done < <(echo "$search")
  echo "linesno="; echo $linesno;
  for lineno in $linenos
  do
  if [! -z "$lineno" ]
  then
     echo "improper line no: $lineno"; echo
     continue;
  fi
  $text=`zcat $file | head -n $(($lineno-1+2)) | tail -n 3`
  echo $text; echo
  done
  else
  echo "no suitable command found"
  fi
done

: <<'output'
/home/ravi/manaus.tar.gz
file=
/home/ravi/manaus.tar.gz
zgrep -n "^customerID=food" /home/ravi/manaus.tar.gz | cut -d":" -f1
zgrep -n "^customerID=food" /home/ravi/manaus.tar.gz | cut -d":" -f1: nothing appropriate.
search=
zgrep -n "^customerID=food" /home/ravi/manaus.tar.gz | cut -d":" -f1
zgrep available
zgrep -n "^customerID=food" /home/ravi/manaus.tar.gz | cut -d":" -f1
linesno=
zgrep -n "^customerID=food" /home/ravi/manaus.tar.gz | cut -d":" -f1
output

Storage Locker
Data is just as precious as anything else. While storage frameworks in the cloud and on-premise promise perpetual availability and sound security, they do not offer any differentiation to data to treat is as either sensitive or not. Moreover, they may change their policies every few years and do not offer any guarantees that the data will not be handled with minimal intervention.
Companies exist for record management, secure storage and secure destruction but they usually service backup data and often manage the archives. Sensitive data on the other hand may not live in an archive but can remain in a database, unstructured data or even shared among trusted subsidiaries. Locker services does not differentiate between live and aged data.
The vulnerabilities, threats and attacks in the cloud are discussed in survey of cloud security and made publically available. These include:
1) shared technology vulnerabilities - increased leverage of resources gives the attackers a single point of attack.
2) Data breach - with data protection moving from cloud consumer to cloud service provider, the risk for data breach grows
3) Account of service traffic hijacking - Since the data moves over internet, anybody who hijacks the account could mount a loss of service
4) Denial of service - a denial of service attack on the cloud provider affects all
5) malicious insider - a determined insider can find more ways to attack and cover tracks in a cloud scenario 6) Internet protocol : IP connectivity is a requirement for data but comes with its own vulnerabilities
7) injection vulnerabilities - XSS, sql injection and other injection vulnerabilities in the management layer affect even otherwise secure data
8) API & browser vulnerabilities - vulnerability in the cloud provider's API may also affect data security
9) Changes to business models - cloud computing may require consumers to change their business models and this introduces regressions from previous security reviews
10) abusive use - cloud computing is inviting all with zero cost subscription. While it is designed to mitigate denial of service attacks, it does not stop malicious users from trying.
11) malicious insider - even insiders of a cloud provider could become malicious
12) availability - the system has to be available at all times and while cloud providers take extra ordinary efforts, they may suffer from outages such as power

One of the  trends in operational practice is to rely on tools that sets thresholds and raises alerts. This translates to incident response instead of active and strenuously polling. As part of the response, we search the logs. Most of these are interactive command line executions but each step may be time consuming due to the volume of the logs. One way to mitigate this is to run a sequential batch script that repeats the commands on smaller chunks of data. This however means we lose the aggregations unless we store intermediary data. Fortunately this was possible using files. However most log archive systems are read only and the files may not be read from. This also restricts parallelizing tasks using library such as celery because those require network access to message broker and the only access allowed is ssh. One way to overcome this is to scatter and gather data from multiple ssh sessions. This is easier to automate because the controller does not have to be local to the log server.
Another option is to leave the log server as-is and draw all the data into a log index. Then the search and reporting stacks can use the index. Since the index is designed to grow to arbitrary size, we can put all the logs in it. Also, the search stack enables as many search sessions as necessary to perform the task. They may even be made available via API, SDK and UI which enable applications to leverage parallelism as appropriate. For example, the SDK can be used with task parallel libraries such as Celery so that the same processing can be done in batches of partitioned data. The data can be partitioned based on historical timeline or they can be partitioned based on other attributes. The log index server also helps the application to preserve search artifacts so that the same can be used later or in other searches. The reporting stack sits over the search stack because the input to the reporting dashboard is the results of search queries. These search queries may be optimized, parallelized or parameterized so that they have near real-time performance. The presence of search and reporting stacks in new log indexing products indicates that these are separate areas of concerns which cannot be mixed with the conventional log readers into a monolithic console session.

Today we continue our discussion on the AWS papers in software architecture which suggests five pillars:

- Operational Excellence for running and monitoring business critical systems.

- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.

- Reliability to  recover from infrastructure or service disruptions

- Performance Efficiency to ensure efficiency in the usage of resources

- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.

The guidelines to achieve the above pillars include:

1. Infrastructure capacity should be estimated not guessed

2. Systems should be tested on production scale to eliminate surprises

3. Architectural experimentation should be made easier with automation

4. There should be flexibility to evolve architectures

5. Changes to the architecture should be driven by data

6. Plan for peak days and test at these loads to observe areas of improvement

We looked at the Operational Excellence, Reliability and security pillar and we reviewed the associated best practices.

One of the  trends in operational practice is to rely on tools that sets thresholds and raises alerts. This translates to incident response instead of active and strenuously polling. As part of the response, we search the logs. Most of these are interactive command line executions but each step may be time consuming due to the volume of the logs. One way to mitigate this is to run a sequential batch script that repeats the commands on smaller chunks of data. This however means we lose the aggregations unless we store intermediary data. Fortunately this was possible using files. However most log archive systems are read only and the files may not be read from. This also restricts parallelizing tasks using library such as celery because those require network access to message broker and the only access allowed is ssh. One way to overcome this is to scatter and gather data from multiple ssh sessions. This is easier to automate because the controller does not have to be local to the log server.

Today we continue our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We looked at the Operational Excellence, Reliability and security pillar and we reviewed the associated best practices.
#codingexercise
Find the nth multiple of k in Fibonacci Series
solution 1 : iterate through the Fibonacci Series testing and counting success
solution 2: Fibonacci multiples of a number are periodic. depending on k determine the period and hence the position of the result.
int GetNthMultipleFibonacci (int k, int n)
{
int multiple = -1;
for (int I = 0; I  < int_max; i++)
{
if (GetFibonacci (i) % k == 0){
    multiple = i + 1;
    break;
}
}
if (multiple == -1) return -1;
int position = n * multiple;
return GetFibonacci (position);
}

Today we continue our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We looked at the Operational Excellence, Reliability and security pillar and we reviewed the associated best practices.
Next we review the performance-efficiency pillar which includes the ability to use computing resources efficiently even with the fluctuations in demand and as technology evolves.
It includes five design principles. These are:
Vendor aware deployments - This implies that we don't need to host and run a new technology. Databases, machine learning, encodings are best done at the cloud level by dedicated teams so that our service may simply use it.
global availability - We deploy the system in multiple regions around the world so they provide lower latency and more availability.
serverless architectures - This notion eliminates ownership of servers for the computations and storage services act as static websites. Even the event services can be used to host the code
experiment more often - with virtual and automate-able resources, we can carry out comparative , we can quickly evaluate which T-shirt size works for us
Mechanical sympathy - This calls for using the technology that best helps us to achieve what we want with our service.
 The four best practice areas in this regard are:
Selection - As wirkloads vary, the solution becomes more nuanced about the choice of products and often involves a hybrid approach to overcome trade-offs. If the choices are done on a cyclical basis.the solution improves over time
Review - This is about evaluating newer technologies and retiring older technologies  The cloud services for example become available in new regions and upgrade their services and features.
Monitoring - This gives continuous feedback on the systems as deployed so that alarms can be set in place for actions to be taken
Trade-offs- The initial design may have considered trade-offs such as consistency, durability and space versus time or latency to deliver higher performance but these also need to be done with subsequent change management
#codingexercise
Find the nth multiple of k in Fibonacci Series
solution 1 : iterate through the Fibonacci Series testing and counting success
solution 2: Fibonacci multiples of a number are periodic. depending on k determine the period and hence the position of the result.
int GetNthMultipleFibonacci (int k, int n)
{
int multiple = -1;
for (int I = 0; I  < int_max; i++)
{
if (GetFibonacci (i) % k == 0){
    multiple = i + 1;
    break;
}
}
if (multiple == -1) return -1;
int position = n * multiple;
return GetFibonacci (position);
}

Today we continue our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We looked at the security pillar and we reviewed its best practices.
They include identity and access management, monitoring controls, infrastructure protection, data protection and incident response.
Next we review the reliability pillar which includes the ability of a system to recover from infrastructure or service disruptions, dynamically acquire computing resources to meet demand, and mitigate disruptions.
It includes five design principles:
Test recovery procedures - Cloud let us simulate failures as well as test recovery procedures because the resources are elastic With simulations we can now test and rectify before a real failure
Automatic recovery from failure : Automation can be event based so that it kicks in only when a threshold is reached.Notification and tracking of failures no longer require polling. Instead recovery and repair can be automated.
Scale horizontally to increase aggregate system reliability. We replace one large resource with multiple small resources to reduce the impact of a single resource
Capacity no longer needs to be guessed. Prior to cloud the resources were saturated frequently leading to more failures. Instead cloud lets us  monitor demand and react accordingly.
Manage change in automation - changes to the infrastructure are done using automation.
The best practice areas for reliability in the cloud include :
Foundations -  Network topology and service limits are two key criteria to establish a foundation. Often the bandwidth compute and storage limits were run-over earlier. Cloud lets us manage foundation better. In AWS, we just have to pay attention to the topology and service limits.
Change Management - Every change may violate the SLA process, therefore change control process are necessary. How a system adapts to changes in demand, how the resources are monitored and how the change is executed determine this best practice.
Failure Management - Becoming aware of failures, responding to them and preventing them from happening again are part of this best practice.
The AWS well architected framework does not provide an example of where such pillars are used in prominence in the industry today. One such example is the management of Governance Risk and Compliance (GRC) The fianancial services industry is highly regulated and has an increasing need to break the tradeoff between compliance and innovation
#codingexercise
Given a number N, find the number of ways you can draw N chords in a circle with 2*N points such that no 2 chords intersect.
If we draw a chord between any two points the set of points get divided into two smaller sets and there can be no chords going from one set to another set. On the other hand the solution for the smaller set in now an optimal sub-problem. Therefore a recurrence involving different configurations of smaller sets is possible. These configurations range in smaller set having variations from 0 to N-1 pairs

Identity – A score to who you are, what you are and where you are
Contents
Identity – A score to who you are, what you are and where you are 1
Introduction: 1
Description: 1
Conclusion: 2

Introduction:
Identity management is a necessity for every online retail business but it involves management chores such as providing various sign-in options to the users so that they may be authenticated and authorized, complying with standards and providing utmost convenience that may prove distractful to their line of business. Federated identity management stepped in to consolidate these activities. You could now sign in to different retail domains and subsidiaries with a single account. Moreover protocols were developed so that identity may be deferred to providers. Interestingly in the recent years social network providers increasingly became a venerable identity provider by themselves. This write-up introduces the notion of score for an identity as an attribute that may be passed along with the identity to subscribing identity consumers. As more and more business participate, score becomes more meaningful metadata for the customer.
Description:
Using scores to represent consumers probably started more than half a century earlier when Fair, Isaac and Co used statistical analysis to translate financial history to a simple score. We may have come a long way in how we measure credit scores for end users but the data belonged to credit bureaus. Credit card companies became authorities in tracking how consumers spend their money and their customers veritably started carrying cards instead of cash. With the rise of mobile phones, mobile payment methods started gaining popularity. Online retail companies want a share of that spend. And the only way they can authenticate and authorize a user to do so was with identity management. Therefore they shared the umbrella of identity management while maintaining their own siloed data regardless of whether they were in the travel industry, transportation industry or the insurance industry. They could tell what the user did on the last vacation, the ride he took when he was there or the claim he made when he was in trouble but there is nothing requiring them to share this data with an identity provider.  Social network and mobile applications became smarter to know the tastes the users may have or acquire and they can make ads more personalized with recommendations but there is no federation of trends and history pertaining to a user across these purchases. On the other hand, the classic problem of identity and access management has been to connect trusted users to sensitive resources irrespective of where these users are coming from and irrespective of where these resources are hosted. The term classic here is used to indicate what does not change. In contrast, business models to make these connections have changed. Tokens were invented to represent user access to a client’s resources so that the identity provider does not have to know where the resources are. Moreover, tokens were issued not only to users but also to devices and applications on behalf of the user so that they may have access to different scopes for limited time. Other access models that pertain to tokens as a form of identity are mentioned here. In the case of integrating domains and web sites with the same identity provider, the data pertaining to a customer only increases with each addition. An identity provider merely has to accumulate scores from all these retailers to make a more generalized score associated with the user. This way existing retail companies can maintain their own data while the identity provider keeps a score for the user.
Conclusion:
An identity and access management solution can look forward to more integrated collaboration with participating clients in order to increase the pie of meaningful information associated with an account holder.

Today we continue our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We looked at the security pillar and we reviewed its best practices.
They include identity and access management, monitoring controls, infrastructure protection, data protection and incident response.
The identity and access management only
allows authenticated and authorized users to access the resources. In AWS, there is a dedicated IAM service that supports multi-factor authentication.
The monitoring controls are used to identify a potential security incident. In AWS, Cloud Trail logs, AWS API calls and CloudWatch provide monitoring of metrics with alarming.
Infrastructure protection includes control methodologies which are defense in depth. In AWS, this is enforce in Compute Cloud, Container Service and Beanstalk with Amazon Machine Image.
Data protection involves techniques that involve securing data, encrypting it, and putting access controls etc.. In AWS, Amazon S3 provides exceptional resiliency.
Incident response means putting in place controls and prevention to mitigate security incidents. In AWS logging and events provide this service. AWS CloudFormation can be used to study in a sandbox kind of environment.
IAM is the AWS service that is essential security and enabled this pillar of software architecture.

#codingexercise
int GetClosest(List<int> sortedSquares, int number)
{
int start = 0;
int end = sortedSquares.Count-1;
int closest = sortedSquares[start];
while (start < end)
{
closest = Math.Abs(sortedSquares[start]-number) < Math.Abs(sortedSquares[end]-number) ? sortedSquares[start] : sortedSquares[end];
int mid = (start + end ) / 2;
if (mid == start) return closest;
if (mid == end) return closest;
if (sortedSquares[mid] == number)
{
return number;
}
if (sortedSquares[mid] < number)
{
  start = mid;
}else{
  end = mid;
}
}
return closest;
}

Today we resume our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We looked at the security pillar now we review its best practices.
They include identity and access management, monitoring controls, infrastructure protection, data protection and incident response.
The identity and access management only
allows authenticated and authorized users to access the resources.
The monitoring controls are used to identify a potential security incident.
Infrastructure protection includes control methodologies which are defense in depth
Data protection involves techniques that involve securing data, encrypting it, and putting access controls etc.
Incident response means putting in place controls and prevention to mitigate security incidents.
IAM is the AWS service that is essential security and enabled this pillar of software architecture.
#codingexercise
We were discussing how to check if a number is Fibonacci:
Check if a number is Fibonacci :
boolean is Fibonacci(uint n)
{
return IsSquare(5*n*n + 4)  || IsSquare(5*n*n - 4);
}
Another way to test for Fibonacci is to binary chop Fibonacci numbers until we get close to the given number.
We discussed this binary chop method here : http://ravinote.blogspot.com/2017/11/we-resume-our-discussion-about.html

Reducing the cyclomatic complexity in software
Introduction: Mature software applications often end up as a spaghetti code – a term used to denote tangled flow of control and the use of nested conditionals. This not only makes the software hard to read but also results in unexpected behavior. Code written in programming languages like C# and Java can be checked with tools such as NDepend and CheckStyle respectively to measure this complexity.  The following are some suggestions to mitigate it.
Description:
1: Refactoring into smaller methods or abstractions especially those that are cumbersome
2: Increased use of boolean variables or results to store intermediary state in processing or for evaluation of multiple conditions. These Boolean variables can also be repeated in subsequent statements
Test code for representing test matrix usually comes up with very little cyclomatic complexity because each test case can be represented by an if condition with multiple conditionals. While such repetitive if conditions are avoided in favor of set once and check once conditions in development code, the latter contributes to cyclomatic complexity. Unpacking the conditions into repetitive but separate lines avoids unnecessary branching and missing of test cases.
3:  Using inheritance or encapsulation or design patterns such as Factory or Strategy so that the logic can be re-organized and not just refactored. For example, multiple throw and catch statements increase cyclomatic complexity but if the catch statements are all in one place and with smaller blocks of code, it will help the same way that switch statements do. In C language, the use of pre-processors was prevalent because it made these repetitions easier to write.  A catch statement for example may have to add an entry into the log, update counters and do other chores that increase code bloat. This probably may be tolerated but putting logic into the catch handler increases this complexity.
4: Designating private methods for validations,  separate methods for business operations and treating dependency calls as merely method and parameters
5: This tip is to not to over-do the practice of reducing this complexity.  Many organizations take pride in the way they write the code such as when it looks like a textbook. This emphasizes the fact that code is of the people, by the people and for the people.
Conclusion:  Cyclomatic complexity may be a desirable metric in static code analysis practices of an organization and worthy to address at the time of checkins.

Reducing the cyclomatic complexity in software
Introduction: Mature software applications often end up as a spaghetti code – a term used to denote tangled flow of control and the use of nested conditionals. This not only makes the software hard to read but also results in unexpected behavior. Code written in programming languages like C# and Java can be checked with tools such as NDepend and CheckStyle respectively to measure this complexity.  The following are some suggestions to mitigate it.
Description:
1: Refactoring into smaller methods or abstractions especially those that are cumbersome
2: Increased use of boolean variables or results to store intermediary state in processing or for evaluation of multiple conditions. These Boolean variables can also be repeated in subsequent statements
Test code for representing test matrix usually comes up with very little cyclomatic complexity because each test case can be represented by an if condition with multiple conditionals. While such repetitive if conditions are avoided in favor of set once and check once conditions in development code, the latter contributes to cyclomatic complexity. Unpacking the conditions into repetitive but separate lines avoids unnecessary branching and missing of test cases.
3:  Using inheritance or encapsulation or design patterns such as Factory or Strategy so that the logic can be re-organized and not just refactored. For example, multiple throw and catch statements increase cyclomatic complexity but if the catch statements are all in one place and with smaller blocks of code, it will help the same way that switch statements do. In C language, the use of pre-processors was prevalent because it made these repetitions easier to write.  A catch statement for example may have to add an entry into the log, update counters and do other chores that increase code bloat. This probably may be tolerated but putting logic into the catch handler increases this complexity.
4: Designating private methods for validations,  separate methods for business operations and treating dependency calls as merely method and parameters
5: This tip is to not to over-do the practice of reducing this complexity.  Many organizations take pride in the way they write the code such as when it looks like a textbook. This emphasizes the fact that code is of the people, by the people and for the people.
Conclusion:  Cyclomatic complexity may be a desirable metric in static code analysis practices of an organization and worthy to address at the time of checkins.

Today we resume our discussion on the AWS papers in software architecture which suggests five pillars:
- Operational Excellence for running and monitoring business critical systems.
- Security to  protect information, systems, and assets with risk assessments and mitigation strategies.
- Reliability to  recover from infrastructure or service disruptions
- Performance Efficiency to ensure efficiency in the usage of resources
- Cost Optimization  to help eliminate unneeded cost and keeps the system trimmed and lean.
The guidelines to achieve the above pillars include:
1. Infrastructure capacity should be estimated not guessed
2. Systems should be tested on production scale to eliminate surprises
3. Architectural experimentation should be made easier with automation
4. There should be flexibility to evolve architectures
5. Changes to the architecture should be driven by data
6. Plan for peak days and test at these loads to observe areas of improvement
We look at the security pillar today:
The security pillar emphasizes protection of information, systems and assets. There are six design principles for security in the cloud:
It implements a strong identity foundation where the privileges are assigned on a need by need basis. and there is separation of concerns. It centralizes privilege management and reduces usage of long term credentials.
It also monitors, alerts and audits actions to respond and take actions.
It applies security at all layers and not just the one at the edge so that the impact radius is covered in full
It automates mechanisms that are necessary for controls and restrictions
It protects data in transit and at rest with the use of access tokens and encyrption
It prepares for security events when many are affected.

#codingexercise
Check if a number is Fibonacci :
boolean is Fibonacci(uint n)
{
return IsSquare(5*n*n + 4)  || IsSquare(5*n*n - 4);
}
A way to test for squares is to binary chop the squares until we find something closest to either or both of the required values

File descriptors on steroids continued
design

While the file descriptor was inherently local to the process, the DFS allowed it to point to a file on a remote computer. Likewise, file system protocols such as CIFS allowed remote servers to be connected through Active Directory and access was granted to users registered there. Deduplication worked on segments that were identical so that space could be conserved. RSync protocol helped replicate between source and destination regardless of whether the destination was a file system or a S3 endpoint. In all these tasks, much of the operations were asynchronous and involved a source and destination. This library utilizes ZeroMQ messaging library for file system operations.
Performance:
ZeroMQ has demonstrated performance for communications. The stacking of file operations from a storage solution perspective over this library can also meet the stringent requirements for cloud level operations. The implementation might vary on purpose, scope, scale and management as we add plugins for a client but the assumption that asynchronous operations on a remote file will not be hampered by ZeroMQ remains sound.

Security:
Enhanced File Descriptors are inherently as secure as sockets. However file system utilities to secure the files work because these behave the same as regular ones to the layers above.

Testing:
The implementation for this Storage Framework must be able to process hundred thousand requests per second with message sizes of a mix from 0 to 64kb for duration of 1 hour with little or no degradation in write latency for the writes to a million files. Checksum may be used to see that the files are correct. Testing might require supportability features in addition to random file writes. The statistics, audit log, history and other management aspects of the queue should be made available for pull via web APIs.

Conclusion:
With smart operating system primitives we can enhance each process to give more power to individual business.
#codingexercise

trace requests and responses from logs and stores.
#codingexercise
Get Fibonacci number
 we compared the following
uint GetTailRecursiveFibonacci(uint n, uint a = 0, uint b = 1)
{
    if (n == 0)
        return a;
    if (n == 1)
        return b;
    return GetTailRecursiveFibonacci(n-1, b, a+b);
}
with the conventional:
uint GetFibonacci (uint n)
{
    if (n == 0)
        return 0;
    if (n == 1)
        return 1;
    return GetFibonacci(n-1) + GetFibonacci(n-2);
}
0 1 1 2 3 5 8 13 21 34

Techniques for HTTP Request Tracing at a web server
Introduction: When we point our browser to a website, we hardly realize the number of exchanges of HTTP request and responses that load the page. Still it is easy to ask the browser to capture and dump the request responses in an archive file. The web server handles many such sessions from different browsers corresponding to the millions of users using it. Consequently, its logs show a mix of requests from different origins at any given time and there is little reconstruction of history. The following are some of the techniques used to tag the requests so that the sequence determination can be made.
Description: Every request to a web server is a discrete item of work for it. Since these items may be referred to later in the logs, they are issued IDs often called a RequestID or RID for short. No two requests have the same ID and its generally not possible to exceed a trillion IDs in a given duration no matter how high the load after which the sequence can rollover. Therefore, RIDs happen to be convenient to look up the requests in the logs. Logs also happen to be a convenient destination to publish the data for any production category server and is well suited for subsequent translation and interpretation by analysis systems that can pull the data without affecting the business-critical systems.
One such way to trace request responses is to establish a previous next link between the requests it makes. Between a bunch of redirects to itself, a server may stamp the RID in an additional field designated as previous RID. If it’s a new request from a user and we don't have a current session, we have none to start with. As the conversation grows the RID is piggybacked on the responses so the new requests formed at the server can have the previous RID propagated to the current. This let us establish a trace.
Another way is for the server to reuse the session ID that the requests are part of. Unfortunately, the sessions are generally restricted to a domain that the web server is hosted in and does not help with the cross domain scenario unless we include additional IDs. Since its difficult to maintain unique integer ID across disparate servers, session IDs can generally be a Unique Universal identifier which has a predefined format and almost all systems know how to generate.
Another technique is for the server to make a stash of a request which can be looked up in a stash store with a key. The stashes are generally encrypted and can include values that may not be logged for privacy. These stashes may easily be purged the same way as logs are discarded. The stashes are done for high value requests and the stashes may be maintained in a list that is piggybacked and carried forward in each conversation.
The above techniques emphasize two key concepts. The web server is the only one that can determine the sequence either by chaining the requests or by propagating a common identifier be it scoped at the session or across domains. The command line tools that serve to analyze the logs can be smart enough to search by pattern or form rich search queries that can elicit the same information. This separates the concerns and keeps the business critical systems from having to deal with the more onerous tasks. That said, there are few command line tools that can discover and render chaining as most are still aimed at regular expression based search.
Conclusion – Request chaining and chain discovery tools will be helpful to trace requests and responses from logs and stores.
#codingexercise
Get Fibonacci number by tail recursion. A tail recursion is one where the recursion is last statement in execution inside the function
uint GetTailRecursiveFibonacci(uint n, uint a = 0, uint b = 1)
{
    if (n == 0)
        return a;
    if (n == 1)
        return b;
    return GetTailRecursiveFibonacci(n-1, b, a+b);
}
0 1 1 2 3 5 8 13 21 34
TailRecursion does not involve the sum of the recursive parts

File Descriptors on steroids continued.

A library or sdk that manages cloud file operations and their logical operations within the process itself, gives more ability for the retail business to own and manage their logic without having to rely on a shared architecture. 

Proposed File Descriptors: 

- Benefits: 

  Free from public cloud policies, restrictions and management of storage tier tasks 

  Flexible design with plugin extensions for every aspect of storage framework where plugins can be appliances so long as it is in compliance with a new wire based file system protocol to be designed 

  Unified new file system protocol that spans nfs for unix, cifs/samba for windows, rsync for replication, aging and deduplication protocols that promotes interoperability 

  The library is the most performant and efficient as compared to any wire level protocols  

  works with ZeroMQ for fan-out, pub-sub, task-distribution and request-reply models 

  

- Drawbacks: 

  Clients have to build additional features themselves 

  Greater control of cloud resources comes at increased TCO 

  Vendor agnostic public cloud resource usages 

  

Differentiation: 

Public cloud provider services such as Microsoft Azure StorSimple also provide framework for storage services. 

Indeed StorSimple is documented to meet the needs of performance and capacity centric applications, and give a complete hybrid cloud storage solution for enterprises with both physical arrays for deployment and virtual arrays for satellite offices that rely on Network Accessible Storage. While StorSimple expands on the usage of Network Access Storage for point to point connectivity, we discuss a stripped down version as an embedded library. This does not require a server running on every computer as part of their operating systems that implements all the features of a distributed file system. Instead it focuses on giving the capability to write to files in a way that works with cloud resources without utilizing their native cloud storage services. For that matter this library does not restrict itself to providing DFS and can also include protocols such as for deduplication and rsync.  
#codingexercise
Get the sum of Binomial Coefficients of degree n
static int GetBinomialCoefficientsSum(int n)
{
            if (n == 0)
                return 1;
            return (1 << (n - 1));
}
This equals the row wise sum of Pascals triangle.