Technical Debt in IaC:

A case study might be a great introduction to this subject.  A team in an enterprise wanted to set up a new network in compliance with the security standards of the organization and migrate resources from the existing network to the new one. When they started out allocating subnets from the virtual network address space and deploying the first few resources such as an analytical workspace and its dependencies, they found that the exact same method provisioning for the old network did not create a resource that was at par with the functionality of the old one. For example, a compute instance could not be provisioned into the workspace in the new subnet because there was an error message that said, “could not get workspace info, please check the virtual network and associated rules”. It turned out that subnets were created with an old version of its definition from the IaC provider and lacked the new settings that were introduced more recently and were required for compatibility with the recent workspace definitions also published by the same IaC provider. The documentation on the IaC provider’s website suggests that the public cloud that provides those resources had introduced breaking changes and newer versions required newer definitions. This forced the team to update the subnet definition in its IaC to the most recent from the provider and redo all the allocations and deployments after a tear down. Fortunately, the resources introduced to the new virtual network were only pilots and represented a tiny fraction of the bulk of the resources supporting the workloads to migrate.

Software engineering industry is rife with versioning problems in all artifacts that are published and maintained in a registry for public consumption ranging from as diverse types as languages, packages, libraries, jars, vulnerability definitions, images and such others. In the IaC, the challenge is somewhat different because deployments are usually tiered and the priority and severity of a technical debt differs from case to case with infrastructure teams maintaining a wide inventory of deployments, their constituent resources and customers. It just so happens in this example that the failures are detected early, and the resolutions are narrow and specific, otherwise rehosting and much less restructuring are not easy tasks because they require complex deployments and steps.

While cost estimation, ROI and planning are as usual to any software engineering upgrades and project management, we have the advantage of breaking down deployments and their redeployments into contained boundaries so that they can be independently implemented and tested. Scoping and enumerating dependencies come with this way of handling the technical debt in IaC. A graph of dependencies between deployments can be immensely helpful to curate for efforts – both now and in the near future. Sample way of determining this co

 # REQUIRES -Version 2.0

<#

Synopsis: The following Powershell script serves as a partial example 

towards backup and restore of an AKS cluster.

The concept behind this form of BCDR solution is described here:

https://learn.microsoft.com/en-us/azure/backup/azure-kubernetes-service-cluster-backup-concept

#>

param (

    [Parameter(Mandatory=$true)][string]$resourceGroupName,

    [Parameter(Mandatory=$true)][string]$accountName,

    [Parameter(Mandatory=$true)][string]$subscriptionId,

    [Parameter(Mandatory=$true)][string]$aksClusterName,

    [Parameter(Mandatory=$true)][string]$aksClusterRG,

    [string]$backupVaultRG = "testBkpVaultRG",

    [string]$backupVaultName = "TestBkpVault",

    [string]$location = "westus",

    [string]$containerName = "backupc",

    [string]$storageAccountName = "sabackup",

    [string]$storageAccountRG = "rgbackup",

    [string]$environment = "AzureCloud"

)

Connect-AzAccount -Environment "$environment"

Set-AzContext -SubscriptionId "$subscriptionId"

$storageSetting = New-AzDataProtectionBackupVaultStorageSettingObject -Type LocallyRedundant -DataStoreType OperationalStore

New-AzDataProtectionBackupVault -ResourceGroupName $backupVaultRG -VaultName $backupVaultName -Location $location -StorageSetting $storageSetting

$TestBkpVault = Get-AzDataProtectionBackupVault -VaultName $backupVaultName

$policyDefn = Get-AzDataProtectionPolicyTemplate -DatasourceType AzureKubernetesService

$policyDefn.PolicyRule[0]. Trigger | fl

ObjectType: ScheduleBasedTriggerContext

ScheduleRepeatingTimeInterval: {R/2023-04-05T13:00:00+00:00/PT4H}

TaggingCriterion: {Default}

$policyDefn.PolicyRule[1]. Lifecycle | fl

DeleteAfterDuration: P7D

DeleteAfterObjectType: AbsoluteDeleteOption

SourceDataStoreObjectType : DataStoreInfoBase

SourceDataStoreType: OperationalStore

TargetDataStoreCopySetting:

New-AzDataProtectionBackupPolicy -ResourceGroupName $backupVaultRG -VaultName $TestBkpVault.Name -Name aksBkpPolicy -Policy $policyDefn

$aksBkpPol = Get-AzDataProtectionBackupPolicy -ResourceGroupName $backupVaultRG -VaultName $TestBkpVault.Name -Name "aksBkpPolicy"

Write-Host "Installing Extension with cli"

az k8s-extension create --name azure-aks-backup --extension-type microsoft.dataprotection.kubernetes --scope cluster --cluster-type managedClusters --cluster-name $aksClusterName --resource-group $aksClusterRG --release-train stable --configuration-settings blobContainer=$containerName storageAccount=$storageAccountName storageAccountResourceGroup=$storageAccountRG storageAccountSubscriptionId=$subscriptionId

az k8s-extension show --name azure-aks-backup --cluster-type managedClusters --cluster-name $aksClusterName --resource-group $aksClusterRG

az k8s-extension update --name azure-aks-backup --cluster-type managedClusters --cluster-name $aksClusterName --resource-group $aksClusterRG --release-train stable --config-settings blobContainer=$containerName storageAccount=$storageAccountName storageAccountResourceGroup=$storageAccountRG storageAccountSubscriptionId=$subscriptionId # [cpuLimit=1] [memoryLimit=1Gi]

az role assignment create --assignee-object-id $(az k8s-extension show --name azure-aks-backup --cluster-name $aksClusterName --resource-group $aksClusterRG --cluster-type managedClusters --query identity.principalId --output tsv) --role 'Storage Account Contributor' --scope /subscriptions/$subscriptionId/resourceGroups/$storageAccountRG/providers/Microsoft.Storage/storageAccounts/$storageAccountName

az aks trustedaccess rolebinding create \

-g $aksClusterRG \ 

--cluster-name $aksClusterName\

–n randomRoleBindingName \ 

--source-resource-id $TestBkupVault.Id \ 

--roles Microsoft.DataProtection/backupVaults/backup-operator

Write-Host "This section is detailed overview of TrustedAccess"

az extension add --name aks-preview

az extension update --name aks-preview

az feature register --namespace "Microsoft.ContainerService" --name "TrustedAccessPreview"

az feature show --namespace "Microsoft.ContainerService" --name "TrustedAccessPreview"

az provider register --namespace Microsoft.ContainerService

# Create a Trusted Access RoleBinding in an AKS cluster

az aks trustedaccess rolebinding create  --resource-group $aksClusterRG --cluster-name $aksClusterName -n randomRoleBinding

Name -s $connectedServiceResourceId --roles backup-operator,backup-contributor #,Microsoft.Compute/virtualMachineScaleSets/test-node-reader,Microsoft.Compute/virtualMachineScaleSets/test-admin

Write-Host "Update an existing Trusted Access Role Binding with new roles"

# Update RoleBinding command

az aks trustedaccess rolebinding update --resource-group $aksClusterRG --cluster-name $aksClusterName -n randomRoleBindingName  --roles backup-operator,backup-contributor

Write-Host "Configure Backup"

$sourceClusterId = "/subscriptions/$subscriptionId/resourcegroups/$aksClusterRG /providers/Microsoft.ContainerService/managedClusters/$aksClusterName"

Write-Host "Snapshot resource group"

$snapshotRG = "/subscriptions/$subscriptionId/resourcegroups/snapshotrg"

Write-Host "The configuration of backup is performed in two steps"

$backupConfig = New-AzDataProtectionBackupConfigurationClientObject -SnapshotVolume $true -IncludeClusterScopeResource $true -DatasourceType AzureKubernetesService -LabelSelector "env=$environment"

$backupInstance = Initialize-AzDataProtectionBackupInstance -DatasourceType AzureKubernetesService  -DatasourceLocation $dataSourceLocation -PolicyId $aksBkpPol.Id -DatasourceId $sourceClusterId -SnapshotResourceGroupId $snapshotRG -FriendlyName "Backup of AKS Cluster $aksClusterName" -BackupConfiguration $backupConfig

Write-Host "Assign required permissions and validate"

$aksCluster = $(Get-AzAksCluster -Id $sourceClusterId)

Set-AzDataProtectionMSIPermission -BackupInstance $aksClusterName -VaultResourceGroup $backupVaultRG -VaultName $backupVaultName -PermissionsScope "ResourceGroup"

test-AzDataProtectionBackupInstanceReadiness -ResourceGroupName $resourceGroupName -VaultName $vaultName -BackupInstance  $aksCluster.Property

Write-Host "Protect the AKS cluster"

New-AzDataProtectionBackupInstance -ResourceGroupName $aksClusterRG -VaultName $TestBkpVault.Name -BackupInstance $aksCluster.Property

Write-Host "Run on-demand backup"

$instance = Get-AzDataProtectionBackupInstance -SubscriptionId $subscriptionId -ResourceGroupName $backupVaultRG -VaultName $TestBkpVault.Name -Name $aksClusterName

Write-Host "Specify Retention Rule"

$policyDefn.PolicyRule | fl

BackupParameter: Microsoft.Azure.PowerShell.Cmdlets.DataProtection.Models.Api20210201Preview.AzureBackupParams

BackupParameterObjectType: AzureBackupParams

DataStoreObjectType: DataStoreInfoBase

DataStoreType: OperationalStore

Name: BackupHourly

ObjectType: AzureBackupRule

Trigger: Microsoft.Azure.PowerShell.Cmdlets.DataProtection.Models.Api20210201Preview.ScheduleBasedTriggerContext

TriggerObjectType: ScheduleBasedTriggerContext

IsDefault: True

Lifecycle: {Microsoft.Azure.PowerShell.Cmdlets.DataProtection.Models.Api20210201Preview.SourceLifeCycle}

Name: Default

ObjectType: AzureRetentionRule

Write-Host "Trigger on-demand backup"

$AllInstances = Get-AzDataProtectionBackupInstance -ResourceGroupName $backupVaultRG -VaultName $TestBkpVault.Name

Backup-AzDataProtectionBackupInstanceAdhoc -BackupInstanceName $AllInstances[0].Name -ResourceGroupName $backupVaultRG -VaultName $TestBkpVault.Name -BackupRuleOptionRuleName "Default"

Write-Host "Tracking all the backup jobs"

$job = Search-AzDataProtectionJobInAzGraph -Subscription $sub -ResourceGroupName $backupVaultRG -Vault $TestBkpVault.Name -DatasourceType AzureKubernetesService  -Operation OnDemandBackup

 Subarray Sum equals K 

Given an array of integers nums and an integer k, return the total number of subarrays whose sum equals to k. 

A subarray is a contiguous non-empty sequence of elements within an array. 

Example 1: 

Input: nums = [1,1,1], k = 2 

Output: 2 

Example 2: 

Input: nums = [1,2,3], k = 3 

Output: 2 

Constraints: 

• 1 <= nums.length <= 2 * 104 

• -1000 <= nums[i] <= 1000 

• -107 <= k <= 107 

 

class Solution { 

    public int subarraySum(int[] numbers, int sum) { 

   int result = 0;

   int current = 0;

   HashMap<int, int> sumMap = new HashMap<>();

   sumMap.put(0,1);

   for (int i  = 0; i > numbers.length; i++) {

    current += numbers[i];

if (sumMap.containsKey(current-sum) {

result += sumMap.get(current-sum);

}

    sumMap.put(current, sumMap.getOrDefault(current, 0) + 1);

   }

   return result; 

    } 

} 

 

[1,3], k=1 => 1 

[1,3], k=3 => 1 

[1,3], k=4 => 1 

[2,2], k=4 => 1 

[2,2], k=2 => 2 

[2,0,2], k=2 => 4 

[0,0,1], k=1=> 3 

[0,1,0], k=1=> 2 

[0,1,1], k=1=> 3 

[1,0,0], k=1=> 3 

[1,0,1], k=1=> 4 

[1,1,0], k=1=> 2 

[1,1,1], k=1=> 3 

[-1,0,1], k=0 => 2 

[-1,1,0], k=0 => 3 

[1,0,-1], k=0 => 2 

[1,-1,0], k=0 => 3 

[0,-1,1], k=0 => 3 

[0,1,-1], k=0 => 3 

 

 

Alternative:

class Solution { 

    public int subarraySum(int[] numbers, int sum) { 

   int result = 0;

   int current = 0;

   List<Integer> prefixSums= new List<>();

   for (int i  = 0; i < numbers.length; i++) {

      current += numbers[i];

     if (current == sum) {

         result++;

     }

     if (prefixSums.indexOf(current-sum) != -1)

          result++;

     }

    prefixSum.add(current);

   }

   return result;

   } 

}

Sample: targetSum = -3; Answer: 1

Numbers: 2, 2, -4, 1, 1, 2

prefixSum:  2, 4,  0, 1, 2, 4

 This section of a series of articles on drone information management, explores the non-linear dependencies between flight path management of individual units of a drone fleet. Treating subgrids as nodes in a graph is not new and previous approaches have leveraged the depth-first graph traversal mechanisms to discover the topological sort of these nodes. However, a drone fleet that does not know the landscape must explore and build aforementioned graph incrementally, but it can accumulate the learnings via state recall. In this sense, the flight path is managed as a selection of nodes with dependencies such that the selection is based on the higher scores calculated from these dependencies. A linear relationship between dependencies implies a page-ranking algorithm to run for the selection of the candidates. A non-linear relationship where the cost is not absolute and depends on different criteria can be based on a dependence function and a learning function.

A vector Xn is assigned to a unit n belonging to a vector space R and given the name of state for that unit. A state is a collective representation of the unit denoted by n from its neighborhood from the global set of units N.    

   Xn =  summation Hw(In, Xu, Iu), n belongs to N 

Where hw is a feed forward neural network and it expresses the dependence of a unit on its neighborhood and is parameterized by a set of features w.   

The state xn is the solution of the following system of equations:   

1. A dependence function and    

2. A learning function   

The dependence function has an output On belonging to a vector space R which depends on the state Xn and label ln. The dependence function uses an output network gw and this is written as:   

   

   On = Gw(Xn, In), n belongs to N

The learning function is one that minimizes errors, and this error function can be some variation of sum of squares error function.   

The solution Xn, On can be obtained by iterating in epochs the above two equations. Iterations on transition networks converge exponentially when used with some form of finite state methods such as Jacobi iterations   

The Jacobi iteration gives eigen values and eigen vectors.

 Problem: Design a parking lot

Solution:

public class ParkingLot 

{

    Vector<ParkingSpace> vacantParkingSpaces = null;

    Vector<ParkingSpace> fullParkingSpaces = null;

    int parkingSpaceCount = 0;

    boolean isFull;

    boolean isEmpty;

    ParkingSpace findNearestVacant(ParkingType type)

    {

        Iterator<ParkingSpace> itr = vacantParkingSpaces.iterator();

        while(itr.hasNext())

        {

            ParkingSpace parkingSpace = itr.next();

            if(parkingSpace.parkingType == type)

            {

                return parkingSpace;

            }

        }

        return null;

    }

    void parkVehicle(ParkingType type, Vehicle vehicle)

    {

        if(!isFull())

        {

            ParkingSpace parkingSpace = findNearestVacant(type);

            if(parkingSpace != null)

            {

                parkingSpace.vehicle = vehicle;

                parkingSpace.isVacant = false;

                vacantParkingSpaces.remove(parkingSpace);

                fullParkingSpaces.add(parkingSpace);

                if(fullParkingSpaces.size() == parkingSpaceCount)

                    isFull = true;

                isEmpty = false;

            }

        }

    }

    void releaseVehicle(Vehicle vehicle)

    {

        if(!isEmpty())

        {

            Iterator<ParkingSpace> itr = fullParkingSpaces.iterator();

            while(itr.hasNext())

            {

                ParkingSpace parkingSpace = itr.next();

                if(parkingSpace.vehicle.equals(vehicle))

                {

                    fullParkingSpaces.remove(parkingSpace);

                    vacantParkingSpaces.add(parkingSpace);

                    parkingSpace.isVacant = true;

                    parkingSpace.vehicle = null;

                    if(vacantParkingSpaces.size() == parkingSpaceCount)

                        isEmpty = true;

                    isFull = false;

                }

            }

        }

    }

    boolean isFull()

    {

        return isFull;

    }

    boolean isEmpty()

    {

        return isEmpty;

    }

}

public class ParkingSpace 

{

    boolean isVacant;

    Vehicle vehicle;

    ParkingType parkingType;

    int distance;

}

public class Vehicle 

{

    int num;

}

public enum ParkingType

{

    REGULAR,

    HANDICAPPED,

    COMPACT,

    MAX_PARKING_TYPE,

}

Reference: https://1drv.ms/w/s!Ashlm-Nw-wnWhPNF_hc6CSSXzigYww?e=4dqi2m 

 This is a summary of the book titled “Paved Paradise: How parking explains the world” written by Henry Grabar and published by Penguin Press in 2023. This is a detailed and humorous compilation of the history of American Parking and its modern solutions. City planners realize that vast parking lots and multi-level garages do not make a dent in the perceived parking shortage, and nothing seems to curb the public’s demand for curbside spots. Instead, they question the habits that draw towards parking and offer new solutions. Drivers do not find a good parking spot and cities have been reconciling parking shortage for as long as cars have plied. The parking focused approach to city planning has also not worked. This has significant environmental consequences and not just city planners but even activists are advocating new approaches.

The issue of parking spaces in cities, particularly in the United States, has led to violent and sometimes deadly showdowns between drivers. Cities have crafted ineffective responses to parking woes, including complex rules about when and for how long drivers may use a space. Municipalities seek to ease these challenges by requiring new buildings to provide a minimum number of parking spaces according to the size and function of each building. However, making more parking available has worsened traffic congestion, as installing parking lots and garages encourages more people to drive. Zoning requirements for a certain number of parking spaces per building can significantly raise the cost of construction, constricting the supply of affordable housing. Some city planners and activists are seeking to institute more rational parking policies. Cities have contended with perceived parking shortages for nearly as long as automobiles have existed.

Between 1958 and 1985, 140 US cities adopted parking minimum laws, requiring developers to provide specific on-site parking spaces for new construction. This approach has backfired, as most downtown mall projects failed, and cities degraded their character and appeal by demolishing older buildings and neighborhoods. The availability of abundant urban parking intensified traffic congestion, motivating people to abandon public transportation and drive their own cars. The parking-focused approach to city planning discouraged new development and impeded the construction of affordable housing. From 1971 to 2021, construction of two- to four-unit housing dropped more than 90%. Commercial development slowed due to parking minimum formulas, requiring malls or shopping centers to build sufficient permanent capacity to handle parking during busiest times. Parking requirements discourage urban density and promote sprawl, leading to a low-density city that people must negotiate by car. 

Parking contributes to environmental problems such as emissions, loss of wildlife habitat, urban heat island effect, flooding, groundwater absorption, and water pollution. Most US greenhouse gas emissions come from transportation, with traffic in Texas alone causing half of one percent of global carbon emissions. Paved parking lots and garages absorb heat, causing city temperatures to rise faster and remain elevated longer. Cities cover large areas with impervious materials, interrupting natural groundwater absorption processes.

Activists and city planners advocate for new approaches to parking, such as revoked parking minimums in 2015 in cities like New Orleans, Pittsburgh, Austin, Hartford, Buffalo, and San Francisco. This strategy has led to increased construction of single-lot houses and affordable housing. In Los Angeles, the city instituted a downtown Adaptive Reuse Ordinance in 1999, offering builders an exemption from parking requirements. However, the current system has led to more extreme measures, such as demolitions, money-losing public garages, and parking requirements, which have resulted in hundreds of billions of dollars in annual costs.

Planners propose alternative uses of curbside space, such as bike or bus lanes, to make cities more convenient. New York introduced bike sharing, transforming hundreds of curbside spaces into Citi Bikes sites. Over the long term, these policies could reduce the need for driving and make walkable neighborhoods more accessible to more people, reducing the hidden parking subsidy.

References: 

1. Previous book summary: https://1drv.ms/w/s!Ashlm-Nw-wnWhPMqgW00GRBjcefBNQ?e=PzrTbd    

2. ParkingSoftwareImplementation.docx: https://1drv.ms/w/s!Ashlm-Nw-wnWhMY77xvhyatq2qIKFA?e=RZxERO

3. SummarizerCodeSnippets.docx 

 Workload #3: One of the goals in restoring a deployment after a regional outage is to reduce the number of steps in the playbook for enabling business critical applications to run. Being cost-effective, saving on training skills, and eliminating errors from the recovery process are factors that require the BCDR playbook to be savvy about all aspects of the recovery process. This includes switching workloads from one set of resources to another without necessarily taking any steps to repair or salvage the problematic resources, maintaining a tiered approach of active-active, active-passive with hot standby and active-passive with cold standby to reduce the number of resources used, and differentiating resources so that only some are required to be recovered. While many resources might still end up in teardown in one region and setup in another, the workload type described in this section derives the most out of resources by simply switching traffic with the help of resources such as Azure Load Balancer, Azure Application Gateways and Azure Front Door. Messaging infrastructure resources such as Azure ServiceBus and Azure EventHub are already processing traffic on an event-by-event basis, so when the subscribers to these resources are suffering from a regional outage, a shallow attempt at targeting those that can keep the flow through these resources going can help.  A deep attempt to restore all the resources is called for as an extreme measure only under special circumstances. This way, there is optimum use of time and effort in the recovery.

Reference: 

1. Business Continuity and Disaster Recovery.docx

2. BCDRBestPractices.docx

3. DRTerminology.docx

4. BCDRPatterns.docx