GitOps is as much a part of Azure Infrastructure engineering as anything native to the public cloud. The convenience that git repositories and associated workflows provide is not specific to the public clouds and can span sovereign clouds and on-premises. In this regard, a few methods need to be called out.

Assuming a workflow file has been authored in the .github/workflows folder of a repository, it is very much like an automation script that can be shared and re-used in different workflows and that is not restricted to just GitHub workflows. All you need is a personal access token. For example,

curl -X POST \

-H "Accept: application/vnd.github.v3+json" \

-H "Authorization: token YOUR_PERSONAL_ACCESS_TOKEN" \

https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches \

-d '{"ref":"main"}

The advantages of GitOps includes auditing which is similar to that of the public cloud. Every run of the workflow is recorded including who did it and when. Secrets and variables allow the workflow to be parameterized and this can be done with the help of another request just prior to the run. For example,

curl -X PUT \

-H "Accept: application/vnd.github.v3+json" \

-H "Authorization: token YOUR_PERSONAL_ACCESS_TOKEN" \

https://api.github.com/repos/OWNER/REPO/actions/secrets/SECRET_NAME \

-d '{"encrypted_value": "NEW_ENCRYPTED_VALUE", "key_id": "KEY_ID"}'

This helps particularly in the case when the Owner or owning organization to the repository has policies in place to require any change to the files in the repository to be done via pull-requests where manual approvals are necessary.

GitOps, therefore, provides version tracking and file sharing convenience that can be packaged to run with dedicated accounts that have very specific fine-grained access to what resources are specified to be acted upon by the workflow file.

A user interface or any floating ui component in any portal can also use GitOps as backend instead of the actual resource, thereby, providing convenience to how an automation is run.

Reference: Previous1 articles (IaCResolutionsPart261.docx: https://1drv.ms/w/c/d609fb70e39b65c8/EZtfWh6GSp5ElYh8itkjwBkBXexIdT-xGnqwQqcrQZk-cQ?e=DwZ2BI).

 

Location queries

Location is a datatype. It can be represented either as a point or a polygon and each helps with answering questions such as getting top 3 stores near to a geographic point or stores within a region. Since it is a data type, there is some standardization available. SQL Server defines not one but two data types for the purpose of specifying location: the Geography data type and the Geometry data type.  The Geography data type stores ellipsoidal data such as GPS Latitude and Longitude and the geometry data type stores Euclidean (flat) coordinate system. The point and the polygon are examples of the Geography data type. Both the geography and the geometry data type must have reference to a spatial system and since there are many of them, it must be used specifically in association with one. This is done with the help of a parameter called the Spatial Reference Identifier or SRID for short. The SRID 4326 is the well-known GPS coordinates that give information in the form of latitude/Longitude. Translation of an address to a Latitude/Longitude/SRID tuple is supported with the help of built-in functions that simply drill down progressively from the overall coordinate span.  A table such as ZipCode could have an identifier, code, state, boundary, and center point with the help of these two data types. The boundary could be considered the polygon formed by the zip and the Center point as the central location in this zip. Distances between stores and their membership to zip can be calculated based on this center point. Geography data type also lets us perform clustering analytics which answers questions such as the number of stores or restaurants satisfying a certain spatial condition and/or matching certain attributes. These are implemented using R-Tree data structures that support such clustering techniques. The geometry data type supports operations such as area and distance because it translates to coordinates.   It has its own rectangular coordinate system that we can use to specify the boundaries or the ‘bounding box’ that the spatial index covers.

The operations performed with these data types include the distance between two geography objects, the method to determine a range from a point such as a buffer or a margin, and the intersection of two geographic locations. The geometry data type supports operations such as area and distance because it translates to coordinates. Some other methods supported with these data types include contains, overlaps, touches, and within.

A note about the use of these data types now follows. One approach is to store the coordinates in a separate table where the primary keys are saved as the pair of latitude and longitude and then to describe them as unique such that a pair of latitude and longitude does not repeat. Such an approach is questionable because the uniqueness constraint for locations has a maintenance overhead. For example, two locations could refer to the same point and then unreferenced rows might need to be cleaned up. Locations also change ownership, for example, store A could own a location that was previously owned by store B, but B never updates its location. Moreover, stores could undergo renames or conversions.  Thus, it may be better to keep the spatial data associated in a repeatable way along with the information about the location. Also, these data types do not participate in set operations. That is easy to do with collections and enumerable with the programming language of choice and usually consist of the following four steps: answer initialization, return an answer on termination, accumulation called for each row, and merge called when merging the processing from parallel workers. These steps are like a map-reduce algorithm. These data types and operations are improved with the help of a spatial index. These indexes continue to be like indexes of other data types and are stored using B-Tree. Since this is an ordinary one-dimensional index, the reduction of the dimensions of the two-dimensional spatial data is performed by means of tessellation which divides the area into small subareas and records the subareas that intersect each spatial instance. For example, with a given geography data type, the entire globe is divided into hemispheres and each hemisphere is projected onto a plane. When that given geography instance covers one or more subsections or tiles, the spatial index would have an entry for each such tile that is covered.  The geometry data type has its own rectangular coordinate system that you define which you can use to specify the boundaries or the ‘bounding box’ that the spatial index covers. Visualizers support overlays with spatial data which is popular with mapping applications that super-impose information over the map with the help of transparent layers. An example is the Azure Maps with GeoFence as described here¹.

References:

1.      Azure Maps and GeoFence: https://1drv.ms/w/s!Ashlm-Nw-wnWhKoMrJB6VrX06DHN-g?e=dWJIgv

 This is a different approach from the generic approach described for UAV swarm flight path management in uncharted territory as described by an earlier article1 using waypoint selections and trajectory smoothing. There are two key differentiating characteristics of UAV swarms that can provide additional insights. One is that the UAV swarm is not necessarily always a one-follow-another sequence through waypoints and trajectory. In fact the formation is dynamic and the center of the UAV swarm has its own position, altitude, velocity and distance from references than the leading and trailing units. Another is that the units are themselves capable of carrying a variety of sensors such as LiDAR data that can be stored and queried and in the cloud to build the knowledge base of the landscape via data processing systems similar to image capturing ones discussed earlier2. The commercial UAV swarm serving cloud-software based solution3 does not limit the types and capabilities of the drones or the storing and querying of the data from the sensors. The generic approach treated each unit of the UAV swarm as a co-ordinate but this approach outlined below makes use of both these factors.

The center of the UAV swarm regardless of its formation and changes to formations when following trajectories through waypoints can also be tracked in addition to each unit of the swarm. The scatter plots of position vs time, altitude vs time, speed vs time, distance from reference point versus time, inter UAV distance vs time and velocity components versus time demonstrate elongated scatter plots which lends itself to correlation. Therefore, logistic regression can be used to predict various aspects of the flight path of the center of the UAV swarm. Logistic regression differs from the other Regression techniques in the use of statistical measures. Regression is very useful to calculate a linear relationship between a dependent and independent variable, and then use that relationship for prediction. This technique is suitable for specific aspects of the flight path for the center of the UAV swarm that doesn’t always have to be the same drone. One advantage of logistic regression is that the algorithm is highly flexible, taking any kind of input, and supports several different analytical tasks. Use of MicrosoftML rxFastTree is recommended for this purpose.

The gradient boost algorithm for rxFastTree is possible with several loss functions including the squared loss function. The algorithm for the least squares regression can be written as :

1. Set the initial approximation

2. For a set of successive increments or boosts each based on the preceding iterations, do

3. Calculate the new residuals

4. Find the line of search by aggregating and minimizing the residuals

5. Perform the boost along the line of search

6. Repeat 3,4,5 for each of 2.

 The ability to store and query LiDAR data exemplifies the extremes in storage and computing requirements. LiDAR uses laser light to measure distances and create highly accurate 3D maps. It captures millions of data points per second, providing detailed information about the environment around each unit of the UAV Swarm. Selecting appropriate software for processing LiDAR data is crucial. Some cloud-based tools and frameworks include PDAL (Point Data Abstraction Library), PCL (Point Cloud Library), Open3D, Entwine, and TerraSolid. These tools can help to build LiDAR processing pipelines. LiDAR data processing includes data ingestion, preprocessing, classification, feature extraction, and analysis and visualization. Each step requires significant computational resources, especially for large datasets. Cloud-based storage solutions offer scalability, flexibility, cost-effectiveness, accessibility, speed and seamless collaboration making them ideal for handling massive LiDAR datasets. Cloud storage providers also offer advanced security features and robust backup mechanisms.

Over ten LiDAR companies are public in the US, with many key players in Europe and Asia. Competition among manufacturers is driving down prices, making LiDAR feasible for various markets. Standardization and regulation has not come about leading to frustrations with varied specifications. Open standard data formats are essential for flexibility and efficiency. Integrating multiple sensors adds calibration and synchronization challenges and raw point cloud data is complex to interpret without expert help. Complexity in processing raw data remains a challenge for real-time applications due to the millions of data points captured per second. The amount of 3D data is rapidly increasing, complicating real-time interpretation. Most advancements in computer vision focus on 2D data, making 3D LiDAR processing complex. Real-time LiDAR applications need actionable insights rather than just raw data. Manufacturers focus on technical specifications, but practical applications require problem-solving insights. Integrating LiDAR into applications is challenging and can lead to costly mistakes. In fact, existing solutions focus on LiDAR-agnostic strategy to support a wide range of sensors. Software solutions that leverage LiDAR software processor expedite real-time application development and insights. Comprehensive features from the software processor and RESTful API for easy integration comes in handy for automations and infrastructure deployments. No one questions LiDAR solutions to provide the state-of-the-art and anonymous spatial intelligence data for improved UAV operations.

Deep learning shows promising results for processing tasks associated with UAV-based image and LiDAR data especially in improving classification, object detection, and semantic segmentation tasks in remote sensing. There are some challenges in using deep learning for such imagery due to difficulty of acquiring sufficient labeled samples. Convolutional Neural Network aka CNNs with object detection is the most common approach. This highlights the importance of real-time processing, domain adaptation, and few-shot learning as potentially emerging technologies. Deep learning architecture complexity must be reduced while maintaining accuracy. Quantization techniques can reduce memory requirements for deep learning models. Domain adaptation and transfer learning are essential for addressing UAV imagery and Generative Adversarial Networks aka GANs are a promising approach for aligning source and target domains. Attention mechanisms improve feature extraction in high-resolution remote sensing images, enhancing tasks like segmentation and object detection. Contrastive loss holds promise in improving model performance. As yet, certain photogrammetric processing such as dense point cloud generation and orthomosaic creation are still unexplored for UAV imageries. With improvement of more labeled datasets, deep learning neural networks can become more generalized for UAV swarms.

 

This is a continuation of articles on text to speech Azure AI services. The earlier article¹ discussed the production of mp3 audio for short texts up to 5000 characters that can be sent via a single API call. This article discusses the conversion of large text with the help of Batch Synthesis API for text to speech. This is helpful for the creation of AudioBooks and even though Speechify ranks high in AI Voice Generator and ElevenLabs ranks high in AI Voice cloning, this will be good enough.

 

import requests

import json

import time

from docx import Document

import os

import uuid

 

# Azure AI Language Service configuration

endpoint = "https://<your_region>.api.cognitive.microsoft.com/texttospeech/batchsyntheses/JOBID?api-version=2024-04-01"

api_key = "<your_api_key>"

 

headers = {

    "Content-Type": "application/json",

    "Ocp-Apim-Subscription-Key": api_key

}

 

def synthesize_text(inputs):

    body = {

        "inputKind": "PlainText", # or SSML

        'synthesisConfig': {

            "voice": "en-US-GuyNeural",

        },

        # Replace with your custom voice name and deployment ID if you want to use custom voice.

        # Multiple voices are supported, the mixture of custom voices and platform voices is allowed.

        # Invalid voice name or deployment ID will be rejected.

        'customVoices': {

            # "YOUR_CUSTOM_VOICE_NAME": "YOUR_CUSTOM_VOICE_ID"

        },

        "inputs": inputs,

        "properties": {

            "outputFormat": "audio-48khz-192kbitrate-mono-mp3"

        }

    }

 

    response = requests.put(endpoint.replace("JOBID", str(uuid.uuid4())), headers=headers, json=body)

    if response.status_code < 400:

        jobId = f'{response.json()["id"]}'

        return jobId

    else:

        raise Exception(f"Failed to start batch synthesis job: {response.text}")

 

def get_synthesis(job_id: str):

    while True:

        url = f'https://<your_region>.api.cognitive.microsoft.com/texttospeech/batchsyntheses/{job_id}?api-version=2024-04-01'

        headers = {

           "Content-Type": "application/json",

           "Ocp-Apim-Subscription-Key": api_key

        }

        response = requests.get(url, headers=headers)

        if response.status_code < 400:

            status = response.json()['status']

            if "Succeeded" in status:

               return response.json()

            else:

               print(f'batch synthesis job is still running, status [{status}]')

               time.sleep(5)  # Wait for 5 seconds before checking again

 

def get_text(file_path):

    with open(file_path, 'r') as file:

        file_contents = file.read()

    print(f"Length of text: {len(file_contents)}")

    return file_contents

 

# Main execution

if __name__ == "__main__":

    input_file_name = ""

    large_text = ""

    inputs = []

    for i in range(2,7):

        input_file_name=f"{i}.txt"

        print(input_file_name)

        if input_file_name:

           document_text = get_text(input_file_name)

           inputs += [

              {

                "content": document_text

              },

           ]

   

    # Start summarization job

    jobId = synthesize_text(inputs)

    print(jobId)

    # Get summary result

    audio = get_synthesis(jobId)

   

    print("Result:")

    print(audio)

 

Sample result:

Result:

{'id': '8cdbd29d-43f3-4878-b83c-b5326688e302', 'status': 'Succeeded', 'createdDateTime': '2025-03-08T08:16:42.8628654Z', 'lastActionDateTime': '2025-03-08T08:16:55.1972806', 'inputKind': 'PlainText', 'customVoices': {}, 'properties': {'timeToLiveInHours': 744, 'outputFormat': 'audio-48khz-192kbitrate-mono-mp3', 'concatenateResult': False, 'decompressOutputFiles': False, 'wordBoundaryEnabled': False, 'sentenceBoundaryEnabled': False, 'sizeInBytes': 38024640, 'succeededAudioCount': 5, 'failedAudioCount': 0, 'durationInMilliseconds': 1584360, 'billingDetails': {'neuralCharacters': 27710}}, 'synthesisConfig': {'voice': 'en-US-GuyNeural'}, 'outputs': {'result': 'https://stttssvcproduse.blob.core.windows.net/batchsynthesis-output/da113ddc2b524d9e8b95c6f6b6ab2a61/8cdbd29d-43f3-4878-b83c-b5326688e302/results.zip?skoid=12345678-6c19-4f12-8d9f-57c205aaba10&sktid=33e01921-4d64-4f8c-a055-5bdaffd5e33d&skt=2025-03-07T13%3A02%3A37Z&ske=2025-03-13T13%3A07%3A37Z&sks=b&skv=2023-11-03&sv=2023-11-03&st=2025-03-08T08%3A11%3A59Z&se=2025-03-11T08%3A16%3A59Z&sr=b&sp=rl&sig=s0DIH6g6gryEgmDEHlbd2ilqC5xfuB2J7HJ%2FddOlHcA%3D'}}

 

Minimum adjacent swaps to make a valid array:

You are given a 0-indexed integer array nums.

Swaps of adjacent elements are able to be performed on nums.

A valid array meets the following conditions:

• The largest element (any of the largest elements if there are multiple) is at the rightmost position in the array.

• The smallest element (any of the smallest elements if there are multiple) is at the leftmost position in the array.

Return the minimum swaps required to make nums a valid array.

Example 1:

Input: nums = [3,4,5,5,3,1]

Output: 6

Explanation: Perform the following swaps:

- Swap 1: Swap the 3rd and 4th elements, nums is then [3,4,5,3,5,1].

- Swap 2: Swap the 4th and 5th elements, nums is then [3,4,5,3,1,5].

- Swap 3: Swap the 3rd and 4th elements, nums is then [3,4,5,1,3,5].

- Swap 4: Swap the 2nd and 3rd elements, nums is then [3,4,1,5,3,5].

- Swap 5: Swap the 1st and 2nd elements, nums is then [3,1,4,5,3,5].

- Swap 6: Swap the 0th and 1st elements, nums is then [1,3,4,5,3,5].

It can be shown that 6 swaps is the minimum swaps required to make a valid array.

Example 2:

Input: nums = [9]

Output: 0

Explanation: The array is already valid, so we return 0.

Constraints:

• 1 <= nums.length <= 105

• 1 <= nums[i] <= 105

class Solution {

    public int minimumSwaps(int[] nums) {

        int min = Arrays.stream(nums).min().getAsInt();

        int max = Arrays.stream(nums).max().getAsInt();

        int count = 0;

        while (nums[0] != min && nums[nums.length-1] != max && count < 2 * nums.length) {

            var numsList = Arrays.stream(nums).boxed().collect(Collectors.toList());

            var end = numsList.lastIndexOf(max);

            for (int i = end; i < nums.length-1; i++) {

                swap(nums, i, i+1);

                count++;

            }

            numsList = Arrays.stream(nums).boxed().collect(Collectors.toList());

            var start = numsList.indexOf(min);

            for (int j = start; j >= 1; j--) {

                swap(nums, j, j-1);

                count++;

            }

        }

        return count;

    }

    public void swap (int[] nums, int i, int j) {

        int temp = nums[j];

        nums[j] = nums[i];

        nums[i] = temp;

    }

}

 

 This article answers the following questions:

1. Can there be multiple static web sites hosted on the same storage account behind the front door in different containers?

2. Can there be different domains associated with each of the static websites?

3. Will the routing and caching change when navigating directly to that path or via domain?

4. Will there be a need to divert traffic on path basis or keep it at apex level for each static website.

Answers:

1. With or without the use of an FD, a storage account can host multiple static websites but the organization differs significantly. The most straightforward approach is to create separate storage accounts for each static website as called out by 1 and 2 in the references

2. Website can be reached via $web if the static website setting is enabled: on the storage account. All paths to any assets in $web can be reached directly via the primary endpoint shown in the screenshot above. This makes it convenient to share assets across different usages and even websites if they make a web request to the same endpoint and path for that asset. Treat this just like permalink for assets that must not change location for many callers.

3. Dedicated websites that differ in purpose must be separated as website1 and website2 folders. In this case, they can both be nested in the $web container but their respective folders will become part of the path. This becomes an acceptable solution even without fd so long as the primary endpoint can be tolerated to be the same for both websites and to have the website1 and website2 as static path and part of url. This completes the built-in support for one or more static websites with just a storage account.

4. Many applications want to avoid path qualifiers in the url simply because it reflects the actual location to a caller who could have been given that asset with an alias so that it is easy to move around the actual location for planned or unplanned reasons.

5. It is at this point that FD comes helpful because 1. you have the luxury to use as many aliases to separate your business usages. Since $web is default, even a single alias can hide the actual storage account while leveraging /website1 and /website2 as earlier because fd has no knowledge about those paths and doesn't do any routing other than to the $web as a backend member that it refers to as origin. 2. you can nest your website anywhere in the storage account without regard to $web.

6. Aliases aka subdomains are also inexpensive and having them reflect the environment or business purpose makes it easy to tell apart any traffic without a lookup. By adding separate origins to send traffic from separate aliases keeps the routing and caching with default values which prevents unexpected behaviors when they are set with custom settings. More than routing, custom caching settings also called cdn settings in fd routes often give a different response for the same request at different times unless the caller leverages purging either on flushed/ad-hoc basis or the built-in scheduled basis. So keeping everything simple naturally moves to having different storage accounts for different purposes/environments.

7. FD also has sophistications for routing and caching that should also be called out because they are simply not possible with a storage account. For example, An FD can allow a static website to be hosted in any container outside the $web as well because it treats that as a path within the origin. The change to the routing is that FD has a rewrite functionality similar to an app-gateway where specifying a path qualifier such as /site1 can be replaced with a full path list /path/to/site1 while leaving the rest of the unmatched parts of the url including the endpoint and the individual asset file with query parameters intact. A sample screenshot is included for this purpose.

8. This successfully helps to translate moniker urls that do not have actual location such as https://foo.optum.com/site1/ to actually translate to "https://foo.optum.com/path/to/index.html" where foo.optum.com automatically reaches the destination storage account without relying on the internal support of that storage account leveraging $web. When testing with this option the following limitations were observed and appear to noteworthy:

a. the pattern match including wild card characters is very tricky to capture all edge cases where callers may or may not include a slash at the end of their url.

b. the only other option to rewrite that helps with routing to multiple websites is redirect and that significantly introduces a whole new set of issues.

The conclusion from the above two in terms of the features available with app gateway indicate that FD has poor support for path based routing but a rich support for mapping 1 apex to 1 origin. In fact, the combination of these two limitations forces deployments of fd to separate origins to different origin groups. while it is easy to leverage priority and weight to distribute of all the traffic specificed by a route to more than one origin in the origin group, such activity is only for load balancing and not for user traffic splitting to different websites.

9. The caching settings include a checkbox and drop down on each route. There are default settings for this that work well with the scheduled purges of the FrontDoor but customizations based on compression and query parameter allow to target different asset types and associated key-values in the urls. If there are further relays before the fd and to the caller, these can introduce significant complexity.

10. The ability to have the TLS work for each alias because the certificate automatically renews elsewhere and is not stored on the front door or in a key vault allows the convenience that each apex level can work independently and maintenance-free which is why it directly shows how many days before the certificate expires on the fd itself but automatically rolls over.

References:

1. https://learn.microsoft.com/en-us/answers/questions/254127/hosting-multiple-static-websites-on-azure-blob-sto

2. https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-static-website

3. https://1drv.ms/w/c/d609fb70e39b65c8/EboOa84huEROjJO7OPF1GwkBFo7QmjvquCoNm_YmLzzJ1w?e=ciAlDa

#codingexercise: CodingExercise-03-06-2025.docx