The following is a Terraform module to create an Azure dashboard for a specific user supplied content and query. It quickly deploys a panel for a query on a supported metric that can be authored in KQL externally and specified here via a URL to the query in the dashboard definitions. Dashboard panel and visualizations can be created from the Shared dashboards URL: https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.Portal%2Fdashboards. 

 

provider "azurerm" { 

  features {} 

} 

 

data "azurerm_subscription" "current" {} 

 

resource "azurerm_resource_group" example_rg { 

  name     = var.resource_group 

  location = "Central US" 

} 

 

resource "azurerm_portal_dashboard" example_dashboard_name { 

  name                = var.dashboard_name 

  resource_group_name = azurerm_resource_group.example_rg.name 

  location            = azurerm_resource_group.example_rg.location 

  tags = { 

    source = "terraform" 

  } 

  dashboard_properties = <<DASH 

{ 

   "lenses": { 

        "0": { 

            "order": 0, 

            "parts": { 

                "0": { 

                    "position": { 

                        "x": 0, 

                        "y": 0, 

                        "rowSpan": 5, 

                        "colSpan": 5 

                    }, 

                    "metadata": { 

                        "inputs": [], 

                        "type": "Extension/HubsExtension/PartType/MarkdownPart", 

                        "settings": { 

                            "content": { 

                                "settings": { 

                                    "content": "${var.query_content}", 

                                    "subtitle": "", 

                                    "title": "${var.dashboard_title}" 

                                } 

                            } 

                        } 

                    } 

                }, 

                "1": { 

                    "position": { 

                        "x": 5, 

                        "y": 0, 

                        "rowSpan": 5, 

                        "colSpan": 6 

                    }, 

                    "metadata": { 

                        "inputs": [], 

                        "type": "Extension/Microsoft_Azure_Monitoring/PartType/MetricsChartPart", 

                        "settings": { 

                            "content": { 

                                "settings": { 

                                    "title": "Important Information", 

                                    "subtitle": "", 

                                    "src": "${var.query_link}", 

                                    "autoplay": true 

                                } 

                            } 

                        } 

                    } 

                } 

            } 

        } 

    }, 

    "metadata": { 

        "owner": "${var.dashboard_owner_email}", 

        "model": { 

            "timeRange": { 

                "value": { 

                    "relative": { 

                        "duration": 24, 

                        "timeUnit": 1 

                    } 

                }, 

                "type": "MsPortalFx.Composition.Configuration.ValueTypes.TimeRange" 

            }, 

            "filterLocale": { 

                "value": "en-us" 

            }, 

            "filters": { 

                "value": { 

                    "MsPortalFx_TimeRange": { 

                        "model": { 

                            "format": "utc", 

                            "granularity": "auto", 

                            "relative": "24h" 

                        }, 

                        "displayCache": { 

                            "name": "UTC Time", 

                            "value": "Past 24 hours" 

                        }, 

                        "filteredPartIds": [ 

                            "StartboardPart-UnboundPart-ae44fef5-76b8-46b0-86f0-2b3f47bad1c7" 

                        ] 

                    } 

                } 

            } 

        } 

    } 

} 

DASH 

} 

 

As with all Terraform modules, this can be run via the following Terraform commands: 

terraform init 

terraform plan 

terraform apply 

and the command terraform destroy can be run when this resource is no longer needed. 

 

This is a continuation of the previous posts on Azure Data Platform and discusses the considerations for a specific scenario of moving data from an on-premises IBM object storage to Azure storage.

Organization of storage assets to support governance, operational management and accounting requirements is necessary for data migration to the cloud. Well-defined naming and metadata tagging conventions help to quickly locate and manage resources. These conventions also help to associate cloud usage costs with business teams via chargeback and showback accounting mechanisms.

Proper naming is essential for security purposes but tagging improves metadata. They serve different purposes.  The naming includes parts that indicate the business unit or project owner. Names could also have parts for workload, application, environment, criticality and other such information. Tags could also include these names but enhance the metadata that do not necessarily need to be reflected in the name.

An effective naming strategy includes one with:

<resourceType>-<workload/application>-<environment>-<region>-<instance>.

For example,

rg-projectx-nonprod-centralus-001

is an example of a good naming convention for a resource group.

The projectx here refers to the name of the project or business capability but it could comprise multiple parts for hierarchical information such as contoso-fin-navigator for <organization>-<department>-<service>

Similarly, the <instance> could comprise of <role>-<instanceSuffix> for example vm-transactional-nonprod-centralus-db-001.

For storage accounts, the project name can contain spaces, but these could be removed. There might be no option to substitute the space character with a hyphen. Container names permit hyphens. A container name must be between 3-63 characters long and all lowercase.

Existing naming conventions can be used, if they have been serving adequately and are compatible with the public cloud naming restrictions.

It is preferable to introduce containers for buckets and storage accounts for namespaces between destination and source mapping on a one-on-one basis.

The current limits for an on-premises IBM object storage include the following:

100 buckets per object storage instance.

10 TB max per object

Unlimited number of objects in an instance.

1024-character key length

Storage classes at bucket level.

Changing storage class requires manually copying data from one bucket to another.

Archive independently of storage class.

IBM COS is accessible via S3 protocol.

And these are well within the limits of Azure storage account

The following script can be used to upload local files and folders to the Azure storage account from an on-premises machine using the AzCopy utility which is a command line tool that can be used to copy blobs or files to or from a storage account, and then transfer data. The tool requires authentication but allows unattended login via a security principal.  It also resumes a previous execution of the copy command with help of journaling. A custom location for the journal folder can be specified via the –resume option. Azure Data Lake Gen2 storage works with only the latest versions of the AzCopy such as v10 onwards. For multi-region deployments, it is recommended to have the data landing in one region and then replicated globally using AzCopy.

#!/bin/sh

throw() {

  echo "$*" >&2

  exit 1

}

 

STORAGE_ACCOUNT_NAME=

CONTAINER_NAME=

LOCAL_FOLDER_PATH=

 

usage() {

  echo

  echo "Usage: $(basename $0) -b arg -c arg -l arg [-h]"

  echo

  echo "-b - The name of the blob storage account."

  echo "-c - The name of the container."

  echo "-l - The name of the local system folder."

  echo "-h - This help text."

  echo

}

 

parse_options() {

while getopts ':b:l:c:h' opt; do

  case "$opt" in

    b)

      STORAGE_ACCOUNT_NAME="$OPTARG"

      ;;

    l)

      LOCAL_FOLDER_PATH="$OPTARG"

      ;;

    c)

      CONTAINER_NAME="$OPTARG"

      ;;

    h)

      echo "Processing option 'h'"

      usage

      exit 0

      ;;

    :)

      echo "option requires an argument.\n"

      usage

      exit 1

      ;;

    ?)

      echo "Invalid command option.\n"

      usage

      exit 1

      ;;

  esac

done

shift "$(($OPTIND -1))"

}

parse_options "$@"

if ([ -z "$LOCAL_FOLDER_PATH" ] || [ -z "$STORAGE_ACCOUNT_NAME" ] || [ -z "$CONTAINER_NAME" ]);

then 

  echo "Invalid command.\n"

  usage

  exit 1

fi

./azcopy login 

./azcopy copy "$LOCAL_FOLDER_PATH" "https://$STORAGE_ACCOUNT_NAME.blob.core.windows.net/$CONTAINER_NAME" --recursive=true 

./azcopy sync "$LOCAL_FOLDER_PATH" "https://$STORAGE_ACCOUNT_NAME.blob.core.windows.net/$CONTAINER_NAME" --recursive=true 

# crontab -e

# */5 * * * * sh /path/to/upload.sh

The script can be made to run periodically so that the delta changes can also be propagated.

 

#codingexercise

Problem Statement: A 0-indexed integer array nums is given.

Swaps of adjacent elements are able to be performed on nums.

A valid array meets the following conditions:

·       The largest element (any of the largest elements if there are multiple) is at the rightmost position in the array.

·       The smallest element (any of the smallest elements if there are multiple) is at the leftmost position in the array.

Return the minimum swaps required to make nums a valid array.

 

Example 1:

Input: nums = [3,4,5,5,3,1]

Output: 6

Explanation: Perform the following swaps:

- Swap 1: Swap the 3^rd and 4^th elements, nums is then [3,4,5,3,5,1].

- Swap 2: Swap the 4^th and 5^th elements, nums is then [3,4,5,3,1,5].

- Swap 3: Swap the 3^rd and 4^th elements, nums is then [3,4,5,1,3,5].

- Swap 4: Swap the 2^nd and 3^rd elements, nums is then [3,4,1,5,3,5].

- Swap 5: Swap the 1^st and 2^nd elements, nums is then [3,1,4,5,3,5].

- Swap 6: Swap the 0^th and 1^st elements, nums is then [1,3,4,5,3,5].

It can be shown that 6 swaps is the minimum swaps required to make a valid array.

Example 2:

Input: nums = [9]

Output: 0

Explanation: The array is already valid, so we return 0.

 

Constraints:

·         1 <= nums.length <= 10⁵

·         1 <= nums[i] <= 10⁵

Solution:

class Solution {

    public int minimumSwaps(int[] nums) {

        int min = Arrays.stream(nums).min().getAsInt();

        int max = Arrays.stream(nums).max().getAsInt();

        int count = 0;

        while (nums[0] != min && nums[nums.length-1] != max && count < 2 * nums.length) {           

            var numsList = Arrays.stream(nums).boxed().collect(Collectors.toList());

            var end = numsList.lastIndexOf(max);

            for (int i = end; i < nums.length-1; i++) {

                swap(nums, i, i+1);

                count++;

            }

 

            numsList = Arrays.stream(nums).boxed().collect(Collectors.toList());

            var start = numsList.indexOf(min);

            for (int j = start; j >= 1; j--) {

                swap(nums, j, j-1);

                count++;

            }

        }

 

        return count;

    }

 

    public void swap (int[] nums, int i, int j) {

        int temp = nums[j];

        nums[j] = nums[i];

        nums[i] = temp;

    }

}

 

Input

nums =

[3,4,5,5,3,1]

Output

6

Expected

6

 

Input

nums =

[9]

Output

0

Expected

0