Today we review some the logic to automate commonly recurring tasks with vCenter.
1)  to list all Virtual Machines:
     we create a container view with a view Type as vim.VirtualMachine. Then we list the container which is the root folder.
        content = service_instance.RetrieveContent()
        container = content.rootFolder
        viewType = [vim.VirtualMachine]
        recursive = True
        containerView = content.viewManager.CreateContainerView(
            container, viewType, recursive)
Each item in the container view is now a virtual machine instance.

2) to find a specific VM:
we create a search index. All VMs have their own uuid so we can use that to search the index:

search_index = si.content.searchIndex
vm = search_index.FindByUuid(None, args.uuid, True, True)

3) Display vm information
From the above, the virtual machine instance has the following:

details = {'name': vm.summary.config.name,
           'instance UUID': vm.summary.config.instanceUuid,
           'bios UUID': vm.summary.config.uuid,
           'path to VM': vm.summary.config.vmPathName,
           'guest OS id': vm.summary.config.guestId,
           'guest OS name': vm.summary.config.guestFullName,
           'host name': vm.runtime.host.name,
           'last booted timestamp': vm.runtime.bootTime,
           }

4) Export a VM so that it can be backed up:
The main task here is to create the ovf descriptor for the VM.  This is a portable format. We do this only if the VM is powered off.
To create the ovf, we first get an NFCLease on the VM. Then for each device Url in the lease, we download the device to a temporary target directory For each downloaded file, we create an ovf file and set its attribute Then we ask the OvfManager to create a descriptor with ovf files we specify and write this descriptor which is an xml file to the same temporary target directory. This temporary target directory with all the files is the backup.
            http_nfc_lease = vm_obj.ExportVm()
            if http_nfc_lease.state == vim.HttpNfcLease.State.ready:
               for deviceUrl in http_nfc_lease.info.deviceUrl:
                    temp_target_disk = os.path.join(target_directory,
                                                    deviceUrl.targetId)
                    current_bytes_written = download_device(
                        headers=headers, cookies=cookies,
                        temp_target_disk=temp_target_disk,
                        device_url=deviceUrl.url,
                        lease_updater=lease_updater,
                        total_bytes_written=total_bytes_written,
                        total_bytes_to_write=total_bytes_to_write)
                    total_bytes_written += current_bytes_written
                    ovf_file = vim.OvfManager.OvfFile()
                    ovf_file.deviceId = deviceUrl.key
                    ovf_file.path = deviceUrl.targetId
                    ovf_file.size = current_bytes_written
                    ovf_files.append(ovf_file)
           ovf_manager = si.content.ovfManager
           ovf_parameters = vim.OvfManager.CreateDescriptorParams()
           ovf_parameters.name = vm_descriptor_name
           ovf_parameters.ovfFiles = ovf_files
           vm_descriptor_result = ovf_manager.CreateDescriptor(obj=vm_obj,
                                                            cdp=ovf_parameters)

Design considerations for backup of large files from vCenter:

Files in the vCenter can be upwards of a hundred GB. Several files may need to be downloaded before they are archived using tools like rsync or bbcp. If the number of files for simultaneous download is large and the size of each file is large, even multiple streams may not be sufficient to collapse the transfer time. Moreover, the local storage needed for repackaging these files can be arbitrarily large.

The optimum solution would be to read the source as stream and write to destination with the same stream.  However, since repackaging is involved, a local copy and transfer can only be addressed with the following techniques:

1.       Keep the local file only until it is repackaged and transferred to destination

2.       Compress and archive the packaged file  before transfer

3.       Maintain a database with the mappings for the source and destination files, their metadata and status for retries

4.       Parallelize based on each file and not folders. This gives more granularities.

5.       Use a task parallel library such as Celery along with a message broker for each transfer of a file.

6.       Use of tools like duplicity may require either the source or the destination to be local. This means they need to be invoked twice for using a local copy as temporary. If the repackaging is not permissible at source, may be the repackaging can be attempted at destination. This works well for remote file storage.

7.       The storage for files must be adequately large to support n number of active downloads of an earmarked size.

8.       There must be policy to prevent more than a certain number of active downloads. This can be facilitated with the bookkeeping status in the database.

9.       Instead of using transactions, it would be helpful to enable states ad retries for such transfers.

Overall, the local storage option is expensive and when it is unavoidable, the speed of the transfer, the number of active transfers, the ease of parallelization and the robustness against failures together with retry logic addresses these pain points.

#codingexercise
 Shuffle a deck of cards:
           Knuth Shuffling :
                  split the cards into 1 to I and I+1 to n-1.
                   pick a card from 1 to I randomly and uniformly (I times)
                   replace with random number card between I+1 to n-1
           Fisher-Yates shuffling:
                   loop over an array
                   swap each element with a random element past the iteration point

void shuffle(ref List<int> cards)
{
var random = new Random()
 for (int i = cards.Count-1; i > 0; i--)
{
      int j = random.next(0,i);
      Swap(ref cards, i, j);
}
}

#codingexercise
Given an array of n numbers with repetition of numbers. You need to find the max length of continuous sub array with at max 3 unique elements.

int maxSubArrayLen(List<int> nums)
{
int len = 0;
int len_window = 0;
for (int i =0; i < nums.Count; i++)
{
len_window = len_window + 1;
if (nums.GetRange(i-len_window, len_window).distinct() > 3 )
     len_window = 0;
else if (len < len_window)
     len = len_window;
}
return len;
}

Take two trees. Invert the second and add it to the bottom of the first tree. Do they have nodes that overlap ?
void GetLeafNodes(node root, ref List<node> result)

{

if (root == null) return;

if (root.left == null && root.right == null) result.add(root);

GetLeafNodes(root.left, ref result);

GetLeafNodes(root.right, ref result);

}

void IsOverlap(node tree1, node tree2)

{

var leaves1 = new List<node>();

var leaves2 = new List<node>();

GetLeafNodes(tree1, ref leaves1);

GetLeafNodes(tree2, ref leaves2);

if (leaves1.Intersect(leaves2)) return true;

else return false;

}

Find a missing card in a standard deck of playing cards where you can remove one card at at time from the deck.
K = number of suits ( say 4 eg 1, 2, 3, 4 for leaf, clover, spade, diamond)
N = number of denominations in each suit (say 13)

card GetMissingCard(List<card> deck, int K, int N)
{
bool found = new bool[K * N]();
for (int i = 0;i < K * N  - 1; i++)
{
found[(deck[i].K-1)*N + deck[i].N-1] = true;
}
var missing = found.toList().First( x => x == false);
return missing;
}

#codingexercise
Given an unsorted array, trim the array such that twice of minimum is greater than maximum in the trimmed array. Elements should be removed either end of the array. Print the trimmed Array
int minRemovals(List<int> nums, int start, int end)
{
if (start >= end) return 0;

var selection = nums.GetRange(start, end-start+1);

if (selection.min() * 2 > selection.max()){

   Console.Write(selection.toString());

  return 0;

}

return min( minRemovals(nums, start+1, end), 

                minRemovals(nums, start, end-1));

}

Find the minimum number of coins that make a given value with an infinite supply of each of the values

int getMinCoins(List<int> coins, int V)

{

var minCounts = new int[V+1];

minCounts[0] = 0;

for (int i = 1; i <= V, i++)

    minCounts[i] = INT_MAX;

for (int i =1; i <= V; i++)

{

   for (int j = 0; j < coins.Count; j++)

   {

     if (coins[j] <= i)

     {

          int prev = minCounts[i-coins[j]];

          if (prev != INT_MAX && prev+1 < minCounts[i])

             minCounts[i] = prev + 1;

     }

   } 

}

return minCounts[V];

}

Duplicity is  a backup tool that can read files and folders from different backends. Here is an example to backup VM instances using a backend that can read the image and disk files on the vcenter.
https://github.com/ravibeta/PythonExamples/tree/master/duplicity 


The same techniques can be applied to Openstack because images and volumes are available just as they are from VMWare.

#codingexercise
Given a sorted array of number , value K and value X, find the K nearest number to the value
Example: Input 12 16 22 30 35 39 42 45 48 50 53 55 56 K = 4 X = 35
Output 22 30 39 42
List<int> GetKNearest(List<int>sorted, int X, int K)
{
var ret = new List<int>();
var index = find(sorted, X, 0, sorted.Count -1);
If ( index == - 1) return null;

int start = index-1;

int end = index +1;

for (int i = 0; i <k; i++)

{

if (start >=0 && end < sorted.Count){

    if (sorted[index]-sorted[start] < sorted[end]-sorted[index])

    {

         ret.Add(sorted[start]);

         start--;

    }else{

         ret.Add(sorted[end]);

         end++;

    }

}else if (start >= 0){

    ret.Add(sorted[start])

    start--;

}else if (end < sorted.Count){

    ret.Add(sorted[end])

    end++;

}

return ret;
}

int find(List<int> sorted, int X, int start, int end)

{
If (end < start) return - 1;

int mid = (start + end)/2

if (sorted[mid] == X) return mid;

if (sorted[mid] < X)

   return find(sorted, X, mid+1, end);

else

   return find(sorted, X, start, mid-1);

}

#coding exercise
Given array of N integers ranging from 0 to N-1. Output maximum repeating integer. Use only O(1) memory
int maxRepeating(List<int> arr, int n, int k)
{
for (int i =0; i < n; i++)
{
arr[arr[i]%k] += k;
}
int max = arr[0];
int result = 0;
for (int i =1; i < n; i++)
{
   if (arr[i] > max)

   {

     max = arr[i];

     result = i;

   }

}
for (int i =0; i <n; i++)
      arr[i] = arr[i] %k;
return result;

}

find maximum element in an array which is first increasing and then decreasing:
int getMax(List<int> arr, int start, int end)
{
if (start == end)
   return arr[start];
if ((end == start + 1) && arr[start] >= arr[end])
   return arr[start];
if ((end == start + 1) && arr[start]  < arr [end])
   return arr[end];
int mid = (start+end)/2;
if (arr[mid] > arr[mid+1] && arr[mid] > arr[mid-1])
   return arr[mid];
if (arr[mid] > arr[mid+1] && arr[mid] < arr[mid-1])
   return getMax(arr, start, mid-1);
else
   return getMax(arr, mid+1, end);
}

Another tree question
void printLeftView(node root, int level, ref int maxlevel)
{
if (root == null) return;
if (maxlevel < level){
   console.write(root.data);
   return level;
}
printLeftView(root.left, level+1, ref maxlevel);
printLeftView(root.right, level+1, ref maxlevel);
}

Convert a binary search tree into in order doubly linked list
Void bst2dllIn (node root, ref node head)
{
If (root==null) { return}
Node prev =null;
Bst2dllIn (root.left, ref head);
If (prev == null)
     Head = root;
      Prev = root;
Else{
Root.prev = prev;
Prev.next = root;
Prev = root;
}
Bs2dllIn (root.right, ref head);
}

Convert a binary search tree into pre order doubly linked list
Void bst2dllPre (node root, ref node head)
{
If (root==null) { return}
Node prev =null;
If (prev == null)
     Head = root;
      Prev = root;
Else{
Root.prev = prev;
Prev.next = root;
Prev = root;
}
Bst2dllPre(root.left, ref head);
Bs2dllPre(root.right, ref head);
}

Convert a binary search tree into postorder doubly linked list
Void bst2dllPost (node root, ref node head)
{
If (root==null) { return}
Node prev =null;
Bst2dllPost (root.left, ref head);
Bs2dllPost (root.right, ref head);
If (prev == null)
     Head = root;
      Prev = root;
Else{
Root.prev = prev;
Prev.next = root;
Prev = root;
}
}