Wednesday, October 8, 2014

#coding exercise
print the area of the largest rectangle in a histogram of unit width heights.
public static int MaxArea(ref List<int> h)
{
            if (h == null || h.Length <= 0) return 0;
            var areas = new List<int>();
            for (int i = 0; i < h.Length; i++)
            {
                areas.add(i, h[i]);
                for (int k = i+1; k < h.Length; k++)
                   {
                     if (h[k] >= h[i])
                         areas[i] += h[i];
                     else
                     {                      
                         break;
                     }
                   }            
            }
            return areas.Max();
}
public class hc
{
// height
Public int h { get; set;}
// frequency
Public int c { get; set;}
}
Public static int MaxAreaByStack (List <int> h)
{
            if (h == null || h.Length <= 0) return 0;
            Int MaxArea =0;
            var hts = new stack<int>();
            Var f = new List<int>();
             Hts.push (h [0]);
             F.Add (1);
            for (int i = 1; i < h.Length; i++)
             {
                       If (h [i] > h [i-1])
                       {

                           For (int k=0; k< hts.count; k++)
                           {
                                F [k] +=1;
                           }
                           Hts.Push (h [i]);

                            F.Add (1);
                       }

                      Else

                     {

                      For ( int k = hts.count -1; k >=0;k--)

                     {

                      If (h [k] <= h [i]) break;

                      Var last = hts.pop ();

                      Var count = f.Last ();

                      F.RemoveLast ();

                      If (last × count > maxArea)
                           MaxArea = last×count;
                     }
                  }
            }
Return maxArea;
}

Tuesday, October 7, 2014

A review of configuration requirements tips and tricks by Carlos Galeona for scale out NAS.
Cluster with OneFS ranging in size of filesystem from 18TB to 15.5 PB that's easy to manage with no RAIDs, LUNs, etc. and easy to grow is the target of the configuration. Minimum system configuration is a cluster with three nodes, two infiniband switches, the same version of OneFS, licenses for different modules. Common tasks involve upgrade, deployment, verification etc. cfengine deployment for host, and cfagent runs from cron for each node is the practice. Management API exists for OneFS 7.1.1  A traditional backup is an example use case. An NFS file system is mounted to a backup server or client and then a native backup is done. A large scale file system could mean multiple backups and parallel tree traversals.  The latter can take a long time. 7.1.1 API allows the creation of changelist that addresses latter.

Sample program to create a web API based administration session, create an NFS export and create a snapshot.

# import relevant libraries
import json
import urllib
#  for data = urllib.urlopen(url, params).read()
import httplib2
http = httplib2.Http()
# for http.request(url) instead of authenticated urlopen

# setup
api_user = 'Your api username'
api_pass = 'Your api password'

# create session
url = 'https://<cluster-ip-or-host-name>:8080/session/1/session'
params = urllib.urlencode({
'username': api_user,
'password': api_pass,
'services': ['NFS','Snapshot']
})
http.add_credentials(api_user, api_pass)
response, content = http.request(url, 'POST', params,
headers={'Content-type': 'application/x-www-form-urlencoded'}
)

# validate
url = url + '?isisessid'
http.add_credentials(api_user, api_pass)response,content = http.request(url)
data = json.loads(content.json())
if data["username"] != api_user:
   raise Error('bad session')

#create nfs export
url = 'https://<cluster-ip-or-host-name>:8080/platform/1/protocols/nfs/exports'
params = urllib.urlencode({
'description': 'sample mount',
'paths': ['/path','/path1'] # under /ifs
})
http.add_credentials(api_user, api_pass)
response, content = http.request(url, 'POST', params,
headers={'Content-type': 'application/x-www-form-urlencoded'}
)
data = json.loads(content.json())
if not data["id"] :
   raise Error('bad export')

# validate
id = data['id']
url = 'https://<cluster-ip-or-host-name>:8080/platform/1/protocols/nfs/exports/' + id + '?describe'
http.add_credentials(api_user, api_pass)
response,content = http.request(url)
data = json.loads(content.json())
if not data["id"] :
   raise Error('bad export')

#create a snapshot
url = 'https://<cluster-ip-or-host-name>:8080/platform/1/snapshot/snapshots'
params = urllib.urlencode({
'name': 'sample snapshot',
'path': '/ifs'
})
http.add_credentials(api_user, api_pass)
response, content = http.request(url, 'POST', params,
headers={'Content-type': 'application/x-www-form-urlencoded'}
)
data = json.loads(content.json())
if not data["id"] :
   raise Error('bad snapshot')

# validate
id = data['id']
url = 'https://<cluster-ip-or-host-name>:8080/platform/1/snapshot/snapshots/' + id + '?describe'
http.add_credentials(api_user, api_pass)
response,content = http.request(url)
data = json.loads(content.json())
if not data["created"] :
   raise Error('bad snapshot')

from datetime import datetime, timedelta
createddate = datetime.strptime(data['created'])
if  datetime.now  - createddate >  timedelta(hours=2):
    raise Error('old snapshot')

#coding exercise
Given a non-negative number represented as an array of digits, plus one to the number.

void plusOne(ref List<int> digits, int position)

{
if (position  >= digits.Length) return;
If (position < 0) return;
if (digits[position] + 1 > 9)
{
   digits [position] = 0;
if (position - 1 < 0) {// check INT_MIN underflow; digits.InsertAt(0, 1);}
Else
   plusOne(ref digits, position - 1);
}
else
    digits[position] += 1;
}



# coding exercise
Validate BST

Bool  isValidBST ( node root)
{
If (root == null) return true;
If (root.left && root.left.data >= root.data ) return false;
If (root.right && root.right.data < root.data ) return false;
Return isValidBST(root.left) && isValidBST(root.right);
}

Given a singly linked list L0->L1->L2->  Ln-1-> Ln
Return interleaved L0->Ln->L1->Ln-1...
List <Node> Interleave ( List <Node> items)
{

If ( items == null || items.length <= 2) return null;
Int n = items.length;
int mid = (n%2==0) ? n/2 : n/2+1;
Var  rem = root.GetRange (mid);
Int I = 1;
Int count = rem.length;
For (int k=0; k < count;k++)
{
  Node last = rem.last ();
  Rem.removeLast ();
  Root.insertat (i, last);
  I = I +2;
}
}
Node* interleave ( node* root, int  n)
{
If ( root == null || n <= 2) return;
int mid = (n%2==0) ? n/2 : n/2+1;
Node* start = root;
Node* last = start->next;

while(start && last)
{
Node* prev = start;
while(last && last->next)
{
  prev = last;
  last = last->next;
}
prev->next = null;
Node* next = start->next;
last->next = next;
start->next = last;
start = next != null ? next->next : null;
last = (start != null) ? start->next : null;
}
}

Monday, October 6, 2014



#coding exercise
Given a binary tree, check if its symmetric around the center
bool isSymmetric( Node root)
{
 if (root == null) return false;
List<List<Node>> items=  GetZigZag(root); // level wise items (already implemented earlier)
for (int i = 0; i < items.Count; i++)
   if (IsPalindrome(items[i]) == false)
        return false;
return true;
}

bool isPalindrome(List<Node> items)
{
  if (items == null) return false;
  int start = 0;
  int end = items.Count - 1;
  while ( start < end)
  {
     if (items[start] != items[end])
         return false;
    start++;
    end--;
   }
   return true;
}

Given a pair  of rectangles aligned along the axis in the positive quadrant and given by
class Rectangle
{
  Point tl; // TopLeft;
  Point br; // BottomRight;
}
class Point
{
 int x;
 int y;
}
Implement the following  methods
static bool IsIntersect( Rectangle r1, Rectangle r2);
static Rectangle Intersecting(Rectangle r1, Rectangle r2);

static bool IsIntersect( Rectangle r1, Rectangle r2)
{
bool isdisjoint = false;
// are  they disjoint along x axis ?
if (r1.tl.x < r2.tl.x)
   isdisjoint = r1.br.x < r2.tl.x;
else
  isdisjoint = r2.br.x < r1.tl.x;
if (isdisjoint == true) return false;

// are they disjoint along y-axis ?
if (r1.br.y < r2.br.y)
   isdisjoint = r1.tl.y < r2.br.y;
else
  isdisjoint = r2.tl.y < r1.br.y;
return isdisjoint == false;
}

static Rectangle Intersecting(Rectangle r1, Rectangle r2)
{
  if (!IsIntersect(r1, r2)) return null;
  Rectangle r = new Rectangle();
  Rectangle left = (r1.tl.x <= r2.tl.x) ? r1 : r2;
  Rectangle right = (r1.tl.x <= r2.tl.x) ? r2 : r1;
  Rectangle bottom  = (r1.tl.y <= r2.tl.y) ? r1 : r2;
  Rectangle top = (r1.tl.y <= r2.tl.y) ? r1 : r2;
  r.tl.x = right.tl.x;
  r.br.x = right.br.x <= left.br.x ? right.br.x : left.br.x;
  r.tl.y = bottom.tl.y;
  r.br.y  = bottom.br.y <= top.br.y ? top.br.y : bottom.br.y;
return r;
}
 

Sunday, October 5, 2014


Ceilometer & Splunk Integration via Python SDK:

Ceilometer is OpenStack’s telemetry. It’s source code is available at https://github.com/openstack/ceilometer. It collects all kinds of measurements and metering information such that no two agents need to be written to collect the same data. It’s primary consumer is the billing system for which it acts as a unique and central point of contact  to acquire all of the information collected across all OpenStack core components. The project strived to be efficient in terms of CPU and network costs. It supports both the push notifications from the existing services and the pull model by polling the infrastructure. Deployers can configure the type of data collected. Publishers are available as a Python implementation. A REST API is also available to expose the data collected by the metering system.

As an example of the events from Ceilometer, we review the PAAS event format. There are a number of PAAS services that have metering payloads. Instead of having each service define its own, the Ceilometer provides a minimum data set as described here.

Splunk is an IT tool that enable searching a needle in a haystack of information. Specifically, it forwards information collected from various sources (via its modular inputs) and indexes them in time series events which can then be searched and analyzed using a wide variety of search operators and charting tools. Splunk also comes with its own SDK and one that is available in Python.
Splunk SDK supports a variety of common operations with Splunk through its programmable APIs. In particular it supports a UDP modular input that can suit Ceilometer.

The host and the port binds the publisher to the subscriber. In this case, we can configure the host and UDP port for ceilometer to publish to Splunk.

#!/usr/bin/env python
#
# Copyright 2013 Ravi Rajamani.
import sys
from splunklib.modularinput import *
class CeilometerIntegrationScript(Script):
    def get_scheme(self):
        scheme = Scheme("Ceilometer Telemetry")
        scheme.description = "Streams events from Ceilometer."
        scheme.use_external_validation = True
        scheme.use_single_instance = True
        scheme.validation = ""
        ceilometer_argument = Argument("data_from_ceilometer")
        ceilometer_argument.data_type = Argument.data_type_string
        ceilometer_argument.description = "Telemetry data from Ceilometer to be produced by this input."
        ceilometer_argument.required_on_create = True
        return scheme
    def validate_input(self, validation_definition):
        data = str(validation_definition.parameters["data_from_ceilometer"])
        if not data:
            raise ValueError("Ceilometer data could not be read.")
    def stream_events(self, inputs, ew):
        """This function handles all the action: splunk calls this modular input
        without arguments, streams XML describing the inputs to stdin, and waits
        :param inputs: an InputDefinition object
        :param ew: an EventWriter object
        """
        for input_name, input_item in inputs.inputs.iteritems():
            # Create an Event object, and set its data fields
            event = Event()
            event.stanza = input_name
            event.data = "number=\"%s\"" % str(input_item["data_from_ceilometer"])
            # Tell the EventWriter to write this event
            ew.write_event(event)
if __name__ == "__main__":
    sys.exit(CeilometerIntegrationScript().run(sys.argv))
#coding exercise
generate Pascal's triangle:
        public static List<List<int>> generatePascalTriangle(int n)
        {
            var ret = new List<List<int>>();
            for (int i = 0; i < n; i++)
            {
                if (i == 0) { ret.Add(new List<int>() { 1 }); continue; }
                ret.Add(new List<int>());
                ret[i].Add(1);
                for (int j = 0; j < ret[i - 1].Count - 1; j++)
                    ret[i].Add(ret[i - 1][j] + ret[i - 1][j + 1]);
                ret[i].Add(1);
            }
            return ret;
        }
       static void Main(string[] args)
      {
            var ret = generatePascalTriangle(11);
            foreach (var r in ret)
            {
                Console.WriteLine();
                for (int i = 0; i < r.Count; i++)
                {
                    Console.Write("{0} ", r[i]);
                }
                Console.WriteLine();
            }
      }

1

1 1

1 2 1

1 3 3 1

1 4 6 4 1

1 5 10 10 5 1

1 6 15 20 15 6 1

1 7 21 35 35 21 7 1

1 8 28 56 70 56 28 8 1

1 9 36 84 126 126 84 36 9 1

1 10 45 120 210 252 210 120 45 10 1

Friday, October 3, 2014

I will cover pricing models in this post.  Specifically we will see how to cluster and classify numerical data.  This kind of data differs from the examples we have seen earlier in that there are many different attributes. One way to overcome this limitation is to use KNN algorithm.  KNN stands for K-nearest neighbors.   To make numerical predictions we need to know which variables are important. Let us take an example of a numerical prediction with one variable.  The book we review gives the example of pricing a wine bottle based on age and initialized by rating. In particular, it says if the age of the bottle is past its peak then it is almost immediately cheap and if it hasn’t attained its peak age yet then its price shoots up.  It does this by adjusting the price with a difference which is calculated as a constant times the delta between the actual age and peak age in the case when age has exceeded the peak. On the other hand, it increases the price to the order of five fold.  The choice to use these weights is interesting as we will see shortly. 

Given that we can come up with a random sample.  To price the items, we will be using the neighbors of the item that are most similar to it so that we can average and get it closer to the others. There are two issues here. One is if we choose a small value for n, then the predicted price will be too dependent on variations. At the same time if we choose a large value for n, then the predicted price will not be accurate. The second issue is that we need to define how similar two items are.  In the latter case, we use the Euclidean distance which is the square root of the sum of squares of the differences between age and rating.  It’s interesting to note that this similarity function treats the age and rating the same. This is a well known weakness of KNN and one that is overcome with  weighted neighbors, inverse function and Gaussian smoothing.

#coding Exercise
Given an array of non-negative integers, each element represents the maximum jump length at that position. Starting from the first of an array can we reach the last?

bool canJump(int[] A)
{
  if (A == null) return false;
 int index = 0;
 while (index < A.Length)
 {
   if (index + A[i] == A.length - 1) return true;
   if (index +A[i] <= index) return false;
   index = index + A[i];
  }
 return false;
}

#coding exercise
Two linked lists are given, representing two non-negative numbers. The digits are stored in reverse order and each node contains a single digit. Add the two numbers and return it as a linked list.
Input: (2 -> 4 -> 3) + (5 -> 6-> 4)
Output: (7->0->8)

List<int> AddTwoNumbers(List<int> l1, List<int> l2)
{
// parameter validation and return
int num1;
int num2;
int factor = 1;
for (int i =0; i< l1.Count; i++)
{
checked { num1 += l1[i] * factor;}
checked { factor = factor * 10;}
}
factor = 1;
for (int i =0; i< l2.Count; i++)
{
checked { num2 += l2[i] * factor;}
checked { factor = factor * 10; }
}
List<int> result = new List<int>();
int num3;
checked { num3 = num1 + num2; }
int count = 0;
int temp = num3;
while(temp != 0)
{
result.Add(temp % 10);
temp = temp / 10;
count++;
}
return result;
}
I'm not feeling well since yesterday night.