Workflow for setting up amazon metrics to be collected: 

1) Edit Billing preferences as shown: 

 

Notice that the AWS billing preferences provide three options:  

The first option is the one that we will be customizing for our customers hence this option is merely for validity check. 

The second option is the one that we are interested in because we can subscribe to CloudWatch then.  

The third option can also be used with automation and hence we set it up with a S3 bucket.  

Notice that the policy to be applied for a bucket named raja0034 S3 billing is as follows:  

{ 

  "Version": "2008-10-17", 

  "Id": "Policy1335892530063", 

  "Statement": [ 

    { 

      "Sid": "Stmt1335892150622", 

      "Effect": "Allow", 

      "Principal": { 

        "AWS": "arn:aws:iam::386209384616:root" 

      }, 

      "Action": [ 

        "s3:GetBucketAcl", 

        "s3:GetBucketPolicy" 

      ], 

      "Resource": "arn:aws:s3:::raja0034billing" 

    }, 

    { 

      "Sid": "Stmt1335892526596", 

      "Effect": "Allow", 

      "Principal": { 

        "AWS": "arn:aws:iam::386209384616:root" 

      }, 

      "Action": [ 

        "s3:PutObject" 

      ], 

      "Resource": "arn:aws:s3:::raja0034billing/*" 

    } 

  ] 

} 

For option 2) of selecting alarms and metrics, we have to set it up on CloudWatch which displays both ECS and EBS Metrics: 

 

Selected Metrics could then include: 

 

Note that the metrics are categorized by service. If you wanted EC2 and EBS specifically, you would change the region to the one where your instances are : 

 

Notice we have changed from East region to West. 

Then you would see the metrics that we want to grab statistics for which we will do periodically and save it in a database as time series data for historical information and query. 
#codingexercise
Void sizeOf1or0onlyRectangle (int [,] matrix, int row, int col, int startx, int starty, int x, int y, ref int size, dir)
{
If (x+1 < row && y +1 < col  && 
Matrix [x+1,y+1] = matrix [x, y] &&
Matrix [x, y+1] = matrix [x,y] &&

Matrix[x+1, y] = matrix [x, y] && dir == diagonal && all_elements_in_new_row_are_same(matrix, startx, starty, matrix[x,y]) && 

all_elements_in_new_col_are_same(matrix, startx, starty, matrix[x,y]))

sizeOf1or0onlyRectangle(matrix,row, col, x+1,y+1, ref size +(x+1)-startx + (y+1)-starty + 1, dir );

If ( y +1 < col  && 

Matrix [x,y+1] = matrix [x, y] && dir == horizontal && all_elements_in_new_row_are_same(matrix, startx, starty, matrix[x,y]))

 sizeOf1or0onlyRectangle(matrix,row, col, x,y+1, ref size +(y+1)-starty, dir );

If (x+1 < row && 

Matrix[x+1, y] = matrix [x, y] && dir==vertical && all_elements_in_new_col_are_same(matrix, startx, starty, matrix[x,y]))

 sizeOf1or0onlyRectangle(matrix,row, col, x+1,y, ref size +(x+1)-startx, dir);

}

Today's post talks about AWS statistics gathering:
Historical data of current value of metrics can be very useful. But AWS also provides the aggregators, sum, max, min and avg.
If the metrics have continuous values and their sum is important, we can maintain a running total by taking the sum every interval. Typically this interval can be set in the call to grab the statistics.
Since the calls are continuous, their return values are also non-overlapping and therefore summation is straightforward.

function addCurrentToCumulative($current, $cumulative){
foreach ( $current as $metric ){

if (array_key_exists($metric['key'], $cumulative) == false){
$cumulative[$metric['key']] = array('key'=>$metric[key], 'value' => 0, 'units' => $metric['count']);
}

if (array_key_exists('cumulate', $metric) && $metric['cumulate']){

if (array_key_exists($metric['key'], $cumulative)){
$cumulative[$metric['key']]['value'] += $metric['sum'];
}else{
$cumulative[$metric['key']]['value'] = $metric['sum'];
}

}else{
$cumulative[$metric['key']]['value'] = $metric['avg'];
}
}
return $cumulative;
}
function saveOrPrintCumulative($cumulative){
print "\n==========================================================\n";
print " Aggregated Metrics\n";
print "==========================================================\n";

foreach($cumulative as $item){
$key = $item['key'];
$value = $item['value'];
$units = $item['units'];
print "Metric -- $key $value $units \n";
}

}

// after every $interval minutes
// cumulate by adding current to total
// update current with grab_stats

$tablename = "Metrics";
$current = array();
$cumulative = addCurrentToCumulative($current, $cumulative);
$current = grab_stats($client, $tablename);
sleep($interval*60); // seconds
$cumulative = addCurrentToCumulative($current, $cumulative);
$current = grab_stats($client, $tablename);
?>

Note the forward moving cumulation ensures that there are no overlaps or errors introduced.

<?php 

require 'vendor/autoload.php'; 

---

9 / 30

use Aws\CloudWatch\CloudWatchClient; 

$key = "Your_key"; 

$secret = "Your_secret"; 

$region = "us-west-2";
$version ="latest";

// Use the us-west-2 region and latest version of each client. 

$sharedConfig = [ 

   'region'  => $region, 

   'version' => $version, 

 'credentials' => array( 

   'key' => $key, 

   'secret'  => $secret, 

 ), 

   'key'    => $key, 

   'secret' => $secret, 

]; 

// Create an SDK class used to share configuration across clients. 

$sdk = new Aws\Sdk($sharedConfig); 

$client = $sdk->createCloudWatch(); 

function grabber($client, $tablename, $metric) { 

 $output = array(); 

 $results = $client->getMetricStatistics(array( 

   'Namespace'  => 'AWS/ECS', 

   'MetricName' => $metric, 

   'Dimensions' => array( 

     array( 

---

10 / 30

       'Name' => 'TableName', 

       'Value' => $tablename, 

     ), 

   ), 

   'StartTime'  => strtotime('-1 days'), //'-'.$interval.' minutes'), 

   'EndTime'    => strtotime('now'), 

   'Period'     => 300, 

   'Statistics' => array('Minimum', 'Maximum', 'Average', 'Sum'), 

 )); 

 echo 'RESULTS='.serialize($results); 

print "-------------------------------------------\n"; 

 print "    $metric\n"; 

 print "-------------------------------------------\n"; 

 foreach ($results as $result){ 

 echo 'RESULT='.serialize($result); 

 if (is_array($result) && array_key_exists('Datapoints', $result)){ 

 foreach ( $result['Datapoints'] as $item ) { 

   $min = $item['Minimum']; 

   $max = $item['Maximum']; 

   $avg = $item['Average']; 

   $sum = $item['Sum']; 

   $time = $item['Timestamp']; 

   print "$time -- min $min, max $max, avg $avg, sum $sum\n"; 

   array_push($output, array('key'=>$time, 'min'=>$min, 'max'=>$max, 'avg'=>$avg,'sum'=>$sum, 'cumulate'=>true, 'units'=>'count')); 

 } 

 } 

 } 

 return $output; 

} 

To detect whether a rectangle of contiguous 1 or zero exists and if so, it's size given the starting position of top left corner as the element in a 2d array of 1 and 0, we use the following:

starting at this point as top left, we walk the rectangle based on increasing diagonal or side bottom or right and making sure all newly encountered elements within the new bounds are of similar value.