In the world of cloud computing and cloud based object storage, how to use buckets ? Are they the same as folders ? How does an object storage differ from a file system ? 

Buckets cannot be simplified to folders although objects can be organized with folders. A bucket cannot have another bucket.  A folder can nest another folder.  A folder is a file system artifact.  A bucket is not. A bucket can have many different objects within it. A bucket is an object storage artifact. Let us see what it means. 

If we take the example of a folder called logs that has say log1.txt, log2.txt and log3.txt, then they exist in the bucket as three different objects with a prefix “logs/” and key names “logs/log1.txt”, “logs/log2.txt”, “logs/log3.txt”. The folder we see in our console is merely for logical organizational convenience within the bucket. It doesn’t have a data structure per se unlike the folder in a file system.  Think prefixes when working with an object store directly instead of folders. 

  

Objects can be moved from one folder to another but not from one bucket to another. If we are creating too many buckets, we are introducing artificial restrictions beforehand. It’s the same as partitioning a disk multiple times when a single undivided partition would do. Moreover, it would eliminate the need to copy or migrate data from one to others. A single seamless bucket is also better positioned to meet future needs than a dozen of them when all else remains the same. There is no loss of organization when there is one bucket or many. 

  

Buckets are wide and deep. They sprawl over several thousand storage devices and span datacenters. In the world of Openstack, they power file systems. For the example when users are using Ceph, they may have no idea that the data they are putting in a filesystem is actually getting stored in the cloud with RADOS object store.  The filesystem meets the users convenience and the habits. We organize based on files and folders and that habit is difficult to change. Moreover that habit serves us well across paradigms such as this. When using an object store directly, we should be mindful of the rich metadata associated with objects and metadata based access of objects. 

  

As with metadata, permissions are also fine grained and available at the object level as well as the folder level. Folders can be made public or private. In a public folder, all the objects that appear within a public folder are available for viewing or downloading to anyone on the internet. If you want to browse a folder that is made public, you will get access denied because the folder is just a naming prefix for an object or group of objects. It doesn’t exist.  A folder can be made public but it cannot be reverted to private. Instead you can individually mark the objects private within a public folder. Buckets have access permissions and versioning status and you can specify which region a bucket should reside in. Logging, versioning and event subscriptions can be turned on for a bucket.  Cost allocation tags (key value pairs) can be assigned at the bucket level. You could create many buckets but you should exhaust the possibilities with something more granular than a bucket. For example, you could use naming conventions for objects.  

Many different objects can exist within a bucket. They can be queried with autocomplete like jump feature of bucket browsing instead of paging through millions of items. Moreover, metadata attributes are available that can be used to label the objects in different ways.  

  

An object store was designed with one of the founding principles to reduce the storage admin’s task of provisioning and maintaining storage.  Anytime we are increasing those tasks, we are not doing something right.  

Some more resources on object storage include the following: 

 A Beginner's Guide To Next Generation Object Storage - DDN 

http://blogs.aws.amazon.com/security/post/Tx1P2T3LFXXCNB5/-Writing-IAM-span-class-matches-policies-span-Grant-access-to-user-specific-fold 

The caveat is that even AWS documentation doesn't explain the usage and goes so far as suggesting that buckets be created homogeneous when in fact they are designed to hold heterogeneous content. Their blogs on the other hand are far more revealing 

  

#codingexercise
Node GetMin(Node root)
{
Node cur = root;
while (cur && cur.left)
    cur = cur.left;
return cur;
} 

RADOS and ViPR

In continuation of the document I wrote about Object store here, I want to bring up two emerging technologies and their role. RADOs is an object store which also allows BlockIO(iSCSI) gateway and NAS gateways. Together with Ceph for Openstack, it eliminates the storage silos that usually form with different protocols. In addition, it can support S3 and Swift APIs. This unified approach also helps with maintainability. Essentially the nature of any object store is to export a server’s local disk space into a single large datastore. Software that maintains this functionality saves data in binary form. As in a cluster computing, the number of nodes participating in this technology is arbitrary and hence the perception of seamlessly scalable storage. This can support billions of objects that are identified by their ID. 

ViPR addresses a different point of unification. It brings the legacy gear typical of a datacenter into a single unified storage. It provides more management convenience that VMWare is known for. Behind its façade, it connects different legacy servers that maintains their protocol centric behavior. By protocols, I mean ftp, http, cifs, nfs, etc. Centralized automated management reduces storage provisioning tasks against each of these protocols. 

How does RADOS work? 

RADOS stands for reliable autonomic distributed object store. It consists of three different layers: 

1. Object Storage Device: An OSD in RADOS is always a folder within an existing filesystem. Together the OSD form the object store. RADOS generates binary objects from the files to be stored and stashes it in the store 

2. Monitoring Servers (MON) These form the interface to the RADOS store and support access to the objects within the store.  

3. Metadata servers (MDS) MDS provides POSIX metadata for objects in the RADOS object store for CEPH clients. 

Ceph is the file-system that accesses the object store in the background. Ceph is closer to the user while RADOS powers it. Without Ceph the data is not accessible to the user. However RADOS is not limited to being the backend of a filesystem. It can work with others. 

ViPR can work with a variety of systems and a variety of services. It’s a one stop management utility for software as a service. While it is widely recognized that each storage platform can handle specific workloads, each comes with its own specific APIs, management and monitoring tools. ViPR eliminates this overhead and provides a common automatic provisioning and management portal. 

ViPR enables automatic storage management for use by VMWare, Openstack and Microsoft clients. It provides access via a unified OPEN REST APIs across a dozen types of virtual storage arrays. Moreover, it’s multivendor support is 11extensible.  

Both ViPR and RADOS seem to provide seamless unlimited storage. While RADOS provides new form of storage, ViPR can transform the existing storage infrastructure into a simple and extensible platform. 
Bool hasNode ( node root, node target) {
If root  == null return false;
If target == null return false;
If root == target  return true;
Return hasNode (root.left, target) || hasNode (root.right, target);

}

Today we complete the review of the book BRIEF by Joseph McCormack.
We were discussing the steps :
Map it
Tell it
Talk it
Show it.
He says as we get older, we tend to skip the outline for our message.This is a big mistake especially when the volume of information is huge to handle, distill or disseminate. Outlining keeps you prepared, organized, clear, contextual and confident. Visual presentations or outlining is increasingly in vogue today thanks to software such as  Mindjet.  Software like Mindjet and tools like Bubbl.us are appealing because they are visual logical and intuitive to use and share. This practice is called mind mapping.
A specific type of mind map is the BRIEF map which draws the focal point or headline for your communication:
B: Background or beginning.
R: reason or relevance
I:  key information
E: Intended ending
F: Expected follow up or questions
Brevity is all about preparation and pre-assembly. A BRIEF map is a visual tool that prepares you to be succinct.
The next step is to tell the message by using a story.  Even though we may not be a journalist, we should think like one in telling a story.  Key considerations of a great story include a strong headline, compelling lead paragraph, a clear sense of conflict, a personal voice, a consistent narrative thread, a logical sequence of events, character development and powerful communication. Storytelling is a way for the making corporate speaking more engaging.
The next step is to talk it.To talk it means you can have controlled conversations where two people are willingly talking and enjoying but being limited by time. A controlled conversation is a discipline and active listening prepares you with what matters most to the person you are talking to.
A TALC track is just that. - It is talk, active listening and converse. When somebody is talking, to let the person talk is very important.  To closely listen what the other person is saying with interest the entire time is equally important. And only converse or bridge to a different topic when a natural pause comes.
The last step is to show it. Its a powerful way to make a picture exceed a thousand words. Phonebooks, newspapers, printed menus and billboards are increasingly replaced by visuals. The author says that we remember only 10% of what we hear, 30% of what we read, and 80% of what we see. Using an image or a video with your story, keep it short, be mindful of time and quality.
Finally, we should gain the decisiveness to know when and where to be brief.
For example, Trim your meetings early when you know its done. Also, we can rethink the meeting and find new ways of engaging attendees. This could even be a standup meeting. And remove the tyrant voice - one that is dominating to the detriment of others.
Those who are loquacious on their Twitter or Facebook are training the world to tune them out. Social media is where the brevity rules. People are already finding different tools to present information. But the effort required to make quality posts cannot be ignored.
TED talks are an amazing forum to practice this. Their motto is "ideas worth sharing" and its the sharing part that gets tricky and they help you practice it. Everyone is in the business of buying or selling ideas so we must make our pitch appropriately.
Another example of brevity in action is good interviews. An interview candidate often makes the mistake of talking too much. Good interviews are just the right amount.
The same applies  to sharing good news. It doesn't need to be piled on as is the case in some corporate communications but it can be allowed to sink in. Likewise, give the bad news straight. Since they happen all the time, dispense with it and channelize your energy back into rebuilding.
Lastly by being brief, you discuss your own essence. 

Today we continue to review the book Brief by Joseph McCormack. The author calls out that people are already drowning in information around them, so much so that their only concern is how to survive. The ubiquity of access to information such as from mobile devices has inundated us with information even before the day has begun. Therefore our attention is already taxed. Although we may get verbal and non-verbal agreements, the message may not have sunk in. We have to understand the four forces that are constantly playing out against us.
These are :
Information Inundation : an ending flood of word, images, sounds and social media.
Inattention : ability to focus more than 10 seconds.
Interruptions: a steady stream of problems competing for our time
Impatience: a growing intolerance for results
You have to make it easier to consume and digest  the idea you are presenting. Lean communication is a new advantage.
Likewise the author continues to guard against the pitfalls of brevity.
1. Cowardice - you hide behind meaningless words instead of taking a clear stand
2. Confidence - you know it all and you are verbose to everyones boredom
3. Callousness - you are selfish and you don't respect other people's time
4. Comfort - you think those you are familiar with have time to spare
5. Confusion - you chose to think out loud when you are not clear yourself.
6. Complication - you feel some things are just too complex and give up
7. Carelessness - you are verbally sloppy and the people get confused even more
and the mental muscle for brevity is acquired with a plan of attack as follows:
Map it - BRIEF maps are used to condense and present succinct information
Tell it - A story summarizes and evokes much better response because it is compelling
Talk it - The TALC tricks (Talk, active, listening, converse) makes it interactive
Show it - Visuals and diagrams attract attention and capture imagination.

Today we start reviewing the summary of another book which is fun to read. It is titled Brief by Joseph McCormack.
 But first a coding exercise
# codingexercise
Find the lowest common ancestor of two nodes in a normal tree ( not a binary tree ) where the nodes may not have links to their parents. Space need not be optimized but time has to be optimized.
There are couple of approaches as suggested by the  question
1) we find the least common ancestor as one whose children don't have both the nodes but it has both the nodes in its subtrees ( The HasNode(root, node) has to invoke multiple traversals and visits a node multiple times so this will not meet the second criteria)
2) We maintain two lists for the path to the nodes as we discover them. Then we look for the last node that is common to both the lists.
We implement the second.

Node GetLowestCommonAncestor(Node root, Node one, Node another)
{
If (root == null || one == null || another == null) return null;
var pathToOne = new List<Node>();
var pathToTwo = new List<Node>();
GetPath(root, one, ref pathToOne);
GetPath(root, another, ref pathToTwo);
return LowestCommon(pathToOne, pathToTwo);
}
 bool GetPath(Node root, Node target, ref List<Node> path )
{
if (root == null || target == null || path == null) return;
bool found = false;
path.add(root);
if (root == target) return true;
for (int i = 0; i < root.children.length; i++)
{
   found = GetNodePath(root.children[i], target, ref path)
   if (found) break;
}
if (!found)
    path.RemoveLast(root);
return found;
}
Node LowestCommon(List<Node> p1, List<Node>p2)
{
if (p1.empty() || p2.empty()) return null
int I = 0;
Node ancestor = null;
while (i < p1.length && i < p2.length)
{
if ( p1[i] == p2[i]) { ancestor = p1[i]; }
i++;
}
return ancestor;
}
Returning to the book summary:
This book talks about lean communication arguing that its the pressing need for time that drives to this new paradigm. And how do we go about doing so ? This book provides a step by step approach to getting to the point quickly and more effectively than ever. His proven BRIEF approach is  Background, Reverence, Information, Ending and Followup. The acronym doesn't merely simplifies and clarifies communication but also air tight the space in which decisions are made so that sloppiness does not creep in.
The book recognizes that in some circles a tendency to think brevity is pushing for less and runs the risk of being superficial and lacking substance. However brevity starts with deep expertise. Only with thorough knowledge can you accurately make a summary. Being brief can demonstrate how you have gone through that learning experience.
The steps in this book are described next.

#codingquestion
Given a set of braces or parenthesis in any order with openers defined as [ { or ( and closers defined a s ) } ] , write a function that determines if the brackets are in the correct order.

public book Validator(string input)
{
var brackets = new Stack();
for (int i = 0; i < input.length; i++)
{
 if (input[i] == '[' || input[i] == '{' || input[i] == '(')
{
  brackets.push(input[i]);
}
 if (input[i] == ']' || input[i] == '}' || input[i] == ')')
{
if (brackets.empty()) return false;
var last = brackets.peek();
 if (input[i] == ']' && last != '[') return false;
 if (input[i] == '}' && last != '{') return false;
 if (input[i] == ')' && last != '(') return false;
brackets.pop();
}
}
if (brackets.empty() == false) return false;
return true;
}

Today we take a moment to discuss topic identification and noise clusters. When the topics are clustered by their distance to the centroids while simultaneously weighting the keywords as explained by Frigui, Nasraoui et al., there is a formation of noise clusters that tend to group all the outliers leaving more cohesive clusters behind. Since it attracts all the outliers, only the documents that are similar to the other clusters will remain in their own clusters.

Today we conclude the review of the book Thirteeners by Daniel E Prosser. We discussed how to benefit from chaos by embracing the creativity and innovation that comes up against this feel of disequilibrium. With Chaos comes transformation.  The fastest way through transformation is to adopt the three concepts of the Breakthrough Solutions Framework
BSF Concept #1 : Perspective is everything and changing it eases Transformation
BSF Concept #2 : Since your circumstances won't change, change your relationships
BSF Concept #3 : All actions and results in business are determined by memes.
Setting the stage for your transformation requires you to pay attention to the following:
vision - overreach for a goal
awareness - disconnect your future from past
strategy - uncover the execution gaps
accountability - establish an accountability
and a culture of connectedness - contributions, acknowledgements communication and gratitude.
To help with the vision, we have to evaluate the past and break from it.  Further we have to declare and map the future with the design of principles and goals and finally what is ultimately possible in that future and identify the overall outcome we are committed to.
To help with the awareness, we have to stop the mistakes of the past. The memes are evidence of the mistakes in the past. Determining the future then is a lot simpler in avoiding these.
To help with the strategy, we need to mind the gap. The gaps are visible in what the military calls as reverse salient which are backward bulges on a forward advance due to enemy strongholds. These identify the weaknesses we need to overcome.
To help with authentic accountability, we understand and negotiate new processes. For example, we can make a promise to take action. A promise is an asset. The task of making and keeping promises improves accountability.
Five steps to effective promise management: we make promises, publish promises, journal promises, act on promises and measure against promises.
The connected leader act intentionally and cause things to happen to stay on purpose. She transforms the organization into a connected culture. A consciously connected leader supports team and finds ways to remove barriers to form a consciously connected company. They have a clear understanding of what the mission critical actions are and will execute the organizations' agreed upon strategy.