Friday, October 17, 2014

Today I want to discuss building REST based APIs  for enabling CRUD Operations. Specifically  we will be discussing the need for a store behind the API. As one example, the data model itself can be exposed as REST  based in which case it becomes OData it helps if the data is in a database in which case we get the guarantees that come with it sell such as the operations are atomic, consistent, isolated and durable. But let us assume that the data is not in the database and that the API layer has to do operations on the wire to fetch the data. In such cases I've used some of the following. First, we keep the resources  well identified. By that I mean we use as many columns as necessary to index the data  indexing has a lot of benefits but the suggestion here is to decide between  keeping the data sorted or building a separate index.  Secondly we could keep a status column that cycles between created modified and deleted and some other states in between  depending on what would help. The status column serves to eliminate such concerns as duplicates,  retries, consistency, concurrency on systems that don't support it already. Let us take a look at a few examples now.
First, the user wants to insert a record. Where do we store this. Is it a separate partition, table, database, database server, etc. If there are many of those, then they can probably be arranged in a row or a grid where they are accessible by their position. In fact if we have redundancies we could use the arrangement for the clones so they are side by side with the master in the array.
Next, how do we represent the record is in this location. This can probably be done by keeping the location information as part of the key for the record. A Multipart name could be helpful. Something like database server. database.schema.table etc. Given this record we know how to reach it and that there are no conflicts.
Next, we look at the operations themselves? Do we need transaction or can we do retries ? Which one is less intensive and cheaper.
Next we look at the operations on the same item? Do we need to make sure the item will be consistent between concurrent operations ? The same goes for sequential train of operations.
Lastly, we look at the state of the records. They should be progressive and cyclical if not terminating.
If the states are forward only, it supports retries. This helps it be robust but the operations may require lookups for each retry.
Retries work something like this:

status retry(Item item)
{
   var state = lookup(item);
   if (state is null)
    {
    // create item
      item.state = created;
     return item.state;
    }
  if (state is created)  
    {
       // modify or delete
       update(item);
      item.state = modified;
      return item.modified;
    }
  if (item is notReady)
   {
     ready(item);
     item.state = ready;
     return item.state;
    }
  item.state = null;
  return item.state;
}

No comments:

Post a Comment