Tuesday, January 21, 2020

Stream storage is not a commodity. It is an emerging market. There are a handful of players at best and very few solutions built on top of their products. The solution architects find developer appeal with the new products that combine analytics with storage. However, the devOps engineers prefer scriptability to programmability to expedite plugging the product in data pipelines.
The conventional solution to addressing the need for scriptability is demonstrated very well by Amazon Web Services (AWS) software development kit (SDK). There exists one for every popular language and widens the access to underlying services. Each service is accessible over the web with Representation state transfer (REST) Application programming interface (API) which is then automated and made more scriptable with client-side local SDK that can be integrated easily with existing and new solutions because it speaks the same language that the client wants.
The stream stores integrated with analytical software with store specific connectors that put to good use the gRPC protocol. The REST interface for the data path was somewhat lacking which made it difficult to embrace the conventional solution. REST and gRPC are essentially data transfer languages except that REST is a way of requesting ‘resources’ from the remote end via standard verbs such as GET, PUT etc and gRPC is a way of specifying arbitrary procedures to be invoked on the server. These routines are the equivalent of verbs and resources and some treat this communication as a refinement of erstwhile Simple Object Access Protocol (SOAP).
The advantages of REST are:
Requires HTTP/1.1
Supports subscription mechanisms with REST hooks
Comes with widely accepted tool and browser support
Well defined road to development of the service that provides this communication
Supports discovery of resource identifiers with subsequent request response models.
Is supportive of software development kit where more than one language can be supported for the use of these communication interfaces.
The disadvantages are:
Is considered chatty because there are a number of requests and responses
Is considered heavy because the payload is usually large.
Is considered inflexible at times with versioning costs
The advantages of gRPC are:
Supports high speed communication because it is lightweight and does not require the traversal of stack all the way up and down the networking layers.
The messages are over “Protocol Buffer” which is known for being efficient in packing and unpacking data
It works over newer HTTP/2
Best for traffic from devices (IoT)
The disadvantages are
Requires client to write code
Does not support browser

The conventional solution therefore has to be enhanced with the provisioning of REST APIs because command line invocation and browser support wins hands down for scriptability.

No comments:

Post a Comment