Let’s compare our description in yesterday's post with the Big Data architectural style
of building services. This can be a
vectorized execution environment and typically involving a size of data not
seen with the traditional database systems. Both the storage and the message
queue handle large volume of data and the execution can be stages as processing
and analysis. The processing can be
either batch oriented or stream oriented.
The analysis and reporting can be offloaded to a variety of technology
stacks with impressive dashboards. While the processing handles the
requirements for batch and real-time processing on the big data, the analytics
supports exploration and rendering of output from big data. It utilizes
components such as data sources, data storage, batch processors, stream
processors, real-time message queue, analytics data store, analytics and
reporting stacks, and orchestration.
Some of the benefits of this application include the
following: The ability to mix technology choices, achieving performance through
parallelism, elastic scale and interoperability with existing solutions.
Some of the challenges faced with this architectural
style include: The complexity where numerous components are required to handle
the multiple data sources, and the challenge to build, deploy and test big data
processes. Different products require as many as skillsets and maintenance with
a requirement for data and query virtualization. For example, U-SQL which is a
combination of SQL and C# is used with Azure Data Lake Analytics while SQL APIs
are used with Hive, HBase, FLink and Spark. With this kind of a landscape, the
emphasis on data security gets diluted and spread over a very large number of
components.
Some of the best practices with this architectural style
leverage parallelism, partition data, apply schema-on read semantics, process
data in place, balance utilization and time costs, separate cluster resources,
orchestrate data ingestion and scrub sensitive data
Some examples include applications that leverage IoT
architecture and edge computing.
Conclusion: Both these
styles serve their purpose of a cloud service very well.
No comments:
Post a Comment