Wednesday, May 30, 2018

We compared BigTable, BigQuery, Cloud Datastore and Firebase in previous post. Big Table is used for Analytics, Maps and GMail. BigQuery is a data warehouse but the myth that you need to get the data into the BigQuery before performing analytics is not always true. If your data is stored in the BigTable or Cloud Storage, it can still be analyzed via BigQuery.
BigQuery uses columnar data storage to perform analytics over its data. But data does not need to reside in the BigQuery if performance is not a limitation.  External data sources such as log files in cloud storage and records in BigTable can be directly queried. With or without ETL jobs and support for streaming data, some of these analysis can be done. against external data sources.
Moreover it is not necessary that BigQuery is used only for slower analytical processing. Even near real-time analysis can be achieved by streaming data into BigQuery.
Furthermore, BigQuery does not use SQL only. It can support user-defined functions via Javascript. The functions take columns as inputs and return a result.
Together with the help of streaming analysis and user defined batch operations, a  wide variety of processing is enabled. Query quotas and pricing may affect the choice between the interactive vs batch but they can be used in an assortment for say a dashboard.

No comments:

Post a Comment