Today we continue discussing the best practice from storage engineering:
406) Catalogs support data modeling, data synchronization, data standardization, and flexible workflows. There are layer of information management using catalog starting with print/translation workflows at the bottom layer, followed by workflow or security for access to the assets, their editing, insertions and bulk insertions, followed by integrations or portals followed by flexible integration capabilities, full/data exports, multiple exports, followed by integration portals for integration with imports/exports, data pools and platforms, followed by digital asset management layer for asset on-boarding and delivery to channels, and lastly data management for searches, saved searches, channel based content, or localized content and the ability to author variants, categories, attributes and relationships to stored assets.
407) Relational databases suffer from limitations for supporting catalog workflows due to the following:
The field inventory is a local view only until it makes its way to the central store.
The relational store involves a one-a-day sync or something periodic
Stale views are served until the refresh happens which is often not fast enough for consumers.
The stale view interferes with analytics and aggregations reports.
Downstream internal and external apps have to work around the delays and stale views with sub-optimal logic.
408) The purpose of the catalog is to form a single view of the product with one central service, flexible schema, high read volume, write spike tolerant during catalog update, and to have advanced indexing and querying and geographical distribution for HA and low latency.
409) Object storage provides local real-time writes while supporting a read dominated workload. More geographical distribution and horizontal scalability helps improve performance
410) The objects can live in the cache as well. A search engine can provide search over the catalog. Functional data access can be provided by the API. The API and the engine can separately cover all operations on the catalog.
406) Catalogs support data modeling, data synchronization, data standardization, and flexible workflows. There are layer of information management using catalog starting with print/translation workflows at the bottom layer, followed by workflow or security for access to the assets, their editing, insertions and bulk insertions, followed by integrations or portals followed by flexible integration capabilities, full/data exports, multiple exports, followed by integration portals for integration with imports/exports, data pools and platforms, followed by digital asset management layer for asset on-boarding and delivery to channels, and lastly data management for searches, saved searches, channel based content, or localized content and the ability to author variants, categories, attributes and relationships to stored assets.
407) Relational databases suffer from limitations for supporting catalog workflows due to the following:
The field inventory is a local view only until it makes its way to the central store.
The relational store involves a one-a-day sync or something periodic
Stale views are served until the refresh happens which is often not fast enough for consumers.
The stale view interferes with analytics and aggregations reports.
Downstream internal and external apps have to work around the delays and stale views with sub-optimal logic.
408) The purpose of the catalog is to form a single view of the product with one central service, flexible schema, high read volume, write spike tolerant during catalog update, and to have advanced indexing and querying and geographical distribution for HA and low latency.
409) Object storage provides local real-time writes while supporting a read dominated workload. More geographical distribution and horizontal scalability helps improve performance
410) The objects can live in the cache as well. A search engine can provide search over the catalog. Functional data access can be provided by the API. The API and the engine can separately cover all operations on the catalog.
No comments:
Post a Comment