Content reselling platform continued

 

Components:

Storage:

Media content is binary and immutable in nature. An S3 storage enables it to have web accessible address. Generating a presigned URL to upload an object enables the user interface to bypass the middle tier for uploads. This will significantly improve the time it takes to upload content. S3 acceleration can help with downloading by pointing to the closest mirror. The metadata and user information can be collected in a relational store for use by the webAPI.

When the application requires to serve the users to the scale of social engineering applications, Presto can be used to store the high-volume data in NoSQL stores and with the ability to bridge a SQL query over the data. Presto from Facebook is a distributed SQL query engine can operate on streams from various data source supporting ad-hoc queries in near real-time. It does not partition based on MapReduce and executes the query with a custom SQL execution engine written in Java. It has a pipelined data model that can run multiple stages at once while pipelining the data between stages as it becomes available. This reduces end to end time while maximizing parallelization via stages on large data sets.   
A data warehouse can be supported in the cloud in virtual data centers. It can support data ingestion in the form of JSON from data pipelines. The ability to perform queries over this warehouse follows the conventional Online Analytical Processing model and serves the reward points very well. While Presto does not dictate the data source, a data warehouse forms a veritable data store. Both can scale but there are cost-benefit ratios to consider when deploying custom stores via something offered from public clouds. 

WebAPI:

The APIs for Media and content publishing can simply be REST/GraphQL APIs for a resource named say Movie, which looks like the following: 

·        Storing movies in a database with user details such as  

o   Movie-form,  

o   user relationship,  

o   target URL to fulfil the reward from backend store, and  

o   active versus inactive state for movie

·        API to create/update/delete movies that includes: 

o   GET /api/v1/movie/ to list 

o   POST /api/v1/movie/ to upload

o   GET /api/v1/movie/:id/ to lookup  

o   PUT /api/v1/movie/:id/ to edit 

o   DELETE /api/v1/movie/:id to delete

·        List and implement movie types 

o   a name in the name.verb syntax. 

o   a payload to simply mirror the representation from the standard API. 

·        send hooks with POST to each of the target URLs for each matching movie 

o   compiling and POSTing the combined payload for the triggering resource and hook resource 

o   sending to known online retail stores with the Movie where X-Hook-Secret header has a unique string and one that matches what was issued by the backend retail store. 

o   confirming the hook legitimacy with a X-Hook-Signature header 

o   Handling responses like the 410 Gone and optionally retrying connection or other 4xx/5xx errors. 

 

Rules Engine:

A Rules Engine can be provided that introduces a callback as a URL for the Movie API servers to call which permits a dynamic policy for the translation of movies and the ability for the organization to set policies. 

·        Rule Execution Sets should be provided by the organization administrators 

o   Any complex rule involving nested if and else should be flattened to ordered list of if statements and appropriate conditions. 

o   Ordering of the conditions should be specified by the administrators. 

·        Exports or Objects returned by the Rule Engine should be published 

o   Multiple objects should be returned as a list 

o   These should be filtered by the object filter specified by the client 

o   Not include any sensitive information.