Applications of Data Mining to Reward points collection service

Continuation of discussion in terms of Machine Learning deployments

Machine learning algorithms are a tiny fraction of the overall code that is used to realize prediction systems in production. As noted in the paper on “Hidden Technical Debt in Machine Learning systems” by Sculley, Holt and others, the machine learning code comprises mainly of the model but all the other components such as configuration, data collection, features extraction, data verification, process management tools, machine resource management, serving infrastructure, and monitoring comprise the rest of the stack. All these components are usually hybrid stacks in nature especially when the model is hosted on-premises. Public clouds do have a pipeline and relevant automation with better management and monitoring programmability than on-premises, but it is usually easier for startups to embrace public clouds than established large companies who have significant investments in their inventory, DevOps and datacenters.

Deployments can be on Infrastructure as a service, Platform as a Service, Software as a service, clusters, containers, computer grids and computer clouds. The choice depends on convenience, cost and effectiveness.  A web server can easily be hosted on a virtual machine and since the VM is hosted in the cloud, it can use the cloud features such as global availability, managed environment, load balancing, and disaster recovery. PaaS can be used where an application runtime is needed because stacks like OpenShift, Cloud Foundry, Pivotal or Azure are runtime environments. The hosting and deployment of applications is managed. Software as a service are complete applications by themselves. These can be accessed from any client including mobile devices. A machine learning model hosted on SaaS can be used remotely from any handheld device with an IP connectivity and a browser. Grids and supercomputers are specialized systems and were required for scientific computations. With newer algorithms for commodity servers in the cloud computing framework and with the creation of higher capability clusters, the ability to use advanced machine learning algorithms has been made easier.

Applications and resources can be managed efficiently with the help of system center tools. They can work across clouds and networks. For example, tools can help monitor resources on-premises as well as public cloud with the help of agents deployed to these resources. The agents collect all information necessary to provide drilldowns as well as summary at hierarchical scopes. A virtual private cloud provides isolation from other resources. This comes in helpful for development and test before deploying to production systems where the operations engineering charts and graphs can now specify the expectations around the metrics.

 

The following chart makes a comparison of all the data mining algorithms including the neural networks: https://1drv.ms/w/s!Ashlm-Nw-wnWxBFlhCtfFkoVDRDa?e=aVT37e

Thank you.