Matrix factorization for recommendation systems.  

What is a recommendation system?  

A recommendation system provides choices when a user makes a purchase or a request. For example, this system shows the user what choices others make so that the user can be better informed. Another example is when the system shows the user what other items are like the one being requested.  

Where can I use a recommendation system?  

Recommendations include suggestions for online shopping, suggesting interesting websites, or finding items such as movies or music. Since it can be based on user preferences or item similarity, the choices are not restricted to any domain. This kind of recommendation system when applied to any catalog in any business, performs as a collaborative filter.   
There are certain filtering cases where divulging which users go with what preferences are helpful to the end-user. At other times, it is preferable to use item-based similarity. The similarity scores are computed in both cases. All other considerations being the same, the item-based approach is better for sparse datasets. Both user-based and item-based approaches perform similarly for the dense dataset.   

How does it work?   

To make a recommendation, first, a group sharing similar tastes is found and then the preferences of the group are used to make a ranked list of suggestions. This technique is called collaborative filtering. A common data structure that helps with keeping track of people and their preferences is a nested dictionary. This dictionary could use a quantitative ranking say on a scale of 1 to 5 to denote the preferences of the people in the selected group.  To find similar people to form a group, we use some form of a similarity score. One way to calculate this score is to plot the items that the people have ranked in common and use them as axes in a chart. Then the people who are close together on the chart can form a group.   

What is matrix factorization?   

User choices are often represented as a matrix.  For example, Netflix provided a training dataset of over 100 million ratings from nearly half a million users on over seventeen thousand movies. Given this matrix of ratings between m users and n movies, matrix factorization decomposes it into m *k users matrix and k * n movies matrix such that it becomes a recommendation system that leverages the latent features between two entities.  

How does Matrix factorization improve recommendation systems?   

Decomposition helps solve for x in the system of linear equations denoted by Ax = b where A is the given matrix and b is the set of recommendations from the participants. The benefit of matrix factorization is that it can be applied to sparse matrices where not all users rate all movies and do so only selectively. Matrix factorization is a technique for solving a linear system of equations.  

Given a user rating of movies in a matrix, it can be decomposed into a dot product of a user embedding matrix and a movie embedding matrix. Embedding refers to latent features corresponding to that entity. The user latent features and movie latent features can then be looked up corresponding to a movie-user rating. The embeddings matrix also serves as input to other neural net layers some of which can involve non-linear layers and introduce more refinements to predictions. Other matrix factorization techniques to utilize the initial matrix form include principal component analysis and singular value decomposition. The former brings out the salient contributors in the ratings while the latter brings out the salient preferences of a user such as 50% comedy fan + 50% romance fan where that user is only sensitive to these latent factors.  

When the model is trained on these ratings by users on movies, it can be used to predict the rating of a new movie by finding how similar it is to the other movies based on the user preferences.  

What are the different types of Matrix Factorization?  

Matrix factorizations differ in the form of factorizations such as lower triangular- upper triangular or matrix and its transpose and such others. Matrix factorizations also differ in the method in which they are factorized. There is one form that performs better for a variety of tasks. This is the Cholesky factorization which has advantages such as   

• Simpler solving   

• Fewer iterations   

• The transformation from uncorrelated to correlated normal variables.    

• more efficient than LU and QR decomposition   

• Best for positive definite matrices   

• Always a good bet for sparse data   

How is QR decomposition different from LU decomposition?  

QR decomposition is used to solve the Least Squares problem which solves a system of linear equations denoted by Xb = Y where the inverse of matrix X cannot be calculated and instead an approximation is taken such that sum of squared deviations is minimized. QR form is one where R is the upper triangular matrix. LU decomposition results in a lower triangular matrix and an upper triangular matrix. Gaussian elimination is a popular form of LU decomposition. It involves adding multiples of one row to the other, swapping two rows, and scaling a row to result in the lower and upper triangular matrices.