The stress test
The main focus of our project was not to simply implement matrix completion -- that has been done many times in the past and is proven to work. What we wanted to do was put the algorithm through a “stress test” to understand its limits.
To do this, we devised the following plan:
- Construct a low rank matrix
- Create a model of uniformity
- Vary the uniformity on unobserved indices in a matrix
- Complete the matrix
- Measure mean squared error between the original matrix and the completed matrix
- Plot mean squared error as a function of uniformity
Constructing a low rank matrix
In order to construct a low rank matrix, remove indices, and compare the results of matrix completion with the actual matrix, we first created two separate matrices of size m x r and r x n with random entries between 0 and 1, where r in the desired rank. We then multiplied those two matrices together, constructing one matrix of size m x n and rank r.
Modeling uniformity
Our second challenge was to generate a model of uniformity that provided a simple way to change how uniform the unobserved indices were. We decided to use a jointly Gaussian distribution, modifying the covariance to control uniformity. The idea is simple: low covariance results in a high concentration of indices removed from the center of the matrix, while high covariance results in an almost perfectly uniform distribution.
In the figure below, the matrix on the left shows a matrix with a very uniform distribution of unobserved entries, characterized by a jointly gaussian distribution with high covariance. The matrix on the right demonstrates a data set with a high degree of nonuniformity. The probability peaks at the center of the matrix, resulting in unobserved indices which are highly concentrated in the middle of the matrix.
Plotting mse and uniformity
After completing the matrix, we calculated the mean squared error between the original matrix and the prediction provided by matrix completion and stored it in an array, along with the corresponding uniformity percentage. We calculated the uniformity percentage by normalizing each covariance by the largest covariance used, providing a scale between 0 and 100%.