Luckily, I was at mlconf last week where Dr.Anandkumar spoke - they called her the "tensor lady" :) She's using tensors in machine learning for a bunch of things -
Latent Variable Models: Training LVM's using local search methods like EM, gradient descent, variational bayes etc. have a bunch of problems - they get stuck on local minima, the algorithms are hard to parallelize with poor convergence. In these cases, tensors yield guaranteed learning using embarassingly parallel algorithms, so faster convergence & can be run on Spark.
Also saw a demo on training 2-layer nets for GMM using tensors, and they learnt the weights rather fast. So using tensors in deep learning shows promise, though the techniques are in their infancy.
One of the challenges the professor mentioned was the availability of open source libraries to do tensor decomposition, which the above methods require.
This isn't quite right. Moment methods (that rely on tensor decompositions) have a few problems:
(i) they have convergence bounds, but in practice need more data than we have available
(ii) they don't do as well as EM usually, but using them to initialize parameters for EM sometimes does better than EM with random initialization schemes
(iii) it turns out variational methods can also be embarrassingly parallelized without losing much accuracy in practice
(iv) right now moment methods don't work for arbitrary graphical models
I believe while you are right in general, she is looking at a class of problems for which tensors handily triumph other methods. You might be interested in these papers -
Latent Variable Models: Training LVM's using local search methods like EM, gradient descent, variational bayes etc. have a bunch of problems - they get stuck on local minima, the algorithms are hard to parallelize with poor convergence. In these cases, tensors yield guaranteed learning using embarassingly parallel algorithms, so faster convergence & can be run on Spark.
Also saw a demo on training 2-layer nets for GMM using tensors, and they learnt the weights rather fast. So using tensors in deep learning shows promise, though the techniques are in their infancy.
One of the challenges the professor mentioned was the availability of open source libraries to do tensor decomposition, which the above methods require.
It was a very successful talk - https://twitter.com/cdubhland/status/594220061025435649
Tensor slides: http://www.slideshare.net/SessionsEvents/animashree-anandkum...