Stochastic Gradients for Large-Scale Tensor Decomposition

Abstract

Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is a recently proposed version of tensor decomposition that allows for a variety of loss functions such as logistic loss for binary data or Huber loss for robust estimation. The stochastic gradient is formed from randomly sampled elements of the tensor. For dense tensors, we simply use uniform sampling. For sparse tensors, we propose two types of stratified sampling that give precedence to sampling nonzeros. Numerical results demonstrate the advantages of the proposed approach and its scalability to large-scale problems.

Publication
arXiv
Date
Citation
T. G. Kolda, D. Hong. Stochastic Gradients for Large-Scale Tensor Decomposition. arXiv:1906.01687, 2019.

Keywords

stat.ML, cs.LG, math.NA

BibTeX

@misc{arXiv-KoHo19,  
author = {Tamara G. Kolda and David Hong}, 
title = {Stochastic Gradients for Large-Scale Tensor Decomposition}, 
month = {June}, 
year = {2019},
eprint = {1906.01687},
eprintclass = {stat.ML},
}