Abstract: The past few years have seen a dramatic increase in the performance of recognition systems thanks to the introduction of deep networks for representation learning. However, the mathematical reasons for this success remain elusive. A key issue is that the neural network training problem is non-convex, hence optimization algorithms may not return a global minima. Building on ideas from convex relaxations of matrix factorizations, this work proposes a general framework which allows for the analysis of a wide range of non-convex factorization problems – including matrix factorization, tensor factorization, and deep neural network training. The talk will describe sufficient conditions under which a local minimum of the non-convex optimization problem is a global minimum and show that if the size of the factorized variables is large enough then from any initialization it is possible to find a global minimizer using a local descent algorithm. Joint work with Ben Haeffele.
Bio: Professor Vidal received his B.S. degree in Electrical Engineering from the Pontificia Universidad Católica de Chile in 1997 and his M.S. and Ph.D. degrees in Electrical Engineering and Computer Sciences from the University of California at Berkeley in 2000 and 2003, respectively. In 2004 he joined the Johns Hopkins University, where he is currently a Professor in the Department of Biomedical Engineering and Director of the Mathematical Institute for Data Science (MINDS). Dr. Vidal is co-author of the book “Generalized Principal Component Analysis” (2016), co-editor of the book “Dynamical Vision” (2006), and co-authored of more than 200 articles in machine learning, computer vision, biomedical image analysis, hybrid systems, robotics and signal processing. Dr. Vidal is Associate Editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence, the SIAM Journal on Imaging Sciences, Computer Vision and Image Understanding, and Medical Image Analysis. He has been Program Chair for ICCV 2015 and CVPR 2014, and Area Chair for all major conferences in machine learning, computer vision, and medical image analysis. Dr. Vidal has received many awards for his work including the 2012 J.K. Aggarwal Prize, the 2009 ONR Young Investigator Award, the 2009 Sloan Research Fellowship, the 2005 NFS CAREER Award, and best paper awards at in computer vision (ICCV-3DRR 2013, PSIVT 2013, ECCV 2004), controls (CDC 2012, CDC 2011) and medical robotics (MICCAI 2012). Dr. Vidal was elected fellow of the IEEE in 2014 and fellow of the IAPR in 2016.