News

Colloquia

An Information Geometric Unification of Non-Maximum Likelihood Learning Objectives

Siwei Lyu
University at Albany, SUNY

April 7, 2011
JEC 3117- 4:00 p.m. to 5:00 p.m.

Abstract:

Many non-maximum likelihood parametric learning methods have been developed to alleviate the intractable computation of the maximum likelihood (ML) learning for high dimensional probabilistic models. In this work, we describe a general learning methodology known as minimum KL contraction that unifies a wide range of non-ML learning methods. Built from a geometric view, in minimum KL contraction learning, we seek optimal parameters that minimizes the reduction of the KL divergence between the data and model distributions after they are transformed with a KL contraction operator. We show that with specific instantiations of the KL contraction operator, it generalizes the objective functions of several important non-ML learning methods, including contrastive divergence, score matching, maximum pseudo-likelihood, maximum composite likelihood, maximum conditional likelihood, and noise contrast estimation. This new unified view of different non-ML learning objectives can provide hints in designing new and more effective learning schemes.

Bio:

Siwei Lyu received his B.S. degree (Information Science) in 1997 and his M.S. degree (Computer Science) in 2000, both from Peking University, Beijing China. He received his Ph.D. degree in Computer Science from Dartmouth College in 2005. From 2000 to 2001, he worked at Microsoft Research Asia (then Microsoft Research China) as an Assistant Researcher. From 2005 to 2008, he was a Post-Doctoral Research Associate at the Howard Hughes Medical Institute and the Center for Neural Science of New York University. Starting in 2008, he is Assistant Professor at the Computer Science Department of University at Albany, State University of New York. He is the recipient of the Alumni Thesis Award of Dartmouth College in 2005, IEEE Signal Processing Society Best Paper Award in 2010, and the NSF CAREER Award in 2010. He has authored one book, and held two U.S. and one E.U. patents. He has published more than 30 conference and journal papers in the research fields of natural image statistics, digital image forensics, machine learning and computer vision.

Hosted by: Dr. Jeff Trinkle

Last updated: March 31, 2011