News

Seminar

Permutation Methods for Model Validation

Malik Magdon-Ismail

September 15, 2010
Lally 102 - 11:00 a.m. to 12:00 p.m.

Abstract:

We give a permutation approach to validation and model selection in data mining, machine learning and statistical inference. We define permutation complexity measures in learning, and give uniform bounds on the out-sample error, similar to a VC-style bounds. We also show concentration of measure for permutation complexity, which directly means that the methods are algorithmically efficient. The theoretical novelty is that we develop methods for analyzing *dependent* sampling schemes (e.g. sampling without replacement), and show how to get concentration of measure in such a dependent sampling setting. We demonstrate the practical value of these techniques with extensive model selection experiments in real as well as synthetic data.

Hosted by: Dr. Elliot Anshelevich (x6491)

Last updated: September 13, 2010