Multi-Objective Programming in SVMs
Jinbo Bi
Department of Mathematical Sciences
Rensselaer Polytechnic Institute
Abstract.
We propose a general framework for support vector machines (SVM)
based on the principle of multi-objective optimization. The
learning of SVMs is formulated as a multi-objective program (MOP) by
setting two competing goals to minimize the empirical risk and the
model capacity. Distinct approaches to solving the MOP introduce
various SVM formulations. The proposed framework enables a more
effective minimization of the VC bound on the generalization risk.
We develop a feature selection approach based on the MOP framework
and demonstrate its effectiveness on hand-written digit data.
- This paper has
been accepted by the 20th International Conference on Machine
Learning, 2003.
- The RSVM package written in C++:
- The MatLab codes for the feature selection experiments:
The MatLab codes basically use the RSVM package to optimize the first step of MOPFS
Algorithm 1 (see paper), and use AMPL commands which call MINOS 5.5 (a commercial
optimization software) to optimize the second step of MOPFS Algorithm
1.
Contact Jinbo Bi (bij2@rpi.edu) for information about this page.