文档介绍：Journal of Artificial Intelligence Research 27 (2006) 25-53 Submitted 10/05; published 9/06
Generative Prior Knowledge for Discriminative Classification
Arkady Epshteyn ******@
Gerald DeJong ******@
Department puter Science
University of Illinois at Urbana-Champaign
201 N. Goodwin
Urbana, IL, 61801 USA
Abstract
We present a novel framework for integrating prior knowledge into discriminative clas-
sifiers. Our framework allows discriminative classifiers such as Support Vector Machines
(SVMs) to utilize prior knowledge specified in the generative setting. The dual objective of
fitting the data and respecting prior knowledge is formulated as a bilevel program, which
is solved (approximately) via iterative application of second-order cone programming. To
test our approach, we consider the problem of using (a semantic database of
English language) to improve low-sample classification accuracy of newsgroup categoriza-
tion. is viewed as an approximate, but readily available source of background
knowledge, and our framework is capable of utilizing it in a flexible way.
1. Introduction
While SVM (Vapnik, 1995) classification accuracy on many classification tasks is often
competitive with that of human subjects, the number of training examples required to
achieve this accuracy is prohibitively large for some domains. Intelligent user interfaces,
for example, must adopt to the behavior of an individual user after a limited amount of
interaction in order to be useful. Medical systems diagnosing rare diseases have to generalize
well after seeing very few examples. Any natural language processing task that performs
processing at the level of n-grams or phrases (which is frequent in translation systems)
cannot expect to see the same sequence of words a sufficient number of times even in large
training corpora. Moreover, supervised classification methods rely on manually labeled
data, which can be expensive to obtain. Thus, it is import