文档介绍：Statistical machine learning
for information retrieval
Adam Berger
April, 2001
CMU-CS-01-110
School puter Science
Carnegie Mellon University
Pittsburgh, PA 15213
Submitted in partial ful
llment of the requirements
for the degree of Doctor of Philosophy.
mittee:
John La
erty, Chair
Jamie Callan
Jaime Carbonell
Jan Pedersen (Centrata Corp.)
Daniel Sleator
Copyright
c 2001 Adam Berger
This research was supported in part by NSF grants IIS-9873009 and IRI-9314969, DARPA AASERT award
DAAH04-95-1-0475, an IBM Cooperative Fellowship, an IBM University Partnership Award, a grant from
JustSystem Corporation, and by Claritech Corporation.
The views and conclusions contained in this document are those of the author and should not be interpreted as
representing the official policies, either expressed or implied, of IBM Corporation, JustSystem Corporation,
Clairvoyance Corporation, or the United States Government.
3
Keywords
Information retrieval, machine learning, language models, statistical inference, Hidden Markov
Models, information theory, text summarization
4
5
Dedication
I am indebted to a number of people and institutions for their support while I conducted
the work reported in this thesis.
IBM sponsored my research for three years with a University Partnership and a Cooper-
ative Fellowship. I am in IBM’s debt in another way, having previously worked for a number
of years in the automatic language translation and speech recognition departments at the
Thomas J. Watson Research Center, where I collaborated with a group of scientists whose
combination of intellectual rigor and scienti
c curiosity I expect never to
nd again. I am
also grateful to Claritech Corporation for hosting me for several months in 1999, and for al-
lowing me to witness and contribute to the development of real-world, practical information
retrieval systems.
My advisor, colleague, and sponsor in this endeavor has been John La
erty. Despite o