文档介绍：Machine Learning  
Question
Lecture 4: Bayesian Learning ¡ There are three doors. One door has a bag of gold.
The other two doors have tigers behind them. The
king asks you to select a door. If you open the
Jacky Baltes
door with the bag of gold behind it, you can leave
Department puter Science
with the gold. Otherwise you are lunch for the
University of Manitoba
tigers. You select the left door. The king opens
the middle door and shows you a caged tiger.
Email: ******@
¡ The king asks you again, which door you want to
/~jacky/... choose. Should you change and pick the left
Teaching/Courses/ door?
Author: Jacky Baltes University of Manitoba 2/12/03 Author: Jacky Baltes University of Manitoba 2/12/03
 
  Introduction Two Roles for Bayesian Methods
¡
Bayes Theorem ¡
Practical learning algorithms
¡
MAP, ML hypothesis ¢ Naïve Bayes learning
¡
MAP learners ¢ Bayesian works
¡
Minimum description length principle ¢ Combine prior knowledge (prior probabilities) with
¡ observed data
Bayes optimal classifier
¢ Requires prior probabilities
¡
Naïve Bayes learner ¡
Provides useful conceptual framework
¡ Example: Learning over text data
¢ Provides gold standard for evaluating other learning
¡ Bayesian works algorithms
¡
Expectation maximization algorithm ¢ Additional insight into Occam's razor
Author: Jacky Baltes University of Manitoba 2/12/03 Author: Jacky Baltes University of Manitoba 2/12/03
Basic Probability and Bayes   Basic Probability and Bayes
Theorem Theorem
¡ Mr. Smith who is correct 75% of the time claims ¡ This problem is underspecified (3 Variables)
that event X will occur
Smith Jones Actual Probability
¡ Mr. Jones who is correct 60% of the time claims N N N P0
N N Y P1
that event X will occur N Y N P2
¡ N Y Y P3
What is p(X)? Y N N P4
Y N Y P5
Y Y N P6
Y Y Y P7
Author: Jacky Baltes Uni