文档介绍：Information Theory,
Inference,
and Learning Algorithms
David . MacKay
******@
c 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
°
Draft January 12, 2003
Please send feedback on this book via
./
1
Information Theory, Pattern Recognition and works
Approximate roadmap for the eight-week course in Cambridge
The course will cover about 16 chapters of this book. The rest of the book is
provided for your interest. The book contains numerous exercises with worked
solutions.
Lecture 1 Introduction to Information Theory. Chapter 1.
Before lecture 2 Work on exercise ().
Read chapters 2 and 4 and work on exercises in chapter 2.
Lecture 2–3 Information content & typicality. Chapter 4.
Lecture 4 Symbol codes. Chapter 5.
Lecture 5 Arithmetic codes. Chapter 6 (sections – only).
Read chapter 8 and do the exercises.
Lecture 6 Noisy channels. Definition of mutual information and capacity. Chapter 9.
Lecture 7–8 The noisy channel coding theorem. Chapter 10 (but not section on-
wards).
Lecture 9 Clustering. Bayesian inference. Chapter 3, 22, 24.
Read chapter 33 (Ising models).
Lecture 10–11 Monte Carlo methods. Chapter 31, 32.
Lecture 12 Variational methods. Chapter 35.
Lecture 13 works – the single neuron. Chapter 42.
Lecture 14 Capacity of the single neuron. Chapter 43.
Lecture 15 Learning as inference. Chapter 44.
Lecture 16 The work. Content-addressable memory. Chapter 45.
About the exercises
I firmly believe that one can only understand a subject by recreating it for
oneself. To this end, I think it is essential to work through some exercises on
each topic. For guidance, each exercise has a rating (similar to that used by
Knuth (1968)) from 1 to 5 that indicates the level of difficulty.
In addition, exercises that are especially mended are marked by a
marginal encouraging rat –. Exercises that require the use of puter
may be marked with a C.
I’ll circulate detaile