文档介绍：Information Theory, Inference, and Learning Algorithms
David . MacKay
Information Theory,
Inference,
and Learning Algorithms
David . MacKay
******@
c 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003
Version (as published) June 26, 2003
Please send feedback on this book via
./
This book will be published by . in September 2003. It will remain
viewable on-screen on the above website, in postscript, djvu, and pdf
formats.
(. replace this page with their own page ii.)
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
1 Introduction to Information Theory . . . . . . . . . . . . . . . . 3
2 Probability, Entropy, and Inference . . . . . . . . . . . . . . . . . 22
3 More about Inference . . . . . . . . . . . . . . . . . . . . . . . . 48
I pression . . . . . . . . . . . . . . . . . . . . . . . . 65
4 The Source Coding Theorem . . . . . . . . . . . . . . . . . . . . 67
5 Symbol Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6 Stream Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7 Codes for Integers . . . . . . . . . . . . . . . . . . . . . . . . . . 132
II Noisy-Channel Coding . . . . . . . . . . . . . . . . . . . . . . 137
8 Correlated Random Variables . . . . . . . . . . . . . . . . . . . . 138
munication over a Noisy Channel . . . . . . . . . . . . . . . 146
10 The Noisy-Channel Coding Theorem . . . . . . . . . . . . . . . . 162
11 Error-Correcting Codes and Real Channels . . . . . . . . . . . . 177
III Further Topics in Information Theory . . . . . . . . . . . . . 191
12 Hash Codes: Codes for Efficient Information Retrieval . . . . . 193
13 Binary Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
14 Very Good Linear Codes Exist . . . . . . . . . . . . . . . . . . . 229
15 Further Exercises on Information Theory . . . . . . . . . . . . . 233
16 Message Passing . . .