文档介绍：Information ThPory
and
ThP Centrval Limit Thtorem
This page intentionally left blank
Information Theory
The Central andLimit Theorem
Oliver Johnson
University of Cambridge, UK
Imperial College Press
Published by
Imperial College Press
57 Shelton Street
Covent Garden
London WC2H 9HE
Distributed by
World Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
USA ofice: Suite 202, 1060 Main Street, River Edge, NJ 07661
UK ofice: 57 Shelton Street, Covent Garden, London WC2H 9HE
British Library Cataloguing-in-PublicationData
A catalogue record for this book is available from the British Library.
INFORMATION THEORY AND THE CENTRAL LIMIT THEOREM
Copyright 0 2004 by Imperial College Press
All rights reserved. This book, or parts thereoj may not be reproduced in any form or by any means,
electronic or mechanical, including photocopying, recording or any informution storuge and retrieval
system now known or to be invented, without written permission from the Publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright
Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to
photocopy is not required from the publisher.
ISBN 1-86094-473-6
Printed in Singapore by World scientific Printers (S)Pte Ltd
To Maria,
Thanks for everything.
Preface
“Information theory must precede probability theory and not
be based on it.” , in [Kolmogorov, 19831.
This book applies ideas from Shannon’s theory munication [Shannon
and Weaver, 19491 to the field of limit theorems in probability.
Since the normal distribution maximises entropy subject to a variance
constraint, we reformulate the Central Limit Theorem as saying that the
entropy of convolutions of independent identically distributed real-valued
random variables converges to its unique maximum. This is called conver-
gence in relative entropy or converge