文档介绍：Mutual Information for Image Registration and Feature Selection M. Farmer CSE-902 Problem Definitions ? Image Registration: – Define a transform T that will map one image onto another image of the same object such that some image quality criterion is maximized. ? Feature Selection: – Given d features, find the best subset of size m, m<d –‘ Best ’ can be defined as ? minimizing the classification error ? maximizing discrimination ability of feature set Measures of Information ? Hartley defined the first information measure: – H = n log s – n is the length of the message and s is the number of possible values for each symbol in the message – Assumes all symbols equally likely to occur ? Shannon proposed variant (Shannon ’ s Entropy) ? weighs the information based on the probability that an e will occur ? second term shows the amount of information an event provides is inversely proportional to its prob of occurring ??? ii ip pH 1 log Three Interpretations of Entropy ? The amount of information an event provides – An infrequently occurring event provides more information than a frequently occurring event ? The uncertainty in the e of an event – Systems with one mon event have less entropy than systems with many equally probable events ? The dispersion in the probability distribution – An image of a single amplitude has a less disperse histogram than an image of many greyscales ? the lower dispersion implies lower entropy Alternative Definitions of Entropy ? The following generating function can be used as an abstract definition of entropy: ? Various definitions of these parameters provide different definitions of entropy. – Actually found over 20 definitions of entropy ???????????????Mi ii Mi iipv pvhPH 1 2 1 1)( )()(?? Alternative Definitions of Entropy Alternative Definitions of Entropy II Glossary of Entropy Definitions # Name # Name # Name # Name 1 Shannon 7 Varma 13 Taneja 19 Belis- Guiasu,Gil 2 Renyi 8 Kapur 14 Sharma- Taneja 20 Picard 3 Aczel-