Modern Information Processing

From Theory to Applications

Bernadette Bouchon-Meunier, Giulianella Coletti, Ronald R. Yager

• Publisher: Elsevier Science
• Year: 2011
• Print ISBN: 9780444520753
• eBook ISBN: 9780080461694
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

The volume "Modern Information Processing: From Theory to Applications, " edited by Bernadette Bouchon-Meunier, Giulianella Coletti and Ronald Yager, is a collection of carefully selected papers drawn from the program of IPMU'04, which was held in Perugia, Italy. The book represents the cultural policy of IPMU conference which is not focused on narrow range of methodologies, but on the contrary welcomes all the theories for the management of uncertainty and aggregation of information in intelligent systems, providing a medium for the exchange of ideas between theoreticians and practitioners in these and related areas.The book is composed by 7 sections: UNCERTAINTYPREFERENCESCLASSIFICATION AND DATA MININGAGGREGATION AND MULTI-CRITERIA DECISION MAKINGKNOWLEDGE REPRESENTATION•The book contributes to enhancement of our ability to deal effectively with uncertainty in all of its manifestations. •The book can help to build brigs among theories and methods methods for the management of uncertainty. •The book addresses issues which have a position of centrality in our information-centric world. •The book presents interesting results devoted to representing knowledge: the goal is to capture the subtlety of human knowledge (richness) and to allow computer manipulation (formalization). •The book contributes to the goal: an efficient use of the information for a good decision strategy.APPLIED DOMAINS· The book contributes to enhancement of our ability to deal effectively with uncertainty in all of its manifestations.· The book can help to build brigs among theories and methods methods for the management of uncertainty.· The book addresses issues which have a position of centrality in our information-centric world.· The book presents interesting results devoted to representing knowledge: the goal is to capture the subtlety of human knowledge (richness) and to allow computer manipulation (formalization).· The book contributes to the goal: an efficient use of the information for a good decision strategy.

Uncertainty

Entropies, Characterizations, Applications and Some History

János Aczél, Faculty of Mathematics University of Waterloo, Waterloo, ON N2L 3G1, Canada. E-mail address: [email protected]

Abstract

Entropies with useful and/or interesting properties are presented. Characterizations are given, based on such properties and some applications are mentioned. Attention is directed to an example of discovery and rediscovery, and to new applications in utility theory.

1 INTRODUCTION, ENTROPY

Defining new entropies in addition to the classical Shannon entropy seems to be an ongoing industry. I am still convinced, however, of what I wrote in [1]:

“In the best of all possible worlds there is an information measure that originated from an applied problem, has interesting properties (usually attractive, reasonable generalizations of properties of Shannon’s entropy or of similar widely used measures), and those characterize it. Less ideal but still acceptable is in my opinion the following situation. Some natural looking weakening or generalization of properties characterizing Shannon type measures are isolated and all measures having these properties are determined. If the properties are indeed intuitive and significant then there is a good chance that the measures thus obtained may have future applications.

But what many authors seem to do is to contrive some generalization of known information measures (usually by sticking parameters almost at random here and there), derive its often not very interesting or natural or even attractive properties and then characterize by several of these properties the ‘measures’ they have defined in the first place. Not many good or useful results can be expected from this kind of activity.”

An earlier version of the present paper appeared in [2], As there, I express also here my belief that two families of probabilistic entropies (of which Shannon’s entropy is a limit case) suffice - in addition to entropies depending on objects other than probabilities. I will define them and state (without proof but with references) some of their properties and characterizations that I consider reasonable, and mention some applications.

Our models will mostly be complete systems of mutually exclusive events E₁,.., E_n (such as the possible outcomes of an experiment), with probabilities p₁, ….,p_n, respectively

Entropies are measures of uncertainty in or measures of information to be gained from such a system (we will mention also entropies for incomplete systems, where

2 SHANNON ENTROPY

The Shannon entropy of a complete system is given (with 0 log₂ 0

0) by

(1)

NOTE 1. If incomplete systems are permitted

then the definition

is used. Here n = 1 is also permissible and gives as entropy of a single event

We take h...

Table of contents

• Cover image
• Title page
• Table of Contents
• Inside Front Cover
• Copyright
• Foreword
• Uncertainty
• Preferences
• Classification and Data Mining
• Aggregation and Multi-Criteria Decision Making
• Knowledge Representation
• Applied Domains
• Author Index