Hostname: page-component-7b7567c8c-kwt9d Total loading time: 0 Render date: 2026-11-14T06:27:49.592Z Has data issue: false hasContentIssue false
  • English
  • Français

On an entropy conservation principle

Part of:Sufficiency and informationCommunication, information

Published online by Cambridge University Press:  14 July 2016

Jérôme Manuceau,
Marylène Troupé and
Jean Vaillant
Jérôme Manuceau*
Affiliation:
University of Antilles-Guyane
Marylène Troupé*
Affiliation:
University of Antilles-Guyane
Jean Vaillant*
Affiliation:
University of Antilles-Guyane
*
∗ Postal address: UFR Sciences, Department of Mathematics, 97169 Pointe-à-Pitre, Guadeloupe, FWI.
∗ Postal address: UFR Sciences, Department of Mathematics, 97169 Pointe-à-Pitre, Guadeloupe, FWI.
∗ Postal address: UFR Sciences, Department of Mathematics, 97169 Pointe-à-Pitre, Guadeloupe, FWI.
Get accessRights & Permissions [Opens in a new window]

Abstract

We present an entropy conservation principle applicable to either discrete or continuous variables which provides a useful tool for aggregating observations. The associated method of modality grouping transforms a variable Z 1 into a new variable Z 2 such that the mutual information I(Z 2,Y) between Y, a variable of interest, and Z 2 is equal to I(Z 1,Y).

Keywords

Entropymutual informationcovariable

MSC classification

Primary:62B10: Information-theoretic topics94A17: Measures of information, entropy
Secondary:94A15: Information theory, general94A24: Coding theorems (Shannon theory)

Information

Type
Short Communications
Information
Journal of Applied Probability , Volume 36 , Issue 2 , June 1999, pp. 607 - 610
Copyright
Copyright © Applied Probability Trust 1999 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Ebrahimi, N., and Pellerey, F. (1995). New partial ordering of survival functions based on the notion of uncertainty. J. Appl. Prob. 32, 202–211.CrossRefGoogle Scholar
El Hasnaoui, A. (1993). Le concept du gain d'information: une nouvelle approche en épidémiologie quantitative. Ph.D. Thesis, Université de Montpellier.Google Scholar
Kullback, S. (1959). Information Theory and Statistics. Wiley, New York.Google Scholar
Manuceau, J., Troupé, M., and Vaillant, J. (1999). Information and prognostic value of some variables in the breast cancer. To appear in European Series in Applied and Industrial Mathematics.Google Scholar
Rao, C.R. (1982). Diversity: Its measurement, decomposition, apportionment and analysis. Sankhya Ser. A. 44, 1–22.Google Scholar
Rao, C. R. (1986). Generalization of ANOVA through entropy and cross entropy functions. In Probability Theory and Mathematical Statistics, Vol. 2. Sciences Press, Utrecht, pp. 477–494.Google Scholar
Rényi, A. (1961). On measures of entropy and information. In Proc. 4th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. University of California Press, Berkeley, CA, pp. 547–561.Google Scholar
Robert, C. (1990). An entropy concentration theorem: applications in artificial intelligence and descriptive statistics. J. Appl. Prob. 27, 303–313.Google Scholar
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal 27, 379–423 & 623–656.CrossRefGoogle Scholar