Information Theory

This monograph explores the interdisciplinary applications of information theory, focusing on the concepts of entropy, mutual information, and their implications in various fields. It explains the fundamental differences between entropy and Shannon's Measure of Information (SMI), presents the application of information theory to living systems and psychology, and also discusses the role of entropy in art. It critically overviews the definition of correlations and multivariate mutual information.These notions are used to build a new perspective for understanding the irreversibility of processes in macroscopic systems, while the dynamical laws governing the microscopic components are reversible. It also delves into the use of mutual information in linguistics, cryptography, steganography, and communication systems. The book details the theoretical and practical aspects of information theory across a spectrum of disciplines and is a useful tool for any scientist interested in what is usually called entropy.

Autorentext

Arieh Ben-Naim is a professor emeritus at the Department of Physical Chemistry, at the Hebrew University of Jerusalem, Israel. Born in Jerusalem, Israel. Studied and worked for his PhD at the Hebrew University of Jerusalem. Post-Doctoral research in Stony-Brook University, NY, and Bell-Telephone Laboratories, NJ. Most of his research work was focused on the theory of water, aqueous solutions, solvation thermodynamics, and the role of water in biochemical processes. He is the author of over 200 articles and 35 books. Recently the author dedicated his time explaining and clarifying to the general public the most mysterious concept in physics: Entropy. He has published over 20 books and articles on Entropy and the Second Law, the most popular of which is "Entropy Demystified," first published in 2006. He has written three books on Information Theory and its relation to Entropy and the Second Law. He had visited many universities around the world, where he lectured on all the topics of his research and interest.

Claude Dufour, born in Brussels in 1950, took an early interest in heat engines. His thesis on the irreversibility of dense gases at MONS University highlighted the link between the subadditivity of entropy and its tendency to increase. From 1982 to 2009, he taught physics and mathematics. One of his proudest achievements has been to enable the weakest students to succeed while maintaining a high standard, thanks to accessible explanations and the requirement of sustained hard work. Retired since 2009, his goal is to publish the main results of his doctoral research in simple terms. His application of information theory not only to entropy but also to correlations provides a simple and innovative explanation of the second law of thermodynamics.