Introduction to Information Theory for Mathematics Students

Math 523 - Fall 2017

Section information: Classes meet on Mondays, Wednesdays and Fridays from 3:00PM to 4:00PM at Busch Hall 202. Office hours are on Tu-Th from 12:00PM to 2:00PM. (This may change later.)

Subject: Despite the title, non-math majors are welcome. The course will cover the main mathematical results comprising Shannon's theory of information, with an emphasis on the stochastic processes aspects of the theory. In addition to developing the fundamentals of the classical theory in detail, we'll study the beginnings of quantum probability and coding theorems. Although information theory is an eminently applied topic, our focus will be on the basic mathematical structure rather than on more technical issues that you would learn in an engineering course on the subject.

Text: Coding Theorems of Classical and Quantum Information Theory (Second Edition), by K.R. Parthasarathy. Texts and Readings in Mathematics, 45. Hindustan Book Agency, 2013.

Tentative list of topics:

• Review of essential facts in probability and measure theory (particularly conditional expectation);
• Entropy theory (entropy of elementary information sources and, more generally, measure-preserving transformations); conditional entropy, mutual information, etc.;
• Source coding, information compression, the Huffman code;
• The ergodic theorem, the Shannon-McMillan-Breiman theorem, the noiseless coding theorem;
• Communication in the presence of noise, information capacity of stationary channels, conditions for ergodicity;
• Elements of quantum probability;
• Quantum information sources, von Neumann entropy, elementary classical-quantum communication channels.

Coursework: Your work will consist entirely of homework assignments. I expect we'll have approximately 5 assignments. You are welcome to collaborate on them, discuss them with me or others, and bring up homework related questions for discussion in class before turning them in. The method for computing grades will be discussed in class. A presentation on a topic related to subject of the course may substitute for a few of the written assignments.

Grades: Your final grade will be based on your homework assignments and/or class presentation.