This course gives an introduction to information theory and its most important applications to communications. The topic is mathematically oriented. The basic concepts of entropy, relative entropy, and mutual information are defined, and their connections to channel capacity, coding, and data compression are presented. In addition to limits for error-free communication, information theory also presents limits for data compression. Whereas coding methods for error control are discussed in the companion course S-72.341, data compression methods are here discussed in more detail, including Huffman, Lempel-Ziv, and Shannon coding. The course book provides a variety of interesting areas of application outside communications, including gambling and investment (stock market).

The teaching language is English. (P = may be included in postgraduate studies.)

**Learning objectives:**Upon completion of the course, the students will be able to- define and apply the basic concepts of information theory (entropy, etc.);
- differentiate between lossy and lossless data compression methods, and describe the most common such methods;
- design an efficient data compression scheme for a given information source;
- calculate the capacity of communication channels;
- sketch Shannon's proof regarding the limits of error-free communication; and
- explain the impact of feedback and/or many senders or receivers on the communication problem.

**Prerequisites:**Basic courses in mathematics.**Literature:**T. M. Cover and J. A. Thomas,*Elements of Information Theory*, Wiley, New York, 1991.**Lectures:**Thursdays 9-12, room H402, first lecture: 16.9.2004 (no lectures on 21.10, 11.11). Slides: Lecture 1. Old slides: Lecture 2, Lecture 3, Lecture 4, Lecture 5, Lecture 6, Lecture 7, Lecture 8, Lecture 9, Lecture 10, Lecture 11, Lecture 12, Lecture 13.**Teacher:**Prof. Patric Östergård, room I447B, tel. 451 2341, e-mail: Patric.OstergardQhut.fi (Q=@)**Tutorials:**Wednesdays 10-12, room F201, first tutorial: 22.9.2004. Tutorial 1 (ps) (pdf) Tutorial 2 (ps) (pdf) Tutorial 3 (ps) (pdf) Tutorial 4 (ps) (pdf) Tutorial 5 (ps) (pdf) Tutorial 6 (ps) (pdf) Tutorial 7 (ps) (pdf) Tutorial 8 (ps) (pdf) Tutorial 9 (ps) (pdf) Tutorial 10 (ps) (pdf)**Assistant:**Petri Lindroos, M.Sc., room E206B, tel. 451 6221, e-mail: pjlindroQhut.fi (Q=@)**To pass the course:**Project, three home assignments, home exam.

If you have any questions, do not hesitate to contact the teacher (or the assistant).

Last update: September 16, 2004.

.