ECE 6605 - Information Theory

Course information

ECE 6605 Information Theory is a graduate-level class that introduces the mathematical theory of communications. This is primarily a theoretical course, whose objective is to understand the fundamental limits of communication systems. Information-theoretic concepts underlie virtually all modern communication systems, and someexposure to information theory is definitely required for students who want to pursue research in communications. The objectives of the course are

  • to introduce the notion of entropy and information;
  • to discuss the fundamental limits of data compression (source coding).
  • to discuss the fundamental limits of transmission systems (channel coding).

If time permits, we will also cover some topics in multi-user information theory.

Prerequisites

A good understanding of basic probability theory (ECE 3075 or equivalent) is required. Although the course will not review probability theory, it is planned that, throughout the course, relevant background will be revised. A background in communications and signal processing is also helpful.

Textbook

Elements of Information Theory 1st Ed., Tom Cover and Joy Thomas, J.W. Wiley 1991

Course schedule

Tuesdays 5:30pm-7pm, Thursdays 4pm-5:30pm

Location: Red room

Office hours

Fridays, 1pm-2:30pm, Office 214

Grade

  • Participation: 2%
  • Homeworks: 10%
  • Midterm 1: 19%
  • Midterm 2: 19%
  • Final: 50%

Online Resources

Network Coding Theory, Raymond Yeung, Shuo-Yen Robert Li, Ning Cai and Zhen Zhang, Now Publishers, 2006

Information Theory, Inference, and Learning Algorithms, David J. C. MacKay, Cambridge University Press, 2003

Claude Shannon - Father of the Information Age

Claude E. Shannon - The 50th anniversary of information theory [xplore]