Information Theory

This course is offered through Coursera — you can add it to your Accredible profile to organize your learning, find others learning the same thing and to showcase evidence of your learning on your CV with Accredible's export features.

Course Date: 08 September 2014 to 12 January 2015 (18 weeks)

Price: free

Course Summary

This course is an introduction to information theory, which emphasizes fundamental concepts as well as analytical techniques. Specific topics include: Information Measures, The I-Measure, Zero-Error Data Compression, Weak Typicality, Strong Typicality, Discrete Memoryless Channels, etc.

Estimated Workload: 6-8 hours/week

Course Instructors

Raymond W. Yeung

Prof. Yeung received the BS, MEng and PhD degrees in electrical engineering from Cornell University in 1984, 1985, and 1988, respectively. He joined AT&T Bell Laboratories in 1988. He came to CUHK in 1991 and has been with the Department of Information Engineering since then, where he is currently the Choh-Ming Li Professor of Information Engineering. Since 2010, he has been serving as Co-Director of the Institute of Network Coding. He is the author of the books A First Course in Information Theory (Kluwer Academic/Plenum Publishers, 2002) and Information Theory and Network Coding (Springer 2008), which have been adopted by over 60 institutions around the world. His research interest is in information theory and network coding. He was a consultant in a project of Jet Propulsion Laboratory for salvaging the malfunctioning Galileo Spacecraft.
Prof. Yeung is a member of the Board of Governors of the IEEE Information Theory Society from 1999 to 2001. He has served on the committees of a number of information theory symposiums and workshops. He was the General Chair of the First Workshop on Network, Coding, and Applications (NetCod 2005), a Technical Co-Chair of the 2006 IEEE International Symposium on Information Theory, and a Technical Co-Chair of the 2006 IEEE Information Theory Workshop, Chengdu. He will organize with David Tse the 2015 IEEE International Symposium on Information Theory in Hong Kong.
Prof. Yeung also has served on the editorial board of a number of academic journals. He was an Associate Editor for Shannon Theory of the IEEE Transactions on Information Theory from 2002 to 2005. He currently serves as an Editor-at-Large of Communications in Information and Systems, an Editor of Foundation and Trends in Communications and Information Theory and an Editor of Foundation and Trends in Networking. He was a recipient of the Croucher Senior Research Fellowship for 2000/01, the Best Paper Award (Communication Theory) of the 2004 International Conference on Communications, Circuits and System, the 2005 IEEE Information Theory Society Paper Award, and the Friedrich Wilhelm Bessel Research Award from the Alexander von Humboldt Foundation in 2007.
Prof. Yeung has been a Changjiang Chair Professor of Xidian University and an Advisory Professor of Beijing University of Posts and Telecommunications. He is a Fellow of the IEEE and the Hong Kong Institution of Engineers.

Course Description

The lectures are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008).
  • Chapter 1 The Science of Information 
  • Chapter 2 Information Measures 
  • Chapter 3 The I-Measure 
  • Chapter 4 Zero-Error Data Compression 
  • Chapter 5 Weak Typicality 
  • Chapter 6 Strong Typicality 
  • Chapter 7 Discrete Memoryless Channels 
  • Chapter 8 Rate-Distortion Theory 
  • Chapter 9 The Blahut-Arimoto Algorithms 
  • Chapter 10 Differential Entropy 
  • Chapter 11 Continuous-Valued Channels
This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text. The electronic version of the 2008 book can be downloaded for free (with institutional subscription).

At the completion of this course, the student should be able to:
  1. Demonstrate knowledge and understanding of the fundamentals of information theory.
  2. Appreciate the notion of fundamental limits in communication systems and more generally all systems.
  3. Develop deeper understanding of communication systems.
  4. Apply the concepts of information theory to various disciplines in information science.


The lectures are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008) listed above.


There will be 15 weeks of lectures, each composed of video clips of around 5-20 minutes with a total length equal to 50 minutes (equivalent to about 2 to 3 hours of classroom teaching). There will be regular homework assignments for you to check your understanding of the lecture videos.

Suggested Reading

Other than the textbook for this course, an excellent reference is Elements of Information Theory, 2nd Edition by T. M. Cover and J. A. Thomas (Wiley 2006).

Course Workload

6-8 hours/week

Review course:

Please sign in to review this course.

Similar Courses

{{ }} {{ }}


{{course.start_date | date:'MMM d'}} — {{ course.end_date | date:'MMM d'}}   ({{ course.time_until_course_starts }} ,   length: {{ course.length_in_weeks }} weeks) Self-paced — no deadlines    
${{ course.price }} p/mfree


Course Activity & Community

Be the first Accredible user to join this course!

uploaded {{ feed_item.model.caption || feed_item.model.url || feed_item.model.file_file_name }} for the course {{ }} — {{ feed_item.time_ago }}

{{ }} {{ comment.text | truncate: (comment.length || comment_display_length) }}   read more hide

{{ comment.time_ago }}

started the course {{ }} — {{ feed_item.time_ago }}
followed {{ }} — {{ feed_item.time_ago }}
followed thier friend {{ }} — {{ feed_item.time_ago }}
{{ feed_item.model.text }} (on the course {{ }}) — {{ feed_item.time_ago }}

{{ }} {{ comment.text | truncate: (comment.length || comment_display_length) }}   read more hide

{{ comment.time_ago }}