English Türkçe için tıklayın
Chairman's Welcome Message Information
 Lisans Öğrencileri Lisansüstü Adayları

Title:  Introduction to Information Theory

Credits: 3


Catalog Description: EE 478 Introduction to Information Theory (3+0+0)3

Information Content, conditional, joint and mutual entropy. Binary symmetric Channels;channels with and without memory. Source coding algorithms and rate-distortion bounds. Channel capacity and Shannon law. Block code, cyclic codes, convolution codes.

Coordinator: Ayşin Ertüzün, Professor of Electrical Engineering


Goals:   The objective of this course is to discuss fundamental concepts and limits in information theory. It starts with basic concepts of information theory such as information content, entroy and contiues to discuss Shannon’s theorems and the rate distortion  theory, to illustrate the role of coding for efficient and reliable communication, to give methods to construct good codes.


Learning Objectives:

At the end of this course, students will be able to:

1.    Compute information content and entropy of different sources

2.    Design source codes

3.    Design error correcting and detecting codes

4.    Commute  channel capacity of various channels


     Jan C. A. van der Lubbe, Information Theory, Cambridge Universty Press, 1997.


 Reference Texts:

1.    Thomas M. Cover and Joy A. Thomas, “Elements of Information Theory”, John Wiley and Sons, Inc. 1991.


2.    Robert M. Gray, “Entropy and Information Theory”, Springer –Verlag, 1990.


3.    Robert Ash, “Information Theory”, Dover Publications


4.    R. Galleger, “Information Theory and Reliable Communication”, John Wiley and Sons


5.    Abramson, ”Information Theory and Coding,”


6.    Masud Mansuripur , “Introduction to Information Theory”, Prentice-Hall Inc.


Prerequisites by Topic:



1.    Probability






1.    Basic Concepts in Information Theory  (2 weeks)    


2.    Variable-Length Source Coding  (2 week)        


3.    Information Channels and Channel capacity  (3 weeks)        


4.    Rate  Distortion Theory  (2 weeks)    


5.    Error-Correcting Codes


•    Linear Block Codes  (1 week)


•    Cyclic Codes (1 week)


•    Convolutional Codes (1 week)




Course Structure: The class meets for three 50-minute sessions per week. 5-6 sets of homework problems are assigned per semester.  There are two in-class mid-term exams and a final exam.


Computer Resources:


Laboratory Resources: None.






1.    Homework sets (10%)


2.    Two Midterms (25% each).


3.    Final exam (40%).



Outcome Coverage:



(a) Apply math, science and engineering knowledge.  This course is covers the principles of information theory and coding.  Basic concepts of information theory, Shannon’s law, source and channel coding, channel capacity are heavily emphasized in lectures, homework sets and exams.


 (c) Design a system, component or process to meet desired needs.  In this course students are equipped with knowledge to design source codes  and or channel code to achieve the desired needs.




Prepared By: Ayşın Ertüzün



Boğaziçi Üniversitesi - Elektrik ve Elektronik Mühendisliği Bölümü

34342 - Bebek / İSTANBUL

Tel: +90 212 359 64 14
Fax: +90 212 287 24 65







Copyright 2009 by Bogazici University EE Dept.    |   Privacy Statement   |   Terms Of Use     |     Login