ACADEMICS
Course Detail

ELE 737 Fundamentals of Information Theory
2016-2017 Summer term information

The course is not open this term

Timing data are obtained using weekly schedule program tables. To make sure whether the course is cancelled or time-shifted for a specific week one should consult the supervisor and/or follow the announcements.

Course definition tables are extracted from the ECTS Course Catalog web site of Hacettepe University (http://ects.hacettepe.edu.tr) in real-time and displayed here. Please check the appropriate page on the original site against any technical problems.

ELE737 - FUNDAMENTALS of INFORMATION THEORY

Course Name Code Semester Theory
(hours/week)
Application
(hours/week)
Credit ECTS
FUNDAMENTALS of INFORMATION THEORY ELE737 Any Semester/Year 3 0 3 10
Prerequisite(s)
Course languageTurkish
Course typeElective 
Mode of DeliveryFace-to-Face 
Learning and teaching strategiesLecture
Question and Answer
Problem Solving
 
Instructor (s)Dr. Emre Aktaž 
Course objectiveThe objective of the course is to introduce ? the notion of entropy and information ? the fundamental limits of data compression ? the fundamental limits of data transmission systems.  
Learning outcomes
  1. Learn and use the main mathematical tools of information theory that quantify and relate information
  2. Learn fundamental limits for systems that store and compress data
  3. Learn fundamental methods of source coding
  4. Learn fundamental limits for systems that communicate data
  5. Utilize information theory in order to gain insight of and design any system that stores, processes, or communicates information
Course ContentIntroduction, review of probability,
Entropy, relative entropy, mutual information, inequalities,
The asymptotic equipartition property,
Data compression,
Channel capacity,
Differential entropy, the Gaussian channel,
Network information theory.
 
ReferencesElements of Information Theory, Cover and Thomas, Wiley Interscience
Gallager, "Claude E. Shannon: A Retrospective on His Life, Work, and Impact", IEEE
Trans. Inform. Theory, vol.47, no.7, Nov. 2001
Wyner, "Fundamental Limits in Information Theory", Proc. of the IEEE, vol.69, no.2,
Feb. 1981
Verdu, "Fifty Years of Shannon Theory", IEEE Trans. Inform. Theory, vol.44, no.6,
Oct. 1998
 

Course outline weekly

WeeksTopics
Week 1Review of probability theory, entropy
Week 2Relative entropy and mutual information
Week 3Jensen?s inequality and its consequences
Week 4Asymptotic equipartition property
Week 5Data compression and Kraft inequality
Week 6Optimal codes, Huffman codes
Week 7Shannon-Fano-Elias coding
Week 8Midterm Exam
Week 9Channel capacity examples
Week 10Channel coding theorem
Week 11Fano?s inequality and the converse to the coding theorem
Week 12Differential entropy
Week 13Gaussian channel
Week 14Network information theory
Week 15Final exam
Week 16Final exam

Assesment methods

Course activitiesNumberPercentage
Attendance00
Laboratory00
Application00
Field activities00
Specific practical training00
Assignments15
Presentation00
Project00
Seminar15
Midterms140
Final exam150
Total100
Percentage of semester activities contributing grade succes050
Percentage of final exam contributing grade succes050
Total100

Workload and ECTS calculation

Activities Number Duration (hour) Total Work Load
Course Duration (x14) 14 3 42
Laboratory 0 0 0
Application000
Specific practical training000
Field activities000
Study Hours Out of Class (Preliminary work, reinforcement, ect)1412168
Presentation / Seminar Preparation000
Project12525
Homework assignment155
Midterms (Study duration)000
Final Exam (Study duration) 12525
Total Workload3170265

Matrix Of The Course Learning Outcomes Versus Program Outcomes

D.9. Key Learning OutcomesContrubition level*
12345
1. Has highest level of knowledge in certain areas of Electrical and Electronics Engineering.    X
2. Has knowledge, skills and and competence to develop novel approaches in science and technology.    X
3. Follows the scientific literature, and the developments in his/her field, critically analyze, synthesize, interpret and apply them effectively in his/her research.   X 
4. Can independently carry out all stages of a novel research project.  X  
5. Designs, plans and manages novel research projects; can lead multidisiplinary projects.  X  
6. Contributes to the science and technology literature.   X 
7. Can present his/her ideas and works in written and oral forms effectively; in Turkish or English.X    
8. Is aware of his/her social responsibilities, evaluates scientific and technological developments with impartiality and ethical responsibility and disseminates them.X    

*1 Lowest, 2 Low, 3 Average, 4 High, 5 Highest

General Information | Course & Exam Schedules | Real-time Course & Classroom Status
Undergraduate Curriculum | Academic Calendar | Open Courses, Sections and Supervisors | Weekly Course Schedule | Examination Schedules | Information for Registration | Prerequisite and Concurrent Courses | Legal Info and Documents for Internship | Information for ELE 401-402 Graduation Project | Program Educational Objectives & Student Outcomes | ECTS Course Catalog | HU Registrar's Office
Graduate Curriculum | Open Courses and Supervisors | Weekly Course Schedule | Final Examinations Schedule | Schedule of Graduate Thesis Defences and Seminars | Information for Registration | ECTS Course Catalog - Master's Degree | ECTS Course Catalog - PhD Degree | HU Graduate School of Science and Engineering