ECE735 - Spring 2005


Professor Yariv Ephraim

STII Room 221



Time: Wednesday 4:30-7:10 pm


Place: Robinson Hall A123


Final Exam: Take home exam to be assigned on Wed 4 May and due on Wed 11 May by 4:30pm

Office Hours: Tuesday 3:00-4:00 pm

Wednesday 3:00-4:00 pm

Other time by appointment


To contact me please use


Course Description:


Data compression is a rapidly growing field with numerous applications. It is fundamental to every aspect of digital signal processing since signals must be quantized before they can be stored in or transmitted over digital media. In today's information technology, speech, image and video signals are quantized when presented over the World Wide Web. Speech and audio coding is essential for cellular communication where bandwidth is limited. Data compression is also receiving extensive attention in storing and analyzing medical images such as X-ray images and tomography. The purpose of this course is to provide a solid theoretical and practical background in data compression and its application. Some concepts from information theory will be reviewed, but prior knowledge is not assumed.


Course Outline:


  • Introduction (week 1)
    • Canonical communication system
    • Achievable rates in lossless coding
    • Distortion and rate
    • Quantization

        Scalar quantization (weeks 2)

o       Structure

    • Necessary conditions for optimality
    • Quantizer design
    • Uniform quantization
    • High resolution quantization
  • Predictive quantization (weeks 3-4)
    • Linear prediction
    • Differential pulse code modulation (DPCM)
    • Delta modulator
  • Transform coding (weeks 5-6)
    • Optimal bit allocation
    • Karhunen-Loeve transform
    • Subband coding
    • Performance
  • Lossless (Entropy) Coding (weeks 7-8)
    • Shannon's lossless coding theorem
    • Prefix codes
    • Kraft inequality
    • Huffman code
    • Arithmetic code
    • Lempel-Ziv code
    • Quantization and entropy coding
  • Mid-term exam (take home) (week 9)
  • Vector quantization (weeks 10-12)
    • Motivation
    • Necessary conditions for optimality
    • The Lloyd algorithm
    • Gain/shape quantization
    • Autoregressive Vector Quantization
    • Recursive and finite-state quantization
  • Lattice quantizaters (week 13)
  • Applications (week 14)
    • Speech, audio and image coding
    • Density estimation

Text Book:


A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Kluwer Academic Publishers, Boston, 1995.



Other References:


1.      T. M. Cover and J. A. Thomas, Elements of Information Theory. John Wiley&Sons, Inc., New York, 1991.

2.      Lecture notes by R. M. Gray

3.      Papers from current literature




ECE-528 or instructor permission




We will communicate via email. Please use Announcements, assignments and solutions will be emailed to you. I will use your email addresses which are on file at the GMU Registrar. If you wish to have your course material delivered to another email address, you may include a .forward command in your GMU directory.




There will be 3-5 homework assignments (10 %)


There will be a mid term and a final exam (each 45%). Both are take-home exams.