Lecture Notes , Homework Sets , Learn Your Grades , Distribution of Grades

Announcements

  • Final Exam Results Learn your grade, Histogram (Jan. 19)
  • Students who will be taking make-up examination for EE 503 should email me as soon as possible. (Jan 18)
  • Midterm 2 papers are graded. Learn your grade, Histogram
  • Final Exam: 14th Jan, Sunday, 13:30, EA Building
  • Some of the following problems will appear on Final Exam:

    Hayes: 7.6, 7.8, 7.12a
    Therrien : 4.16, 4.18
  • Some of the following problems will appear on 2nd Exam:

    Therrien: 6.2, 6.7 6.12, 6.16a,b , 6.26a
    Hayes: 7.4, 7.9
  • Midterm 1 Results
  • Reading Material: Interested students may examine Stoica's paper on application LMMSE techniques and Matrix Algebra for MIMO capacity. Derivation given is a nice illustration of how far we can go even with basic tools.
  • Hw #5 is assigned (due: 15th Dec)
  • Lecture Notes for Stochastic Modeling are added (Nov 29th Lecture)
  • Midterm Exam #1 will be on Nov 26th Sunday, at 11:00.
  • Therrien: 2.4, 2.23, 2.22, 4.2, 4.23, 5.4, 5.7, 5.27
  • Homework #4 solutions by Mehmet Duzgun
  • Some of the following problems will appear in Midterm 1.
    The total points assigned to these problems is 40 out of 100.

    Hayes: 3.3, 3.4, 3.6, 3.12, 3.13, 3.16, 3.18
    Therrien: 2.4, 2.23, 2.22, 4.2, 4.23, 5.4, 5.7, 5.27

  • Midterm #1: On Nov 26th, Sunday starting at 10:30. (Changed to 11:00)
  • Hw #4: Problems 3.2,3.3,3.4,3.6,3.15,3.26,3.27 from Hayes. (Due: 22th Nov)
  • Check out: AR-MA-ARMA Processes JAVA Demo
  • Check out: JAVA Demos of Signal Processing Techniques
  • Chapter from Therrien on the diagonalization of auto-correlation matrix is added to lecture notes folder (Oct 30)
  • Notes on decorrelating transforms are added
  • HW #3 is posted. (Oct. 13)
  • Random Vectors notes are added. You can read first few pages of Telatar Paper to learn complex vectors -> real vectors mapping.
  • Probability Review Notes are added to Lecture Notes folder.
  • Solutions of HW #1 is posted (Solution by Ozlem Ipek)
  • HW #2 is posted.
  • Lecture notes on linear equation systems are posted.
  • HW #1 is posted.

 

EE 503 Signal Analysis and Processing

Outline in pdf form

 

Short Description:

 

The course aims to unify the knowledge of linear system theory, digital signal processing basics and stochastic processes into the framework of statistical signal processing. The course goal is to establish a firm foundation for estimation theory (parameter estimation, signal modeling), Wiener Filtering (approached from the direction of linear MSE estimation) and linear prediction. Some more advanced topics such as AR, MA, ARMA, Harmonic processes, linear decorrelating transform, series expansion of random processes, spectral factorization, causal – non causal IIR Wiener filters  are also introduced along the path.  

 

 

Outline of Topics:

 

  1. Review of Some Linear Algebra Concepts:
    1. Matrices as Transformations

                                                               i.      Linear Space, Linear Operators in Linear Space

                                                             ii.      Equivalent representations with finite/infinite matrices

                                                            iii.      Isomorphism between finite energy functions and finite power sequences (L2 ó l2 spaces)

                                                           iv.      Representation of points in alternative coordinate systems, representation of operators in alternative coordinate systems

                                                             v.      Diagonalization of operators (Eigenfunctions ó Eigenvectors)

                                                           vi.      Hermitian Operators ó Hermitian Matrices, Orthogonal Bases

                           Ref: Strang, Wolf, Lancaster

    1. Matrices as Linear Combiners

                                                               i.      Range and Null space of the combination process

                                                             ii.      Linear independence of vectors (points in linear space)

                                                            iii.      Projection to Range/Null Space, Direct Sums

                           Ref: Scharf

    1. Matrices as Equation Systems

                                                               i.      Linear constraints (equations), intersection of constraints

                                                             ii.      Under-Over determined systems, Unique-None-Infinite solution systems

                                                            iii.      LS solution for inconsistent equation systems (overdetermined)

1.       Projection to range space,

2.       Pseudo Inverse, SVD

                                                           iv.      Minimum norm solutions for systems with infinite solutions

                                                             v.      SVD and its properties.

                           Ref: Scharf

  1. Review of some DSP Concepts
    1. Basic Idea: Discrete time processing of continuous time signals

                                                               i.      Sampling Theorem (going to discrete time without any loss of information)

                                                             ii.      Bandlimited Interpolation (going back to continous time after processing)

    1. Discrete Time Operations:

                                                               i.      Z-Transform, discrete time LTI systems, convolution, convolution matrices, diagonalization of convolution matrices

  1. Review of some Random Processes Concepts:
    1. Random variables, random vectors (or a sequence of  random variables), random processes
    2. Moment descriptors (especially 2nd order moment description of R.P’s, mean, variance, correlation, auto-correlation, power spectrum density etc.) 
    3. Stationarity, Wide Sense Stationarity
    4. PSD and its properties, spectral factorization
    5. Linear Time Invariant Processing  of  WSS R.P’s
    6. Ergodicity

             Ref: Hayes, Papoulis, Ross

  1. Signal Modeling
    1. LS methods, Pade, Prony (Deterministic methods)
    2. AR, MA, ARMA Processes (Stochastic approach), Yule-Walker Equations, Non-linear set of equations for MA system fit,

                                                               i.      All-pole modeling

1.       Covariance Method

2.       Auto-correlation Method

    1. Harmonic Processes, Wold decomposition
    2. Decorrelating transforms such as Fourier Transforms for Harmonic Processes and KL transform in general.
    3. Applications: Signal Compression, Signal Prediction, System Identification, Spectrum Estimation.

             Ref: Hayes, Papoulis

  1. Some Topics in Estimation Theory
    1. Cost Functions: Mean Square, Mean absolute, max error
    2. MSE, ML, absolute error estimators
    3. Min MSE estimators

                                                               i.      Regression line, orthogonality

    1. Linear min MSE estimators
    2. Linear unbiased min MSE  estimators
    3. Bias, consistency, efficiency, bias-error variance trade-off.
    4. Discussion of LS estimator for Ax=b + n systems.
    5. Wiener Filters as optimal estimators

                                                               i.      Linear predictors defined from Wiener filters

                                                             ii.      Levinson-Durbin recursion for efficient solution of Wiener-Hopf equations.

                                                            iii.      Lattice Structures for efficient implementation of Wiener filters

    1. IIR Wiener Filters

                                                               i.      Non-causal, Causal                    

     Ref: Hayes, Scharf

 

References:

[Hayes] :  M. H. Hayes, Statistical Signal Processing and Modeling, Wiley, New York, NY, 1996 (Available in bookstore; level: moderate)

 

[Scharf] : Louis L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Company, Inc., Reading, MA, 1991.(level : advanced)

 

[Papoulis] : A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd edition, McGraw Hill, 1991. (level: reference book, mostly advanced)

 

[Ross]: S. M. Ross, Introduction to probability models, 7th ed. Harcourt Academic Press, 2000. (level : introductory but complete)

 

[Wolf] : Kurt Bernardo Wolf , Integral Transforms in Science and Engineering

Plenum Pub Corp, January 1979 (level: advanced)

 

[Lancaster]: P. Lancaster and M. Tismenetsky. The Theory of Matrices. Academic Press, Boston, 2nd edition, 1985.  (level: complete text, very valuable as a linear algebra reference)

 

Cagatay Candan, Fall 2006

METU Electrical-Engineering Department