Title: STATISTICAL SIGNAL ANALYSIS
Catalog Description: Characteristics of random processes. Correlation function and power spectral density of stationary processes. Noise mechanisms, the Gaussian and Poisson processes. Statistical estimation theory, linear mean square filtering, optimum Wiener and Kalman filtering. Signal detection theory and statistical significance tests.
Coordinator: F. Kerem HARMANCI, Assistant Professor
Goals: Refresh the notions of probability, random variables and processes. Analyze the properties and classes of random processes. Introduce useful random process models such as discrete linear models, Gauss-Markov processes, Poisson processes, martingales. Discuss the concept of optimal linear filtering (Wiener, Kalman) with applications. Give basics of measurement noise. Explain basics of detection and estimation.
At the end of this course, students will be able to:
1. Have a deeper understanding of the theory and relevance of random variables and processes.
2. Be able to analyze statistical models of systems and signals.
3. Be capable of reasoning out statistical modeling tasks
4. Be capable of carrying out statistical filtering algorithms
Textbook: K.S. Shanmugan, A.M. Breipohl, Random Signals, Wiley, 1988.
1) H. Stark, J.W. Woods, Probability and Random Processes with Applications to Signal Processing, 3rd Ed. Prentice Hall 2002
2) A. Papoulis, S. U. Pillai, Probability, Random Variables and Stochastic Processes, 4th ed., McGraw Hill 2002
3) M.H. Hayes, Statistical Digital Signal Processing and Modeling, Wiley 1998.
4) C.W. Therrien, Discrete Random Signals and Statistical Signal Processing, Prentice-Hall, 1996.
Prerequisites by Topic:
- Signals and system
- Probability theory
1. Review of probability: Sample space. Axioms and laws of probability. Orderings, combinations and permutations. Joint, marginal and conditional probability distributions. (one week)
2. Random variables: Discrete random variables and probability mass function. Continuous random variables and probability distribution functions. Geometric, Pascal, multinomial, Gauss, exponential, gamma, chi-square distributions. (two weeks)
3. Random vectors: Multivariate distributions. Properties of multivariate Gaussian distribution. Complex random variables. Functions of vector random variables. Characteristic functions and moment generators. (one week)
4. Random processes: Concept and definition of a random process. Classification of random processes. System theory and random processes. (one week)
5. Descriptions of random processes: Joint distributions and analytic description. Independent increment processes. Random walk, Gaussian process, Poisson process. (two weeks)
6. Properties of random processes: Autocorrelation, cross-correlation, power spectral density, cross-power spectral density, coherence function. Ergodicity and stationarity. (two weeks)
7. Special random processes: Time-series models: AR, MA, ARMA processes. Markov sequences and processes. Gaussian process. Point processes: Poisson process, Shot noise. (two weeks)
8. Linear minimum-mean square error filtering: Scalar and vector linear minimum mean square estimator. Optimality and limitations of linear estimators. The concept of innovations. (one week)
9. Wiener filters: Digital Wiener filtering with stored data and with real-time data. Realizable and unrealizable solutions. (one week)
10. Binary detection with single and possibly multiple observations. Neyman-Pearson Lemma and likelihood ratio testing. Bayes decision rule. (one week)
11. Kalman filters: Innovations. Recursive estimators. Scalar Kalman filter. Vector Kalman filter. (Time permitting)
Course Structure: The class meets for three lectures a week, each consisting of two 50-minute sessions. 9-10 sets of homework problems and additional MATLAB homeworks are assigned per semester. There are two in-class mid-term exams and a final exam.
Computer Resources: Students use MATLAB in some of their homeworks.
Laboratory Resources: None.
1. Homework sets (20%)
2. Two mid-term exams (25% each).
3. A final exam (30%).
(a) Apply math, science and engineering knowledge. This course requires probability theory, linear system theory, vector-matrix algebra. Engineering intuition and commonsense is required to translate signal processing instances for the proper application of statistical tools.
(b) Design a system, component or process to meet desired needs. The students have to design a few statistical filtering algorithms.
(c) Use of modern engineering tools. Students use Matlab in this class.
Prepared By: F. Kerem HARMANCI