Title: Special Topics (Optimal Filtering for Signal Processing)
Catalog Description: Optimal Filtering for Signal Processing
Review of discrete-time system models and state equations. Bayesian estimation. Linear optimal filters and Kalman filters, grid based filters. Nonlinear filtering and suboptimal nonlinear filters. Markov Chain Monte Carlo methods. Particle Filters, importance sampling, selection of the importance function, resampling. Different types of particle filters. Cramer-Rao bounds for nonlinear filters. Applications of particle filters and some research directions using particle filters.
Coordinator: Ayşin Ertüzün, Professor of Electrical Engineering
Goals: The optimal filters used in signal enhancement and data analysis are developed with the introduction of statistical ideas to filtering problems. The objective of this course is to study the optimal filters first in the data driven context and then in the Bayesain context and to try to unify these concepts for some filters. Bayesian theory will be presented, MCMC and sampling techniques as well as sequential MCMC methods will be studied. Some suboptimum filters will be studied along with the optimal filters.
At the end of this course, students will be able to:
- Use different techniques and algorithms such as MCMC or sequential MCMC techniques for estimation of parameters
- Apply data driven approaches to real life problems.
- Apply Bayesian approaches and particle filtering approach to real life problems.
- Compare results of data driven approaches and Bayesain approaches
- Compute Cramer Rao Lower Bound for nonlienar filters
- D. J.C. Mackay, Information Theory, Inference and Learning Algorithms, Cambridge University Press,2003 or http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
- B. Ristic, S. Arulampalam and N. Gordon, Beyond the Kalman Filter Particle Filters for Tracking Applictions, Artech House, 2004.
- Arnaud Doucet, Nando de Freitas, adn Neil Gordon, Sequential Monte Carlo Methods in Practice
- D. Gamerman and H. F. Lopes, Markov Chain Monte Carlo, Chapman &Hall/CRC (2/e), 2006.
- J.J.K.O. Ruanaidh and W.J. Fitzgerald, Numerical Bayesian Methods Applied to Signal Processing
- Brian D.O. Anderson and John B. Moore, Optimal Filtering, Prentice Hall
- Simon Haykin, Adaptive Filter Theory, Prentice Hall (4/e)
- Christian P. Robert and G. Casella, Monte Carlo Statistical Methods, Springer- Verlag
- Daniel B. Rowe, Multivariate Bayesian Statistics: Models for Source Separation and Signal Umnixing, CRC press
- Wendy L. Martinez, Angel R. Martinez, Computational Statistics Handbook with MATLAB”, CRC Press
- Gilks, Richardson, Spregelhalter , Markov Chain Monte Carlo in Practice, CRC Press
- P. C." Gregory, Bayesian Logical Data Analysis for the Physical Sciences : A Comparative Approach with Mathmatica Support , Cambridge University Press
- Andrew Gelman and John B. Carlin, Hal S. Stern and Donald B. Ruben, Bayesian Data Analysis, Chapman &Hall/CRC(2/e)
- W. M. Bolstad, Introduction to Bayesian Statistics, John Wiley &Sons Inc., 2004.
Prerequisites by Topic:
- Probability and Statistical Signal Analysis
- some knowledge of information theory
- some knowledge of optimization theory
• Introduction (1 hr)
• Wiener Filter theory (2 hr)
• Review of State Space Concepts and Kalman Filter (3 hrs)
• Bayesian Theory (1 hr)
• Deterministic Inference Techniques (1 hr)
• Probabilistic Inference Techniques (2 hrs)
• Gibbs Sampler (1 hr)
• Markov Chain Monte Carlo (2 hr)
• Time Series Analysis (1 hr)
• Revisiting the Kalman Filter in the Bayesian Context (1 hr)
• Grid Based Methods (1 hr)
• Supoptimal Filtering (4 hrs)
o Extended Kalman Filter
o Unscented Kalman Filter
o Gaussian Sum Filter
• Sequential Inference (2 hrs)
• Importance Sampling (1 hr)
• Sequential Monte Carlo Methods (5 hrs)
o Particle Filters
o Versions of Particle Filters
• Cramer-Rao Bounds for Nonlinear Filters (1 hr)
• Applications and research directions for particle filters (4 hrs)
• Project presentations (3 hrs)
Course Structure: The class meets for three 50-minute sessions per week. 5-6 sets of homework problems are assigned per semester. There is one term project for which students write a report and give an oral presentation in class.
Computer Resources: Students are encouraged to use MATLAB to solve their homework problems.
Laboratory Resources: None.
- Homework sets (5%)
- Term Project (45% ).
(a) Apply math, science and engineering knowledge. This course covers the principles and basic optimal and suboptimal filters in frequentist and Bayesian theory. Different tools from probability theory and stochastic signals, information theory, optimization theory and Bayesian theory are heavily used in lectures, homework sets and exams.
(c) Design a system, component or process to meet desired needs. In this course students are equipped with knowledge to design algorithms to apply Wiener filters, Kalman filters and Partcile filters to different problems as well as use their knowledge for solving real life problems.
(d) Ability to function on multi-disciplinary terms. The applications of optimal filters are very vast from communications to biomedical signal processing, from signal processing to text processing to physics, robotics and economics. The theoretical and practical aspects of this course can be used in many multi-disciplinary areas.
(g) Ability to communicate effectively. In this course a term project is to be done by each student on a selected topic related to the course material. The final assessment of the project includes the evaluation of a talk to be given on the selected topic as well as a written report. The presentation aims at effective communication skills.
(j) Knowledge of comtemporary issues. The aim of this course is to study the basic theory and practical issues of optimal filters. Bayesian signal processing and sequential Monte Carlo methods is a very hot and new topic in Signal Processing area and we address papers recently published during the lectures; the topics of the term projects are selected to consider applications and new problems related to these new topics.
(k) Use of modern engineering tools. Students are encouraged to use MATLAB to solve their homework problems.
Prepared By: Ayşın Ertüzün