Title: Special Topics (Independent Component Analysis)
Catalog Description: Independent Component Analysis
Review of random vectors and independence, gradients and optimization mehtods, estimation theory,information theory, principle component analysis and whitening. Basic independent component analysis (ICA), different interpretations of independent component analysis. ICA by maximization of nongaussianity, ICAby minimizaiton of Mutual Information, ICA by Nonlienar Deccorelation and Nonlinear Principle Component Analysis, Basic ICA Methods, Noisy ICA,Nonlinear ICA,Methods uisng Time Structure, Convolutive Mixtures and Blind Deconvolution, Applications of ICA.
Coordinator: Ayşin Ertüzün, Professor of Electrical Engineering
Goals: The objective of this course is to study Independent Component Analysis(ICA) as a statistical and computational tool. Practical algorithms for computing the independent components and mixing matrices will be studied. Some extensions of ICA for more complicated problems will be elaborated and applications will be discussed.
At the end of this course, students will be able to:
- Compute Independent Components of Mixtures
- Use different techniques and algorithms for computation of independent components
- Apply concepts of ındepedent component analysis to different real life applications
Textbook: A. Hyvarinen, J. Karhunen, E. Oja, Independent Component Analysis, John Wiley and Sons, 2001.
- Mark Girolami (ed.), Advances in Independent Component Analysis, Springer, 2000.
- Simon Haykin (ed.), Unsupervised Adaptive Filtering (Vol. I) Blind Source Separation, John Wiley & Sons Inc
- Te-Won Lee, Independent Component Analysis Theory and Applications, Kluwer Academic Publishers
- A.Chchocki and S. Amari, Adaptive abd Blind Signal and Image Processign Algorithms Learning Algorithms and Applications, John Wiley & Sons Inc., 2002.
- Asoke Kumar Nandi(ed.), Blind Estimation Using Higher Order Statistics, Kluwer Academic Publishers, 1999.
Prerequisites by Topic:
- Probability and Statistical Signal Analysis
- Matrix Algebra
- Numerical Analysis
- Gradients and Optimization Algorithms
1. Introduction and Principle Component Analysis and Whitening and Orthogonalizaiton (1 week)
2. Basic Definition, Restrictions of Independent Component Analysis; Relationship between decorrelation whitening and ICA (1 week)
3. ICA by Maximization of Nongaussianity (2 weeks)
4. ICA by Maximum Likelihood Estimation (1 weeks)
5. ICA by Minimization of Mutual Information (1 week)
6. ICA by Nonlinear Decorrelation and Nonlinear PCA (1 week)
7. Practical Considerations and Comparison of Basic ICA Methods (1 week)
8. Noisy ICA and Applications(1 week)
9. ICA with overcomplete Bases and Applications (1 week)
10. Nonlinear ICA (1 week)
11. Methods Using Time Structure, Convolutive Mixtures and Blind Deconvolution and Applications (1 week)
Course Structure: The class meets for three 50-minute sessions per week. 5-6 sets of homework problems are assigned per semester. There is one term project and a take-home final exam.
Computer Resources: Students are encouraged to use MATLAB to solve their homework problems. They also use packages for ICA such as FastICA and its variations.
Laboratory Resources: None.
- Homework sets (30%)
- Term Project (30% each).
- Take-home final exam (40%).
(a) Apply math, science and engineering knowledge. This course covers the principles and basic algorithms of ICA. Different tools from probability theory and stochastic signals, higher order statistics, information theory and optimization theory are heavily used in lectures, homework sets and exams.
(c) Design a system, component or process to meet desired needs. In this course students are equipped with knowledge to design algorithms to compute independent components as well as use their knowledge for solving real life problems.
(d) Ability to function on multi-disciplinary terms. The applications of ICA are very vast from communications to biomedical signal processing, from signal processing to text processing. The theoretical and practical aspects of this course can be used in many multi-disciplinary areas.
(g) Ability to communicate effectively. In this course a term project is to be done by each student on a selected topic related to the course material. The final assessment of the project includes the evaluation of a talk to be given on the selected topic as well as a written report. The presentation aims at effective communication skills.
(j) Knowledge of comtemporary issues. The aim of this course is to study the basic theory and practical issues of ICA as well as its special forms. ICA is a very hot and new topic in Signal Processing area and we address papers recently published during the lectures; the topics of the term projects are selected to consider applications and new problems related to ICA.
(k) Use of modern engineering tools. Students are encouraged to use MATLAB to solve their homework problems. They also use packages for ICA such as FastICA and its variations. There is at least one computer oriented problems in each homerwork set.
Prepared By: Ayşın Ertüzün