EE 457

Title: INTRODUCTION TO OPTIMIZATION THEORY

Credits: 3

Catalog Description: Unimodal search, unconstrained optimization with respect to a single variable, optimization with respect to a multiple variables, constrained optimization, calculus of variations, principles of optimality and dynamic programming, maximum principle, Kuhn-Tucker conditions of optimality

Coordinator: Okyay Kaynak, Professor of Electrical Engineering

Goals: This course is an introduction to optimization theory, both constrained and unconstrained. Both single and multi variable cases are covered. Some hours are reserved for gradient free optimization techniques in the form of genetic algorithms.

Learning Objectives:

At the end of this course, students will be able to:

1. Understand different optimization techniques.
2. Find the minimizer of both single and multi variable functions with or without constraints.
3. Compare the computational load and the convergence performance of different optimization techniques.
4. Develop a genetic algorithm for a given optimization problem

Textbook: Edwin K. P. Chong and Stanislaw H. Zak; An Introduction to Optimization, Second Edition, Wiley, 2001

Reference Text: Chinyere Onwubiko; Introduction to Engineering  Design Optimization, Prentice Hall, 2000

Prerequisites by Topic:

• Linear Algebra
• Matrix theory

Topics:

1. Introduction and Review of Math. Preliminaries
2. Basics of Unconstrained Optimization
3. One-dimensional Search Methods
4. Gradient Methods
5. Newton Methods
6. Conjugate Direction Methods, Quasi-Newton Methods
7. Linear Least Squares
8. Random Search Algorithms (Simulated Annealing and Genetic Algorithms)
9. Algorithms for Constrained Problems
10. Linear Programming Problems,
11. Simplex Method, Duality
12. Non-Simplex (Karmakar’s) Methods
13. Problems with Equality Constraints
14. Problems with Inequality Constraints – KKT Condition
15. On-Line Adaptive Parameter Tuning

Course Structure: The class meets for two lectures a week, one consisting of two 50-minute sessions, the other one 50-minute session. The size of the class is usually small and this enables to closely evaluate the progress of the students. 

Computer Resources: None

Laboratory Resources: None

Grading:

1. 2 Mid-term exams (25% each).
2. A final exam (40%).
3. Homeworks, in class participation (10%)

• Apply math, science and engineering knowledge. The students learns to use their linear algebra and matrix theory knowledge in driving the equations for the minimizer of a function and finding the minimizer

• Design a system or a component to meet desired needs.  The discussions held in class focus on how to choose the best optimization technique for a given problem as well as choosing the best direction and the step size for the technique used (an optimization problem in itself).

• Recognize the need for, and an ability to engage in life-long learning. Throughout the course, how fast the technology is changing is stressed, necessitating continual learning. In the face abundance of knowledge, the paradigm shift from “just-in-case teaching” to “just-in-time learning” is explained.

Prepared By: Okyay Kaynak

Last Revised:  November 11, 2003