EE457

Title: INTRODUCTION TO OPTIMIZATION THEORY

Credits: 3

Catalog Description: Unimodal search, unconstrained optimization with respect to a single variable, optimization with respect to a multiple variables, constrained optimization, calculus of variations, principles of optimality and dynamic programming, maximum principle, Kuhn-Tucker conditions of optimality
Coordinator: Okyay Kaynak, Professor of Electrical Engineering

Goals: This course is an introduction to optimization theory, both constrained and unconstrained. Both single and multi variable cases are covered. Some hours are reserved for gradient free optimization techniques in the form of genetic algorithms.

Learning Objectives:

At the end of this course, students will be able to:

Understand different optimization techniques.
Find the minimizer of both single and multi variable functions with or without constraints.
Compare the computational load and the convergence performance of different optimization techniques.
Develop a genetic algorithm for a given optimization problem

Textbook:  Edwin K. P. Chong and Stanislaw H. Zak; An Introduction to Optimization, Second Edition, Wiley, 2001

Reference Texts: Chinyere Onwubiko; Introduction to Engineering  Design Optimization, Prentice Hall, 2000

Prerequisites by Topic:

Linear Algebra
Matrix theory

Topics:

1.          Introduction and Review of Math. Preliminaries

2.          Basics of Unconstrained Optimization

3.          One-dimensional Search Methods

5.          Newton Methods

6.          Conjugate Direction Methods, Quasi-Newton Methods

7.          Linear Least Squares

8.          Random Search Algorithms (Simulated Annealing and Genetic Algorithms)

9.          Algorithms for Constrained Problems

10.      Linear Programming Problems,

11.      Simplex Method, Duality

12.      Non-Simplex (Karmakar’s) Methods

13.      Problems with Equality Constraints

14.      Problems with Inequality Constraints – KKT Condition

Course Structure: The class meets for two lectures a week, one consisting of two 50-minute sessions, the other one 50-minute session. The size of the class is usually small and this enables to closely evaluate the progress of the students.

Computer Resources: None

Laboratory Resources: None

1.       2 Mid-term exams (25% each).

2.       A final exam (40%).

3.       Homeworks, in class participation (10%)

Outcome Coverage:

(a) Apply math, science and engineering knowledge. The students learns to use their linear algebra and matrix theory knowledge in driving the equations for the minimizer of a function and finding the minimizer

(c) Design a system or a component to meet desired needs.  The discussions held in class focus on how to choose the best optimization technique for a given problem as well as choosing the best direction and the step size for the technique used (an optimization problem in itself).

(i) Recognize the need for, and an ability to engage in life-long learning. Throughout the course, how fast the technology is changing is stressed, necessitating continual learning. In the face abundance of knowledge, the paradigm shift from “just-in-case teaching” to “just-in-time learning” is explained.

Prepared By: Okyay Kaynak

Boğaziçi Üniversitesi - Elektrik ve Elektronik Mühendisliği Bölümü

34342 - Bebek / İSTANBUL

Tel: +90 212 359 64 14
Fax: +90 212 287 24 65