Optimization: Basis and Non-Derivative Approaches

National Taiwan University of Science and Technology

Department of Electrical Engineering

Summer, 2024

Prerequisite:Basic Engineering Mathematics

Instructor:  Shun-Feng Su,

Office : T2 502-3

Phone: ext 6704

E-mail : sfsu@mail.ntust.edu.tw

Classroom : TBD

Time : (8/16, 17, 18, 19, and 20) Morning

References :  E. K. P. Chong and S. H. Żak, An Introduction to Optimization, 

Classnote: Available on http://intelligence.ee.ntust.edu.tw/su. Please select the course information and the click the nonlinear programming icon to download

Tests : One Midterm and one final.

Optimization: Basis and Non-Derivative Approaches

Course Description 

Optimization is central to many occasions involving decision or finding good solutions in various research problems. In this talk, I shall provide some fundamental concepts and ideas about optimization. This talk will also introduce one group of optimization techniques – non-derivative optimization, like genetic algorithms, ant systems, and particular swarm optimization. Non-Derivative Optimization approaches are to mimic various natural phenomena, like natural selection process or animal behaviors so as to find the best candidate for the problem. Those search processes are to find the next candidates by using experience obtained from previous search together with some random search mechanisms. Some remarks for those approaches are given to provide some ideas for how to implement them.

最佳化是在很多領域中常面對的問題，在決策過程中當面對不同可選擇的可能中，如何選擇最好的是最佳化所要解決的。在這門課中，我們將介紹相關的最佳化理論及方法。基本上，去提供所有的常用可能的最佳化數值技術是不可能，因此，本課程主要在於提供學生充分之知識與概念，以足於當面對任何最佳化問題時知道如何解決。

Tentative Outline

• Introduction of Optimization Problems  
• One-Dimensional Search 
• Gradient Methods 
• Newton’s Method and Its Variations 
• Nonlinear Constrained Optimization
• Non-Derivative Optimization