Object library of algorithms for dynamic optimization problems; Benchmarking SQP and nonlinear interior point methods

• Autorzy: Błaszczyk J. , Karbowski A. , Malinowski K.
• Języki publikacji: EN

Abstrakty

EN

The main purpose of this paper is to describe the design, implementation and possibilities of our object-oriented library of algorithms for dynamic optimization problems. We briefly present library classes for the formulation and manipulation of dynamic optimization problems, and give a general survey of solver classes for unconstrained and constrained optimization. We also demonstrate methods of derivative evaluation that we used, in particular automatic differentiation. Further, we briefly formulate and characterize the class of problems solved by our optimization classes. The solution of dynamic optimization problems with general constraints is performed by transformation into structured large-scale nonlinear programming problems and applying methods for nonlinear optimization. Two main algorithms of solvers for constrained dynamic optimization are presented in detail: the sequential quadratic programming (SQP) exploring the multistage structure of the dynamic optimization problem during the solution of a sequence of quadratic subproblems, and the nonlinear interior-point method implemented in a general-purpose large-scale optimizer IPOPT. At the end, we include a typical numerical example of the application of the constrained solvers to a large-scale discrete-time optimal control problem and we use the performance profiles methodology to compare the efficiency and robustness of different solvers or different options of the same solver. In conclusions, we summarize our experience gathered during the library development.

Słowa kluczowe

EN

dynamic optimization; large-scale optimization; sequential quadratic programming; nonlinear interior-point methods; object-oriented numerical computations; automatic differentiation; performance data analysis

PL

optymalizacja dynamiczna; programowanie kwadratowe sekwencyjne; różniczkowanie automatyczne; obliczenia numeryczne; analiza danych

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2007
• Tom: Vol. 17, no 4
• Strony: 515--537
• Opis fizyczny: Bibliogr. 52 poz., rys., tab., wykr.

Twórcy

autor

Błaszczyk J.

• Institute of Control and Computation Engineering, Faculty of Electronics and Information Technology, Warsaw University of Technology, ul. Nowowiejska 15/19, 00-665 Warsaw, Poland

autor

Karbowski A.

• Institute of Control and Computation Engineering, Faculty of Electronics and Information Technology, Warsaw University of Technology, ul. Nowowiejska 15/19, 00-665 Warsaw, Poland; Research and Academic Computer Network (NASK), ul. Wąwozowa 18, 02-796 Warsaw, Poland

autor

Malinowski K.

• Institute of Control and Computation Engineering, Faculty of Electronics and Information Technology, Warsaw University of Technology, ul. Nowowiejska 15/19, 00-665 Warsaw, Poland; Research and Academic Computer Network (NASK), ul. Wąwozowa 18, 02-796 Warsaw, Poland

Bibliografia

• [1] Arnold E. and Puta H. (1994): An SQP-type solution method for constrained discrete-time optimal control problems. In: Computational Optimal Control (R. Bulirsch and D. Kraft, Eds.), Birkhäuser Verlag, Basel, Switzerland, pp. 127-136.
• [2] Arnold E., Tatjewski P. and Wołochowicz P. (1994): Two methods for large-scale nonlinear optimization and their comparison on a case study of hydropower optimization. Journal of Optimization Theory and Applications, Vol. 81, No. 2, pp. 221-248.
• [3] Benson H. Y., Shanno D. F. and Vanderbei R. J. (2001): Interior-point methods for nonconvex nonlinear programming: Filter methods and merit functions. Technical Report ORFE-00-06, Operations Research and Financial Engineering, Princeton University. Available at http://www.princeton.edu/~rvdb/tex/loqo4/loqo4_4.pdf.
• [4] Benson H. Y., Shanno D. F. and Vanderbei R. J. (2002): A comparative study of large-scale nonlinear optimization algorithms. Technical Report ORFE-01-04, Operations Research and Financial Engineering, Princeton University. Available at http://www.princeton.edu/~rvdb/tex/loqo5/loqo5_5.pdf.
• [5] Bertsekas D. P. (1982): Projected Newton methods for optimization problems with simple constraints. SIAM Journal on Control and Optimization, Vol. 20, No. 2, pp. 221-246.
• [6] Błaszczyk J., Karbowski A. and Malinowski K. (2002a): Object library of algorithms for unconstrained dynamic optimization problems. Proceedings of the 14-th National Conference on Automatic Control (KKA), Vol. I, Zielona Góra, Poland, pp. 451-456.
• [7] Błaszczyk J., Karbowski A. and Malinowski K. (2002b): Object library of algorithms for dynamic optimization problems without constraints or with simple bounds on control. Proceedings of the 8th IEEE International Conference on Methods and Models in Automation and Robotics, Vol. 1, Szczecin, Poland, pp. 257-262.
• [8] Błaszczyk J., Karbowski A. and Malinowski K. (2003): Object library of algorithms for dynamic optimization problems with general constraints. Proceedings of the 9th IEEE International Conference on Methods and Models in Automation and Robotics, Vol. 1, Międzyzdroje, Poland, pp. 271-276.
• [9] Bryson A. E. (1998): Dynamic Optimization. Menlo Park CA: Addison-Wesley, p. 550.
• [10] Byrd R. H., Hribar M. E. and J. Nocedal (1999): An interior point algorithm for large scale nonlinear programming. SIAM Journal on Optimization, Vol. 9, No. 4, pp. 877-900.
• [11] Byrd R. H., Gilbert J. Ch. and Nocedal J. (2000): A trust region method based on interior point techniques for nonlinear programming. Mathematical Programming, Vol. 89, pp. 149-185.
• [12] Chamberlain R. M., Powell M. J. D., Lemarechal C. and Pedersen H. C. (1982): The watchdog technique for forcing convergence in algorithms for constrained optimization. Mathematical Programming Study, Vol. 16, pp. 1-17.
• [13] Dolan E. D. and Moré J. J. (2002): Benchmarking optimization software with performance profiles. Mathematical Programming, Vol. 91, No. 2, pp. 201-213.
• [14] Fiacco A. V. and McCormick G. P. (1968): Nonlinear Programming: Sequential Unconstrained Minimization Techniques. John Wiley and Sons, New York/London.
• [15] Findeisen W., Szymanowski J. and Wierzbicki A. (1980): Theory and Computational Methods of Optimization. Polish Scientific Publishers, Warsaw (in Polish).
• [16] Fletcher R. (1987): Practical Methods of Optimization. JohnWiley and Sons, New York, NY, USA.
• [17] Fletcher R. (1995): An optimal positive definite update for sparse Hessian matrices. SIAM Journal on Optimization, Vol. 5, No. 1, pp. 192-218.
• [18] Fletcher R. and Leyffer S. (2002): Nonlinear programming without a penalty function. Mathematical Programming, Vol. 91, No. 2, pp. 239-269.
• [19] Fletcher R., Leyffer S. and Toint Ph. L. (2006): A brief history of filter methods. Technical Report ANL/MCSP1372-0906, Mathematics and Computer Science Division, Argonne National Laboratory. Available at http://www.optimization-online.org/DB_ FILE/2006/10/1489.pdf.
• [20] Franke R. (1994): Anwendung von Interior-Point-Methoden zur Lösung zeitdiskreter Optimalsteuerungsprobleme. M.S. thesis, Techniche Universität Ilmenau, Fakultät für Informatik und Automatsierung, Institut für Automatisierungsund Systemtechnik Fachgebiet Dynamik und Simulation ökologischer Systeme, Ilmenau, Germany, (in German).
• [21] Franke R. (1998): OMUSES - A tool for the optimization of multistage systems and HQP - A solver for sparse nonlinear optimization. Version 1.5. Department of Automation and Systems Engineering, Technical University of Ilmenau, Germany. Available at ftp://ftp.systemtechnik.tu-ilmenau.de/pub/reports/omuses.ps.gz.
• [22] Franke R. and Arnold E. (1997): Applying new numerical algorithms to the solution of discrete-time optimal control problems. In: Computer-Intensive Methods in Control and Signal Processing: The Curse of Dimensionality, (Warwick K. and Kárný M., Eds.), Birkhäuser Verlag, Basel, Switzerland, pp. 105-118.
• [23] The solver Omuses/HQP for structured large-scale constrained optimization: Algorithm, implementation, and example application. Proceedings of the 6-th SIAM Conference on OPTIMIZATION, Atlanta.
• [24] Goldfarb D. and Idnani A. (1983): A numerically stable dual method for solving strictly convex quadratic programs. Mathematical Programming, Vol. 27, No. 1, pp. 1-33.
• [25] Gondzio J. (1994): Multiple centrality corrections in a primal-dual method for linear programming. Technical Report. 20, Department of Management Studies, University of Geneva, Geneva, Switzerland. Available at http://www.maths.ed.ac.uk/~gondzio/software/correctors.ps.
• [26] Griewank A., Juedes D., Mitev H., Utke J., Vogel O. And Walther A. (1999): ADOL-C: A package for the automatic differentiation of algorithms written in C/C++, Version 1.8.2, March 1999. Available at http://www.math. tu-dresden.de/~adol-c/.
• [27] Karmarkar N. (1984): A new polynomial-time algorithm for linear programming. Combinatorica, Vol. 4, No. 4, pp. 373-395.
• [28] Luus R. (2000): Iterative Dynamic Programming. CRC Press, Inc., Boca Raton, FL, USA.
• [29] Mehrotra S. (1992): On the implementation of a primal-dual interior point method. SIAM Journal on Optimization, Vol. 2, No. 4, pp. 575-601.
• [30] Misc J.-P. (2003): Large scale nonconvex optimization. SIAM's SIAG/OPT Newsletter Views-and-News, Vol. 14, No. 1, pp. 1-25. Available at http://fewcal.uvt.nl/sturm/siagopt/vn14_1.pdf.
• [31] Morales J. L., Nocedal J., Waltz R. A., Liu G. and Goux J.-P. (2001): Assessing the potential of interior methods for nonlinear optimization. Technical Report OTC 2001/4, Optimization Technology Center of Northwestern University. Available at http://www.ece.northwestern.edu/~morales/PSfiles/assess.ps.
• [32] Nocedal J. and Wright S. J. (1999): Numerical Optimization. Berlin: Springer-Verlag.
• [33] De O. Pantoja J. F.A. (1988): Differential dynamic programming and Newton's method. International Journal of Control, Vol. 47, No. 5, pp. 1539-1553.
• [34] Powell M. J. D. (1978): A fast algorithm for nonlinearly constrained optimization calculations. In: Numerical Analysis, Dundee (G. A.Watson, Ed.), Dundee: Springer-Verlag, pp. 144-157.
• [35] Powell M. J. D. (1985): On the quadratic programming algorithm of Goldfarb and Idnani. Mathematical Programming Study, Vol. 25, pp. 46-61.
• [36] Salahi M., Peng J. and Terlaky T. (2005): On Mehrotratype predictor-corrector algorithms. Technical report, Advanced Optimization Lab, Department of Computing and Software, McMaster University, Hamilton, Ontario, Canada. Available at http://www.optimization-online.org/DB_FILE/2005/03/1104.pdf.
• [37] Schittkowski K. (1980): Nonlinear Programming Codes: Information, Tests, Performance. Berlin: Springer-Verlag.
• [38] Schittkowski K. (1983): On the convergence of a sequential quadratic programming method with an augmented Lagrangian line search function. Mathematishe Operations Forschung und Statistik, Ser. Optimization, Vol. 14, No. 2, pp. 197-216.
• [39] Schwartz A. and Polak E. (1997): Family of projected descent methods for optimization problems with simple bounds. Journal of Optimization Theory and Applications, Vol. 92, No. 1, pp. 1-31.
• [40] Tenny M. J., Wright S. J. and Rawlings J. B. (2002): Nonlinear model predictive control via feasibilityperturbed sequential quadratic programming. Technical Report TWMCC-2002-02, Texas-Wisconsin Modeling and Control Consortium. Available at http://jbrwww.che.wisc.edu/jbr-group/tech-reports/twmcc-2002-02.pdf.
• [41] Tits A. L., Wächter A., Bakhtiari S., Urban T. J. and Lawrence C.T. (2002): A primal-dual interior-point method for nonlinear programming with strong global and local convergence properties. Technical Report TR 2002-29, Institute for Systems Research, University of Maryland. Available at http://www.ee.umd.edu/~andre/pdiprev.ps.
• [42] Ulbrich M., Ulbrich S. and Vicente L. N. (2004): A globally convergent primal-dual interior-point filter method for nonlinear programming. Mathematical Programming, Vol. 100, No. 2, pp. 379-410.
• [43] Vanderbei R. J. and Shanno D. F. (1997): An interior-point algorithm for non-convex nonlinear programming. Technical Report SOR-97-21, Statistics and Operations Research, Princeton University. Available at http://www.sor.princeton.edu/~rvdb/ps/nonlin.ps.gz.
• [44] Wächter A. (2002): An Interior Point Algorithm for Large-Scale Nonlinear Optimization with Applications in Process Engineering. Ph.D. dissertation, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA. Available at http://www.research.ibm.com/people/a/andreasw/papers/thesis.pdf.
• [45] Wächter A. and Biegler L. T. (2000): Failure of global convergence for a class of interior point methods for nonlinear programming. Mathematical Programming, Vol. 88, No. 3, pp. 565-574.
• [46] Wächter A. and Biegler L. T. (2005): Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM Journal on Optimization, Vol. 16, No. 1, pp. 1-31.
• [47] Wächter A. and Biegler L. T. (2006): On the implementation of a primal-dual interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, Vol. 106, No. 1, pp. 25-57.
• [48] Waltz R. A. and Plantenga T. (2006): KNITRO 5.0 User's Manual. Available at http://www.ziena.com/docs/knitroman.pdf.
• [49] Wierzbicki A. (1984): Models and Sensitivity of Control Systems. Elsevier, Amsterdam.
• [50] Wright S. J. (1993): Interior point methods for optimal control of discrete time systems. Journal of Optimization Theory and Applications, Vol. 77, No. 1, pp. 161-187.
• [51] Wright S. J. (1997): Primal-Dual Interior-Point Methods. SIAM, Philadelphia, PA.
• [52] Yakowitz S. and Rutherford B. (1984): Computational aspects of discrete-time optimal control. Applied Mathematics and Computation, Vol. 15, No. 1, pp. 29-45.