Decomposition Techniques in Mathematical Programming

Decomposition Techniques in Mathematical Programming

Antonio J. Conejo Roberto Mínguez Enrique Castillo Raquel García-Bertrand Decomposition Techniques in Mathematical Programming Engineering and Science Applications ABC

Professor Antonio J. Conejo Universidad de Castilla La Mancha E.T.S. Ingenieros Industriales Avda. Camilo José Cela s/n 13071 Ciudad Real Spain E-mail: antonio.conejo@uclm.es Dr. Roberto Mínguez Universidad de Castilla La Mancha E.T.S. Ingenieros de Caminos Avda. Camilo José Cela s/n 13071 Ciudad Real Spain E-mail: roberto.minguez@uclm.es Professor Enrique Castillo Escuela de Ingenieros de Caminos Universidad de Cantabria Avda. de los Castros s/n 39005 Santander Spain E-mail: castie@unican.es Dr. Raquel García-Bertrand Universidad de Castilla La Mancha E.T.S. Ingenieros Industriales Avda. Camilo José Cela s/n 13071 Ciudad Real Spain E-mail: raquel.garcia@uclm.es Library of Congress Control Number: 2005934995 ISBN-10 3-540-27685-8 Springer Berlin Heidelberg New York ISBN-13 978-3-540-27685-2 Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com c Springer-Verlag Berlin Heidelberg 2006 Printed in The Netherlands The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: by the authors and TechBooks using a Springer LATEX macro package Cover design: design & production GmbH, Heidelberg Printed on acid-free paper SPIN: 11511946 89/TechBooks 543210

To Núria, Mireia Zhen and Olaia Xiao To my family To my parents, my sister, my grandma, and my aunt To JoséAgustín, to my brothers Javier and Jorge Luis, and especially to my parents

Preface Optimization plainly dominates the design, planning, operation, and control of engineering systems. This is a book on optimization that considers particular cases of optimization problems, those with a decomposable structure that can be advantageously exploited. Those decomposable optimization problems are ubiquitous in engineering and science applications. The book considers problems with both complicating constraints and complicating variables, and analyzes linear and nonlinear problems, with and without integer variables. The decomposition techniques analyzed include Dantzig-Wolfe, Benders, Lagrangian relaxation, Augmented Lagrangian decomposition, and others. Heuristic techniques are also considered. Additionally, a comprehensive sensitivity analysis for characterizing the solution of optimization problems is carried out. This material is particularly novel and of high practical interest. This book is built based on many clarifying, illustrative, and computational examples, which facilitate the learning procedure. For the sake of clarity, theoretical concepts and computational algorithms are assembled based on these examples. The results are simplicity, clarity, and easy-learning. We feel that this book is needed by the engineering community that has to tackle complex optimization problems, particularly by practitioners and researchers in Engineering, Operations Research, and Applied Economics. The descriptions of most decomposition techniques are available only in complex and specialized mathematical journals, difficult to understand by engineers. A book describing a wide range of decomposition techniques, emphasizing problem-solving, and appropriately blending theory and application, was not previously available. The book is organized in five parts. Part I, which includes Chapter 1, provides motivating examples and illustrates how optimization problems with decomposable structure are ubiquitous. Part II describes decomposition theory, algorithms, and procedures. Particularly, Chapter 2 and 3 address solution procedures for linear programming problems with complicating constraints and complicating variables, respectively. Chapter 4 reviews and summarizes

VIII Preface duality theory. Chapter 5 describes decomposition techniques appropriate for continuous nonlinear programming problems. Chapter 6 presents decomposition procedures relevant for mixed-integer linear and nonlinear problems. Chapter 7 considers specific decomposition techniques not analyzed in the previous chapters. Part III, which includes Chapter 8, provides a comprehensive treatment of sensitivity analysis. Part IV provides in Chapter 9 some case studies of clear interest for the engineering profession. Part V contains some of the codes in GAMS used throughout the book. Finally, Part VI contains the solutions of the even exercises proposed throughout the book. Relevant features of this book are 1. It provides an appropriate blend of theoretical background and practical applications in engineering and science. 2. Many examples, clarifying, illustrative, and computational, are provided. 3. Applications encompass electrical, mechanical, energy, and civil engineering as well as applied mathematics and applied economics. 4. The theoretical background of the book is deep enough to be of interest to applied mathematicians. 5. Practical applications are developed up to working algorithms that can be readily used. 6. The book includes end-of-chapter exercises and the solutions of the even numbered exercises are given in a Part VI. This makes the book very practical as a textbook for graduate and postgraduate courses. 7. The book addresses decomposition in linear programming, mixed-integer linear programming, nonlinear programming, and mixed-integer nonlinear programming. It provides rigorous decomposition algorithms as well as heuristic ones. Required background to fully understand this book is moderate and includes elementary algebra and calculus, and basic knowledge of linear and nonlinear programming. Over the last two decades, the two senior authors of this book have been involved in research projects that required the solution of large-scale complex optimization problems. We have received advice and relevant observations from many colleagues. We would like to express our appreciation to Prof. Gerald B. Sheblé from Iowa State University, Prof. Mohammad Shahidehpour from Illinois Institute of Technology, Prof. Francisco D. Galiana from McGill University, Prof. Víctor H. Quintana from University of Waterloo, Prof. Francisco J. Prieto from Universidad Carlos III of Madrid, and Prof. Benjamin F. Hobbs from Johns Hopkins University. We are also thankful to quite a few colleagues and former students for suggestions and insightful observations that have improved our book. Particularly, we would like to thank Prof. Steven A. Gabriel from the University of Maryland, and Prof. Bruce F. Wollenberg from the University of Minnesota.

Preface IX We are deeply grateful to the University of Castilla-La Mancha, Spain, for providing us with an outstanding research environment. Ciudad Real and Santander, Spain June 2005 A.J. Conejo E. Castillo R. Mínguez R. García-Bertrand

Contents Part I Motivation and Introduction 1 Motivating Examples... 3 1.1 Motivation... 3 1.2 Introduction... 7 1.3 Linear Programming: Complicating Constraints............ 8 1.3.1 TransnationalSodaCompany... 8 1.3.2 StochasticHydroScheduling... 12 1.3.3 RiverBasinOperation... 19 1.3.4 Energy Production Model........................ 23 1.4 LinearProgramming:ComplicatingVariables... 28 1.4.1 Two-YearCoalandGasProcurement... 28 1.4.2 Capacity Expansion Planning..................... 32 1.4.3 The Water Supply System........................ 36 1.5 Nonlinear Programming: Complicating Constraints.......... 39 1.5.1 ProductionScheduling... 39 1.5.2 Operation of a Multiarea Electricity Network........ 42 1.5.3 TheWallDesign... 45 1.5.4 Reliability-based Optimization of a Rubblemound Breakwater.................... 48 1.6 Nonlinear Programming: Complicating Variables............ 53 1.6.1 Capacity Expansion Planning: Revisited............ 53 1.7 Mixed-Integer Programming: Complicating Constraints...... 55 1.7.1 UnitCommitment... 55 1.8 Mixed-Integer Programming: Complicating Variables........ 57 1.8.1 Capacity Expansion Planning: Revisited 2.......... 57 1.8.2 The Water Supply System: Revisited............... 60 1.9 ConcludingRemarks... 61 1.10 Exercises... 62

XII Contents Part II Decomposition Techniques 2 Linear Programming: Complicating Constraints... 67 2.1 Introduction... 67 2.2 Complicating Constraints: Problem Structure.............. 70 2.3 Decomposition......................................... 73 2.4 The Dantzig-Wolfe Decomposition Algorithm............... 77 2.4.1 Description... 77 2.4.2 Bounds......................................... 87 2.4.3 Issues Related to the Master Problem.............. 88 2.4.4 Alternative Formulation of the Master Problem...... 93 2.5 ConcludingRemarks... 99 2.6 Exercises...100 3 Linear Programming: Complicating Variables...107 3.1 Introduction...107 3.2 Complicating Variables: Problem Structure................ 110 3.3 Benders Decomposition.................................. 111 3.3.1 Description...111 3.3.2 Bounds......................................... 116 3.3.3 The Benders Decomposition Algorithm............. 116 3.3.4 Subproblem Infeasibility.......................... 128 3.4 ConcludingRemarks...135 3.5 Exercises...136 4 Duality...141 4.1 Introduction...141 4.2 Karush Kuhn Tucker First- and Second-Order Optimality Conditions............................................. 142 4.2.1 Equality Constraints and Newton Algorithm........ 147 4.3 DualityinLinearProgramming...149 4.3.1 Obtaining the Dual Problem from a Primal Problem instandardform...150 4.3.2 ObtainingtheDualProblem...151 4.3.3 DualityTheorems...154 4.4 DualityinNonlinearProgramming...161 4.5 Illustration of Duality and Separability.................... 176 4.6 ConcludingRemarks...181 4.7 Exercises...181 5 Decomposition in Nonlinear Programming...187 5.1 Introduction...187 5.2 Complicating Constraints................................ 187 5.3 Lagrangian Relaxation.................................. 187

Contents XIII 5.3.1 Decomposition.................................. 188 5.3.2 Algorithm...................................... 194 5.3.3 Dual Infeasibility................................ 195 5.3.4 Multiplier Updating.............................. 195 5.4 Augmented Lagrangian Decomposition.................... 205 5.4.1 Decomposition.................................. 205 5.4.2 Algorithm...................................... 207 5.4.3 Separability..................................... 208 5.4.4 Multiplier Updating.............................. 208 5.4.5 Penalty Parameter Updating...................... 208 5.5 Optimality Condition Decomposition (OCD)............... 210 5.5.1 Motivation: Modified Lagrangian Relaxation........ 211 5.5.2 Decomposition Structure......................... 213 5.5.3 Decomposition.................................. 214 5.5.4 Algorithm...................................... 216 5.5.5 Convergence Properties........................... 217 5.6 Complicating Variables.................................. 223 5.6.1 Introduction...223 5.6.2 Benders Decomposition........................... 223 5.6.3 Algorithm...................................... 225 5.7 From Lagrangian Relaxation to Dantzig-Wolfe Decomposition 233 5.7.1 Lagrangian Relaxation in LP...................... 234 5.7.2 Dantzig-Wolfe from Lagrangian Relaxation......... 236 5.8 ConcludingRemarks...238 5.9 Exercises...239 6 Decomposition in Mixed-Integer Programming...243 6.1 Introduction...243 6.2 Mixed-Integer Linear Programming....................... 244 6.2.1 The Benders Decomposition for MILP Problems..... 245 6.2.2 Convergence...250 6.3 Mixed-Integer Nonlinear Programming.................... 251 6.4 ComplicatingVariables:NonlinearCase...251 6.4.1 The Benders Decomposition...................... 251 6.4.2 Subproblem Infeasibility.......................... 253 6.4.3 Convergence...257 6.5 Complicating Constraints: Nonlinear Case................. 257 6.5.1 Outer Linearization Algorithm.................... 258 6.5.2 Convergence...264 6.6 ConcludingRemarks...264 6.7 Exercises...264

XIV Contents 7 Other Decomposition Techniques...271 7.1 Bilevel Decomposition................................... 271 7.1.1 ARelaxationMethod...272 7.1.2 The Cutting Hyperplane Method.................. 277 7.2 BilevelProgramming...280 7.3 Equilibrium Problems................................... 282 7.4 Coordinate Descent Decomposition....................... 285 7.4.1 Banded Matrix Structure Problems................ 287 7.5 Exercises...297 Part III Local Sensitivity Analysis 8 Local Sensitivity Analysis...303 8.1 Introduction...303 8.2 StatementoftheProblem...304 8.3 Sensitivities Based on Duality Theory..................... 305 8.3.1 Karush Kuhn Tucker Conditions.................. 305 8.3.2 Obtaining the Set of All Dual Variable Values....... 307 8.3.3 Some Sensitivities of the Objective Function........ 308 8.3.4 A Practical Method for the Sensitivities oftheobjectivefunction...310 8.3.5 A General Formula for the Sensitivities oftheobjectivefunction...310 8.4 A General Method for Obtaining All Sensitivities........... 315 8.4.1 Determining the Set of All Feasible Perturbations.... 317 8.4.2 Discussion of Directional and Partial Derivatives..... 318 8.4.3 Determining Directional Derivatives if They Exist... 320 8.4.4 Partial Derivatives............................... 320 8.4.5 Obtaining All Sensitivities at Once................ 321 8.5 ParticularCases...321 8.5.1 NoConstraints...321 8.5.2 SameActiveConstraints...323 8.5.3 TheGeneralCase...326 8.6 Sensitivities of Active Constraints......................... 339 8.7 Exercises...341 Part IV Applications 9 Applications...349 9.1 TheWallDesign...349 9.1.1 Method 1: Updating Safety Factor Bounds.......... 355 9.1.2 Method 2: Using Cutting Planes................... 359 9.2 TheBridgeCraneDesign...361

Contents XV 9.2.1 Obtaining Relevant Constraints................... 364 9.2.2 ANumericalExample...365 9.3 NetworkConstrainedUnitCommitment...368 9.3.1 Introduction...368 9.3.2 Notation...369 9.3.3 ProblemFormulation...370 9.3.4 SolutionApproach...371 9.4 ProductionCosting...374 9.4.1 Introduction...374 9.4.2 Notation...375 9.4.3 ProblemFormulation...376 9.4.4 SolutionApproach...377 9.5 HydrothermalCoordination...381 9.5.1 Introduction...381 9.5.2 Notation...382 9.5.3 ProblemFormulation...383 9.5.4 SolutionApproach...384 9.6 Multiarea Optimal Power Flow........................... 385 9.6.1 Introduction...385 9.6.2 Notation...386 9.6.3 ProblemFormulation...387 9.6.4 SolutionApproach...389 9.7 Sensitivity in Regression Models.......................... 389 Part V Computer Codes A Some GAMS Implementations...397 A.1 Dantzig-Wolfe Algorithm................................ 397 A.2 Benders Decomposition Algorithm........................ 403 A.3 GAMS Code for the Rubblemound Breakwater Example..... 407 A.4 GAMSCodefortheWallProblem...410 A.4.1 TheRelaxationMethod...410 A.4.2 The Cutting Hyperplanes Method................. 414 Part VI Solution to Selected Exercises B Exercise Solutions...421 B.1 ExercisesfromChapter1...421 B.2 ExercisesfromChapter2...426 B.3 ExercisesfromChapter3...435 B.4 ExercisesfromChapter4...441 B.5 ExercisesfromChapter5...451 B.6 ExercisesfromChapter6...475

XVI Contents B.7 ExercisesfromChapter7...500 B.8 ExercisesfromChapter8...506 References...531 Index...537