A computational study of on-demand accuracy level decomposition for twostage stochastic programs Christian Wolf, Universität Paderborn, Germany Csaba I. Fábián, Kecskemét College, Hungary Achim Koberstein, Goethe Universität Frankfurt am Main, Germany Leena Suhl, Universität Paderborn, Germany 2013-07-10
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 2 Agenda Introduction On-Demand Accuracy Implementation and Results Conclusion
Introduction Problems in the real world face uncertainty Stochastic programming can be used to deal with uncertainty in the input data for linear or mixed-integer problems Specialized algorithms are needed to solve stochastic programming problems efficiently Widely used algorithm is the L-shaped method On-demand accuracy concept recently developed by Oliveira and Sagastizábal (2012) for level bundle methods Fábián (2013) proposed a special non-proximal variant We implemented this on-demand accuracy variant in a state-of-the-art L- shaped solver Use all computed information in a computationally efficient way A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 3
Two-Stage Stochastic Linear Program with Recourse (SLPR) A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 4
Deterministic Equivalent Formulation A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 5
L-shaped method Decomposition method to solve two-stage stochastic programs L-shaped method by Van Slyke and Wets [1969] Based upon Benders Decomposition [1962] Approximation of the recourse function by outer linearization Division by stage Master problem with recourse function approximation Second stage subproblems to evaluate recourse function for given solution Multi-Stage problems can be solved by nested application of L-shaped method (Birge 1985) Several implementations of Nested L-shaped method exist We extend our parallel nested L-shaped method implementation (W. and Koberstein 2013) A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 6
Master problem with optimality and feasibility cuts Need primal solution, to build right hand side of second-stage problems Warm start of the problem is possible for simplex algorithm, even if cuts are added ( solution stays dual feasible ) A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 7
Second-Stage subproblem A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 8
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 9 Dual formulation of subproblem Need dual solution to form cuts
Optimality cut generation for scenarios A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 10
Cut Aggregation Partition the set of scenarios into aggregates Single cut = One partition with all descendants Multi cut = #descendants many partitions with one descendant in each Hybrid Cut = More partitions than single, less than multi cut (Trukhanov 2010) We only consider single cut for now A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 11
Cut generation for an arbitrary level of aggregates A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 12
Level decomposition Regularization technique to combat zigzagging of first-stage solutions Adaption of the level method (Lemaréchal 1995) to solve two-stage stochastic programming problems (Fábián and Szőke 2007) Project the current iterate to the current level set A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 13
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 14 Agenda Introduction On-Demand Accuracy Implementation and Results Conclusion
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 15 Cut proliferation can be a problem
On-demand accuracy Store all multi-cut information, do not throw it away after aggregation Generate a new single cut when needed out of this information Saves computational time : no need to solve all subproblems just to get another cut Reduces cut proliferation: only one cut is added at a time Concept was recently proposed by Oliveira and Sagastizábal (2012) in the context of level bundle methods How do we generate such a cut? When do we generate such a cut? A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 16
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 17 How to generate an on-demand cut? Evaluate stored cut components for current solution Equivalent to evaluating current first stage solution for given dual second stage basic solution => valid recourse function approximation Find largest recourse function approximation for current solution
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 18 How to generate an on-demand cut? Recourse function approximation Can generate corresponding cuts out of stored information
When to generate an on-demand cut? Check if recourse function approximation given by stored multi-cut information is better than recourse function approximation given by current master problem solution plus some part of the existing gap, according to Fábián (2013) If so, add the ODA cuts (insubstantial iteration) Add Cuts Resolve Master Recheck If not, solve second stage as usual (substantial iteration) κ < (1-λ), according to Fábián (2013) A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 19
Drawbacks and Advantages of ODA In every iteration, have to create additional recourse function approximation Need more memory, due to storing all cut components, even more than for the multi-cut method Can actually use all the information we computed No need for premature cut proliferation, add cuts when needed Allows to skip second stage every now and then A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 20
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 21 Agenda Introduction On-Demand Accuracy Implementation and Results Conclusion
Some notes on implementation Implementation within Coin-OR Framework (C++) Allows release as open source software Stochastic modelling language is available too (FlopC++) FlopC++ can write SMPS files and access solver directly Stochastic modelling interface (Smi) is used to read in SMPS files and store stochastic data Uses OpenSolverInterface (Osi) for access to external solvers Parallelization, can solve all subproblems in parallel Warm Start A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 22
Testbed for numerical results Intel i7-3770: 8 cores with 3.4 Ghz 16 GB RAM Windows 7 64 bit Cplex 12.4 64 bit 83 Test Instances for Two-Stage problems Slptestest compiled by Andy Felt Saphir problems by Achim Koberstein Sampling methods test set compiled by Stephen Wright rand problems, generated via genslp by Zverovich et al. POSTS test set compiled by Derek Holmes Deak test set, by István Deák (150 instances, not included in most of the results) A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 23
Measuring results Sum Arithmetic Mean Geometric Mean Shifted Geometric Mean Performance Profile A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 24
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 25 On-demand accuracy applied to Benders
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 26 On-demand accuracy - statistics Benders-ODA-κ=0.1 Benders-ODA-κ=0.3 Benders-ODA-κ=0.5 Benders-ODA-κ=0.7 Benders-ODA-κ=0.9 Benders Sum 3295.71 2783.57 2516.19 2529.54 3173.32 5567.97 Arithmetic Mean 39.71 33.54 30.32 30.48 38.23 67.08 Geometric Mean 2.58 2.34 2.23 2.22 2.37 3.43 Shifted Geometric Mean 13.79 12.35 11.82 11.89 13.44 18.33
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 27 Level decomposition Influence of Lambda
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 28 Level decomposition with ODA Kappa Influence
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 29 More detailled statistics Level-λ=0.1 Level-λ=0.3 Level-λ=0.5 Level-λ=0.7 Level-λ=0.9 Sum 2129.34 1654.82 1551.07 1493.66 1856.82 Arithmetic Mean 25.65 19.94 18.69 18.00 22.37 Geometric Mean 2.28 1.99 2.03 2.09 2.50 Shifted Geometric Mean 10.60 8.75 8.40 8.22 9.08 Level-ODA,,λ=0.5 κ=0.1 κ=0.3 κ=0.5 κ=0.7 κ=0.9 Sum 1175.74 1146.10 1163.32 1108.07 1138.49 Arithmetic Mean 14.17 13.81 14.02 13.35 13.72 Geometric Mean 1.82 1.78 1.75 1.71 1.77 Shifted Geometric Mean 7.15 6.91 6.85 6.66 6.79
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 30 Overall comparison
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 31 Overall comparison - statistics Level-ODA,λ=0.5,κ=0.7 Level-λ=0.5 Benders-ODA-κ=0.5 Benders Sum 1108.07 1551.07 2516.19 5567.97 Arithmetic Mean 13.35 18.69 30.32 67.08 Geometric Mean 1.71 2.03 2.23 3.43 Shifted Geometric Mean 16.66 18.40 21.82 28.33 Level-ODA,λ=0.5,κ=0.7 Level-λ=0.5 Benders-ODA-κ=0.5 Benders Overall Iterations 67.55 60.46 193.72 165.73 Substantial Iterations 34.37 60.46 36.43 165.73 Insubstantial Iterations 33.18 157.3
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 32 Selected Problems, solution times in seconds Instance Level-ODA,λ=0.5,κ=0.7 Level,λ=0.5 Benders-ODA Benders DEQ gbd 98 133 75 121 249 LandS 69 181 84 229 135 20_term-1000 8 15 72 116 7 20_term-2000 21 41 217 205 13 20_term-3000 28 45 314 291 24 ssn-1000 5 10 88 244 21 ssn-2000 14 25 107 437 58 ssn-3000 24 40 130 621 80 storm-1000 1 2 1 3 23 storm-2000 2 4 3 6 53 storm-3000 3 5 3 9 84
Our findings on deak test-set A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 33
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 34 Statistics for deak test set Level-ODA,λ=0.5,κ=0.7 Level-λ=0.5 Benders-ODA-κ=0.5 Benders Sum 46.23 49.87 36.26 66.84 Arithmetic Mean 0.31 0.33 0.24 0.45 Geometric Mean 0.24 0.25 0.18 0.26 Shifted Geometric Mean 0.31 0.33 0.24 0.43
Conclusion Using computed information is actually helpful On-demand accuracy is already helpful in the classical L-shaped method Runs in 45 % of solution time of normal L-shaped method on our test set Level decomposition also gains from on-demand accuracy Level-ODA runs in 71 % of solution time of Level decomposition Level-ODA runs in 20% of solution time of normal L-shaped method We showed these results on a diverse test-set with a competitive stateof-the-art algorithm Effects of number of aggregates should be tested in the future More test problems are always welcome A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 35
A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 36 References D. P. Morton, An enhanced decomposition algorithm for multistage stochastic hydroelectric scheduling, Annals of Operations Research 64 (1) (1996) 211 235 R. Van Slyke, and R. J.-B. Wets, L-shaped linear programs with applications to optimal control and stochastic programming, SIAM Journal on Applied Mathematics 17 (4) (1969) 638 663 J. F. Benders, Partitioning procedures for solving mixed-variables programming problems, Numerische Mathematik 4 (1962) 238 252 J. R. Birge, Decomposition and Partitioning Methods for Multistage Stochastic Linear Programs, Operations Research 33 (5) (1985) 989 1007 H. Gassmann, MSLiP: A computer code for the multistage stochastic linear programming problem, Mathematical Programming 47 (1) (1990), 407 423 F. Altenstedt, Aspects on asset liability management via stochastic programming, Ph.D. thesis, Chalmers University of Technology and Göteborg University, 2003 S. Trukhanov, L. Ntaimo, and A. Schaefer, Adaptive multicut aggregation for two-stage stochastic linear programs with recourse, European Journal of Operational Research 206 (2) (2010) 395 406 D. Holmes, A (PO)rtable (S)tochastic programming (T)est (S)et (POSTS), http://users.iems.northwestern.edu/~jrbirge/html/dholmes/post.html, 1995. K. Ariyawansa, A. Felt, On a new collection of stochastic linear programming test problems, INFORMS Journal on Computing 16 (3) (2004) 291 299. A. Koberstein, E. Lukas, M. Naumann, Integrated Strategic Planning of Global Production Networks and Financial Hedging under Uncertain Demand and Exchange Rates, Working paper, 2011. P. Kall and J. Mayer, On testing SLP codes with SLP-IOR, New Trends in Mathematical Programming: Homage to Steven Vajda (1998) 115 135. J. Linderoth, A. Shapiro, and S. Wright, The empirical behavior of sampling methods for stochastic programming, Annals of Operations Research Volume 142, Number 1, 215-241 W. Oliveira, and C. Sagastizábal, Level bundle methods for oracles with on-demand accuracy, submitted for publication, Preprint. Instituto Nacional de Matemática Pura e Aplicada IMPA, 2012 C. Fábián, Computational aspects of risk-averse optimization in two-stage stochastic models, Optimization Online (2013) C. Wolf, and Achim Koberstein, Dynamic sequencing and cut consolidation for the parallel hybrid-cut nested L-shaped method, European Journal of Operational Research 230(1) (2013) 143-156 Zverovich, V., Fábián, C. I., Ellison, E. F. D., & Mitra, G. (2012). A computational study of a solver system for processing two-stage stochastic LPs with enhanced Benders decomposition,mathematical Programming Computation, 4, 211 238
Public Service Announcement A computational study of on-demand accuracy level decomposition for two-stage stochastic programs 37