Interior Point Methods and Column Generation

Transcription

1 School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Interior Point Methods and Column Generation Jacek Gondzio J.Gondzio@ed.ac.uk URL: Bromont, June

2 Outline Part 1: IPMs for Optimization IPM tricks: log barrier, central path polynomial complexity optimal partition Part 2: Warmstarting IPMs Part 3: Column Generation with IPM cutting stock problem vehicle routing problem with time windows Conclusions Bromont, June

3 Part 1: Interior Point Methods for Optimization Bromont, June

4 Interior Point Methods re-born in 1984 Narendra Karmarkar, AT&T Bell Labs Shocking mathematical concept: Take linear optimization problem and add nonlinear function to the objective. A step against common sense and centuries of mathematical practice: nonlinearize linear problem Bromont, June

5 Logarithmic barrier -ln x lnx j replaces the inequality x j 0. Observe that min e n j=1 lnx j max n j=1 x j 1 x The minimization of n j=1 lnx j is equivalent to the maximization of the product of distances from all hyperplanes defining the positive orthant: it prevents all x j from approaching zero. Bromont, June

6 LP Problem: min c T x s.t. Ax = b, x 0. LP Barrier Prob: min c T x µ n lnx j s.t. Ax = b. j=1 Lagrangian: L(x,y,µ) = c T x y T (Ax b) µ n j=1 Stationarity: x L(x,y,µ) = c A T y µx 1 e = 0 y L(x,y,µ) = Ax b = 0. Denote: s = µx 1 e, i.e. XSe = µe. lnx j, Bromont, June

7 Complementarity in the Interior Point Method The first order optimality conditions (FOC) Ax = b, A T y + s = c, XSe = µe, x,s 0, where X = diag{x j }, S = diag{s j } and e = (1,,1) R n. Analytic centre (µ-centre): a (unique) point (x(µ), y(µ), s(µ)), x(µ) > 0, s(µ) > 0 that satisfies FOC. The interior point method gradually reduces the complementarity products x j s j µ 0 j = 1,2,...,n. Bromont, June

8 Interior Point Methods Theory: convergence in O( n) or O(n) iterations Practice: convergence in O(log n) iterations Expected number of IPM iterations: Problem Dimension LP QP 1, , , ,000, ,000, ,000, ,000, but one iteration may be expensive! Bromont, June

9 Complementarity x j s j = 0 j = 1,2,...,n. Simplex Method guesses an optimal partition: For basic variables, s B = 0 and (x B ) j (s B ) j = 0 j B. For non-basic variables, x N = 0 hence (x N ) j (s N ) j = 0 j N. Interior Point Method uses ε-mathematics: Replace x j s j = 0 j = 1,2,...,n by x j s j = µ j = 1,2,...,n. Force convergence µ 0. Bromont, June

10 First Order Optimality Conditions Simplex Method: Ax = b A T y + s = c XSe = 0 x, s 0. Interior Point Method: Ax = b A T y + s = c XSe = µe x, s 0. s s s s x Basic: x > 0, s = 0 Nonbasic: x = 0, s > 0 x x "Basic": x > 0, s = 0 "Nonbasic": x = 0, s > 0 x G, IPMs 25 years later, EJOR 218 (2012), Bromont, June

11 Part 2: Warmstarting IPMs Bromont, June

12 A need to solve a sequence of similar problems column generation cutting plane methods subproblems in the block-angular LPs (Dantzig-Wolfe decomp., Benders decomp.) B&B, (and B&Cut, B&Cut&Price, etc) SQP any sequence of similar problems example: computing efficient frontier in Markowitz portfolio optimization Bromont, June

13 Warm Starts Which method should be used? Simplex Method, or Interior Point Method. When is the Simplex Method better? few indices change optimal partition B & B, adding one cut in CPM, etc. When is the Interior Point Method better? many indices change optimal partition adding many cuts in CPM, dealing with a general change of problem data, etc Conjecture: The more changes in the (large) problem the more attractive IPM-based warm starts are. Bromont, June

14 Difficulty of IPM Warm Starts Modified Problem Original Problem Bromont, June

15 Warm Starting in Mitchell, PhD Thesis, Cornell Univ Goffin & Vial et al., development of ACCPM G. & Sarkissian, development of PDCGM in 1995 G., Math. Prog. 83 (1998) G. & Vial, COAP 14 (1999) ACCPM Analytic Centre Cutting Plane Method PDCGM Primal-Dual Column Generation Method Bromont, June

16 Warmstarting Heuristic Idea: Start close to the (new) central path, not close to the (old) solution Modified Problem Original Problem G., Mathematical Programming 83 (1998) Bromont, June

17 Warm Start with µ-centres Old Problem: New Problem: min c T 0 x xt Q 0 x s.t. A 0 x = b 0, x 0, min c T x xt Qx s.t. Ax = b, x 0, We assume: c c 0, Q Q 0, A A 0, b b 0. Bromont, June

18 Warm Starting in Yildirim & Wright, SIOPT 12 (2002) G. & Grothey, SIOPT 13 (2003) Fliege, Maths of OR 31 (2006) Benson & Shanno, COAP 38 (2007) Benson & Shanno, COAP 40 (2008) G. & Grothey, SIOPT 19 (2008) John & Yildirim, COAP 41 (2008) Colombo, G. & Grothey, MP 127 (2011) Colombo & Grothey, follow-up reports in 09,10 Engau, Anjos & Vannelli, SIOPT 20 (2010) 1828 Benson & Mahanta, report in 2009 Ordonez & Waltz, report in 2009 Bromont, June

19 IPM Warmstarts: Theoretical Results Yildirim & Wright, SIOPT 12 (2002) G. & Grothey, SIOPT 13 (2003) Lemma. Let (x,y,s) N (γ 0 ) for problem (LP) then the full Newton step ( x, y, s) in the perturbed problem ( LP) is feasible and (x + x,y + y,s + s) Ñ (γ) provided that δ bc = ξ c 2 + A T (AA T ) 1 γ ξ b 2 P /γ µ, where P = I S 1 A T (AXS 1 A T ) 1 AX, ξ b = b Ax, ξc = c A T y s. Bromont, June

20 LOQO vs OOPS warmstarting NETLIB problems Benson & Shanno, COAP 38 (2007) G. & Grothey, SIOPT 19 (2008) Unblocking technique... Average savings: LOQO (B&S, 2007) % OOPS (G&G, 2008) % Bromont, June

21 Part 3: Primal-Dual Column Generation Method Joint work with two PhD students: Pablo Gonzalez-Brevis and Pedro Munari Bromont, June

22 Column Generation (CG) MP... RMP newrmp Bromont, June

23 Column Generation (CG) Consider an LP, called the master problem (MP): z := min j N c j λ j, s.t. j N a j λ j = b, λ j 0, j N. N is too big; The columns a j are implicit elements of A; We know how to generate them! Bromont, June

24 CG: Restricted master problem (RMP): N N z RMP := min j N c j λ j, s.t. j N a j λ j = b, λ j 0, j N. Optimal λ for the RMP feasible ˆλ for the MP; ˆλj = λj, j N, and ˆλj = 0 otherwise; Hence, z z RMP = UB (Upper Bound). How to know it is optimal? Call the oracle! Bromont, June

25 CG: Oracle: check the feasibility of the dual u; Reduced costs: s j = c j u T a j, j N; But the columns are not explicit and, hence, z SP := min{c j u T a j a j A}. (we reset z SP := 0, if z SP > 0); Lower Bound: LB = z RMP + κz SP z, where κ i N λ i, If z SP < 0, then new columns are generated; Otherwise, an optimal solution of the MP was found! Bromont, June

26 Appealing features of IPMs Use IPM to solve the RMP: no degeneracy issues Terminate RMP solution early: get stable dual solution ū Bromont, June

27 PDCGM Algorithm Parameters: ε max, D, δ, κ 1. set LB =, UB =, gap =, ε = 0.5; 2. while (gap > δ) do 3. find a well-centred ε-optimal ( λ,ũ) of the RMP; 4. UB = z RMP ; 5. call the oracle with the query point ũ; 6. LB = κ z SP + b T ũ; 7. gap = (UB LB)/(1 + UB ); 8. ε = min{ε max, gap/d}; 9. if ( z SP < 0) then add new columns into the RMP; 10. end(while) Bromont, June

28 CSP: Column Generation Formulation Gilmore and Gomory (1961) formulation: min s.t. p P p P λ p, a p λ p d, λ p 0 and integer, p P. Columns are cutting patterns; We do not need to enumerate all of them; They can be dynamically generated knapsack problem. Bromont, June

29 VRPTW: Column Generation Formulation Desrochers et al. (1992): min s.t. p P p P c p λ p a p λ p = 1, λ p {0,1}, p P. Columns are possible vehicle paths; The columns can be dynamically generated shortest path problem with resource constraints. Bromont, June

30 Computational experiments Solving LP relaxations Comparison of: Standard column generation (SCG): simplex-type methods of IBM/CPLEX v Primal dual column generation (PDCGM): interior point solver HOPDM. Analytic centre cutting plane (ACCPM): open-source solver OBOE/COIN-OR. Bromont, June

31 Cutting stock problem SCG PDCGM ACCPM Cuts Class iters time iters time iters time 10 Small Large Small Large Small Large instances: 178 small (m 199), 84 large (m 200) Bromont, June

32 CSP: Larger Instances: BPP-U09??? family SCG PDCGM ACCPM Instance m iters time iters time iters time U U U U U U U U Bromont, June

33 Vehicle routing problem with time windows SCG PDCGM ACCPM Cuts Class iters time iters time iters time 10 Small Medium Large Small Medium Large instances: 29 small (n = 25), 29 med (n = 50), 29 large (n = 100) Bromont, June

34 VRPTW: Larger Instances SCG PDCGM ACCPM Instance n iters time iters time iters time R C RC R C RC R C RC Bromont, June

35 Integer optimization Bromont, June

36 Integer VRPTW solved to optimality Branch-Price-and-Cut, Pedro Munari s PhD DLH08 I-PDCGM Problem cuts nodes time cuts nodes time C C RC RC R R DLH08: Desaulniers, Lessard & Hadjar, Transportation Science 42 (2008) Solomon, Operations Research 35 (1987) Homberger&Gehring, EJOR 162 (2005) Bromont, June

37 Conclusions A completely new perspective is needed to exploit the insight offered by IPMs in a number of combinatorial optimization applications: column generation cutting plane methods B & B, (and B & Cut, B & Cut & Price, etc) Warmstarting works well in the CG context: problems are re-optimized in 3-5 IPM iterations Bromont, June

38 References G., Gonzalez-Brevis, Munari, New developments in the primal-dual column generation technique, ERGO Tech Rep, Edinburgh, Munari, G., Using the primal-dual interior point algorithm within the branch-price-and-cut method, ERGO Tech Rep, Edinburgh, G., Gonzalez-Brevis, A new warmstarting strategy for the primal-dual column generation method, ERGO Tech Rep, Edinburgh, Bromont, June

39 Example: Cutting Stock Problem (CSP) A set N of large pieces of wood of length W is given. We need to cut them into smaller pieces. We need d j units of small piece j M of length w j. Minimize the number of units of large pieces of wood. Define binary variable y i which takes value 1 if i-th large piece of wood is cut and 0 if it is not used. Define integer variable x ij which determines the number of units of small piece of wood j M obtained by cutting the large piece i N. Bromont, June

40 Cutting Stock Problem (CSP) Kantorovich s formulation: min s.t. j M i N i N y i x ij d j j M, x ij w j Wy i i N, y i {0,1} i N, x ij 0 and integer j M, i N. LP relaxation gives very weak bound. Bromont, June

41 Vehicle Routing Problem with Time Windows A company delivers goods to customers i C. The company has vehicles k V and each of them starts at a depot, travels to several customers and returns to the depot. The visit of vehicle k to customer i needs to take place in a specific time window: a i s ik b i, where s ik is the time when vehicle k reaches customer i. Objective: Minimize the total cost of delivery. Define binary variable x ijk which takes value 1 if vehicle k travels from customer i to customer j (k V, i,j C) and takes value zero otherwise. Bromont, June

42 Vehicle Routing Problem with Time Windows Constraints: Exactly one vehicle leaves customer i: k V j N Vehicle capacity constraint: i C d i j N x ijk = 1, x ijk q, i C k V Each vehicle leaves the depot and returns to it: j N x 0jk = 1 and j N x i(n+1)k = 1, k V Bromont, June

43 VRPTW: Constraints (continued) Time-window constraint s ik + t ij M(1 x ijk ) s jk, i,j N, k V. Since x ijk is binary the above constraint has the following meaning: If x ijk = 1 (vehicle k travels from customer i to customer j) then s ik + t ij s jk that is, the arrival time of vehicle k to customer j is greater than or equal the sum of time when vehicle k arrives to customer i and the time t ij it takes to travel from i to j. Otherwise (if x ijk = 0) the presence of big M guarantees that the constraint is always inactive. Bromont, June

44 VRPTW min j N s.t. i C x 0jk = 1, k V k V d i j N i N i N j N i N x ihk j N j N c ij x ijk x ijk = 1, i C, x ijk q, k V, x i(n+1)k = 1, k V, x jhk = 0, h C, k V, s ik + t ij M(1 x ijk ) s jk, i,j N, k V, a i s ik b i, i N, k V, x ijk {0,1}, i,j N, k V. Bromont, June