Solutions to exam in SF1811 Optimization, June 3, 14 1.(a) The considered problem may be modelled as a minimum-cost network flow problem with six nodes F1, F, K1, K, K3, K4, here called 1,,3,4,5,6, and eight arcs (F1,K1), (F1,K), (F1,K3), (F1,K4), (F,K1), (F,K), (F,K3), (F,K4), here called (1,3), (1,4), (1,5), (1,6), (,3), (,4), (,5), (,6). The suggested solution corresponds to a spanning tree with the arcs (1,3), (1,4), (,4), (,5) och (,6), i.e. a basic solution. It is a feasible basic solution since all the balance equations (in all nodes) are satisfied and all variables are non-negative. The simplex variables y i are obtained from the equations y i y j = c ij for basic variables, and y 6 =. This gives y = (5, 3,, 1,, ). Then the reduced costs for the non-basic variables are obtained from r ij = c ij y i + y j. This gives: r 15 = 8 5 + ( ) = 1, r 16 = 6 5 + = 1, r 3 = 6 3 + ( ) = 1. Since all r ij, the suggested solution is optimal. 1
1.(b) The vector ˆx is a global optimal solution to the problem of minimizing the function f(x) = 1 xt Hx if and only if H is positive semi-definite and Hˆx =. To check if H is positive semi-definite, we try to LDL T -factorize H. 1 1 H = 1 1. 1 k Add +1 times row 1 to row. Then add +1 times column 1 to column. This gives E 1 = 1 1 and E 1 HE T 1 = 1 1. 1 1 k Add +1 times row to row 3. Then add +1 times column to column 3. This gives E = 1 and E E 1 HE T 1 E T = 1. 1 1 k 1 The LDL T -factorization is ready, and one gets H = LDL T = 1 1 1 1 1 k 1 1 1 1 1 1. From this, it follows that H is positive semi-definite if and only if k 1. The system Hx = always has at least the solution x = (the trivial solution), and it has an infinite number of solutions if and only if H is singular. From above, H is both positive semi-definite and singular if and only if k = 1. Then H = 1 1 1 1. 1 1 Some calculations give that the set of solutions to Hx = is x = (t, t, t) T for t IR, and this is then the set of global optimal solutions to the problem of minimizing 1 xt Hx.
.(a) (Sorry for the Swedish text) Inför slackvariabler x 4, x 5 och x 6 så att problemet blir på standardformen där A = 1 1 1 1 1 1 1 minimera c T x då Ax = b,, b = x, och c = (1, 1, 4,,, ) T. I startlösningen ska enligt uppgiftslydelsen x 1, x och x 3 vara basvariabler, dvs β = (1,, 3) och δ = (4, 5, 6). 1 1 Motsvarande basmatris ges av A β = 1 1. 1 1 Basvariablernas värden i baslösningen ges av x β = b, där vektorn b beräknas ur ekvationssystemet A β b = b, 1 1 b1 b1 1 dvs 1 1 b =, med lösningen b = b = 1. 1 1 b3 b3 1 Här använde vi den givna räknehjälpen. Vektorn y med simplexmultiplikatorerna värden erhålls ur systemet A T β y = c β, 1 1 y 1 1 y 1 1 dvs 1 1 y = 1, med lösningen y = y =. 1 1 y 3 4 y 3 Här använde vi igen den givna räknehjälpen. Reducerade kostnaderna för icke-basvariablerna ges av r T δ = c δ y T A δ = = (,, ) ( 1,, ) 1 = ( 1,, ). 1 Eftersom r δ1 = r 4 = 1 är minst, och <, ska vi låta x 4 bli ny basvariabel. Då behöver vi beräkna vektorn ā 4 ur systemet A β ā 4 = a 4, 1 1 ā 14 1 dvs 1 1 ā 4 =, med lösningen ā 4 = 1 1 ā 34 Här använde vi igen den givna räknehjälpen. Det största värde som den nya basvariabeln x 4 kan ökas till ges av { } bi t max = min ā i4 > = b 3 = i ā i4 ā 34.5. ā 14.5 ā 4 =.5. ā 34.5 Minimerande index är i = 3, varför x β3 = x 3 inte längre får vara kvar som basvariabel. 3
Nu är alltså β = (1,, 4) och δ = (3, 5, 6). 1 1 1 Motsvarande basmatris ges av A β =. 1 Basvariablernas värden i baslösningen ges av x β = b, där vektorn b beräknas ur ekvationssystemet A β b = b, 1 1 1 b1 b1 dvs b =, med lösningen b = b =. 1 b3 b3 Vektorn y med simplexmultiplikatorernas värden erhålls ur systemet A T β y = c β, 1 1 y 1 1 y 1 dvs 1 1 y = 1, med lösningen y = y = 1. y 3 y 3 1 Reducerade kostnaderna för icke-basvariablerna ges av r T δ = c δ y T A δ = = (4,, ) (, 1, 1) 1 1 = (, 1, 1). 1 1 Eftersom r δ så är den aktuella baslösningen optimal. Alltså är punkten x 1 =, x =, x 3 =, x 4 =, x 5 =, x 6 = optimal. Optimalvärdet är c T x = 4..(b) If the primal problem is on the standard form then the dual problem is on the form which for the current example becomes minimize c T x subject to Ax = b, x, maximize b T y subject to A T y c, maximize y 1 + y + y 3 subject to y 1 + y 1, y 1 + y 3 1, y + y 3 4, y 1, y, y 3. It is well known that an optimal solution to this dual problem is given by the vector y of simplex multipliers in the optimal basic solution from.(a), i.e. y = (, 1, 1) T. We note that this is a feasible solution to the dual problem with b T y = 4 = the optimal value of the primal problem. 4
3. The gradient of f is f(x) = The Hessian of f is F(x) = ( f, f, f ), where f = 4x 3 j + 3x j + x j + 1. x 1 x x 3 x j f x 1 f x f x 3, where f x j = 1x j + 6x j +. (b) f is convex on IR 3 if and only if F(x) is positive semi-definite for all x IR 3. A diagonal matrix is positive semi-definite if and only if all diagonal elements are. But f x j = 1(x j + 1 x j + 1 6 ) = 1((x j + 1 4 ) 1 16 + 1 6 ) >, so F(x) is positive definite for all x IR 3. Thus, f is (strictly) convex on IR 3. (a) The given starting point is x (1) = 1. 1 If F(x (1) ) is positive definite, which it is according to above, then the first Newton direction d (1) is obtained as the solution to the system F(x (1) )d = f(x (1) ) T, i.e. 1 1/ d = 1, with the unique solution d (1) = 1/. 8 1/4 1/ We try first with t 1 = 1, so that x () = x (1) + t 1 d (1) = x (1) + d (1) = 1/. 3/4 Then f(x () ) = 85/56 < 4 = f(x (1) ), so t 1 = 1 is accepted. Now we have made a complete iteration with Newtons method and obtained the next iteration 1/ point x () = 1/ with f(x () ) = 85/56. 3/4 (c) Here comes a possible lower bound on f(x). First, we note that x j + x j = (x j + 1 ) 1 4 1 4. From this, it follows that x 4 j + x3 j = x j (x j + x j) 1 4 x j, from which we get that x 4 j + x3 j + x j + x j 1 4 x j + x j + x j = 3 4 (x j + 4 3 x j) = 3 4 ((x j + 3 ) 4 9 ) = 3 4 ( 4 9 ) = 1 3. This holds for each j, so we obtain that 3 f(x) = (x 4 j + x 3 j + x j + x j ) 1 3 1 3 1 3 = 1. Thus, L = 1 is a lower bound on f(x), but possibly not the highest one. 5
4.(a) The perimeter problem can be written as the following minimization problem: minimize f(x) = 4x 1 4x subject to h(x) = x 1 a + x 1 a 1 =. We assume that x 1 and x are >, but don t formulate that explicitly in the optimality conditions, it will naturally be satisfied anyhow by the optimal solution. The Lagrange conditions for the problem become: 4 + λx 1 a 1 =, 4 + λx a =, x 1 a + x a 1 =. Since every feasible point is also a regular point (the gradient of the only constraint function is never the zero vector) the Lagrange conditions are necessary conditons for an optimal solution. Some calculation give that the only solution (with x 1 > and x > ) is x 1 = 4.(b) a 1, x a 1 + a = a, u = a 1 + a a 1 + a, with perimeter 4x 1 + 4x = 4 a 1 + a. The area problem can be written as the following minimization problem: minimize f(x) = 4x 1 x subject to h(x) = x 1 a + x 1 a 1 =. Again, we assume that x 1 and x are >, but don t formulate that explicitly in the optimality conditions. The Lagrange conditions for the problem become: 4x + λx 1 a 1 4x 1 + λx a =, =, x 1 a + x a 1 =. Since every feasible point is also a regular point (the gradient of the only constraint function is never the zero vector) the Lagrange conditions are necessary conditons for an optimal solution. Some calculation give that the only solution (with x 1 > and x > ) is x 1 = a 1, x = a, u = a 1 a, with area 4x 1 x = a 1 a. 6
5.(a) The Lagrange function for the problem is given by n c j n L(x, y) = f(x) + y g(x) = + y ( x j V ) = yv + x j n ( c j x j + yx j ). The Lagrange relaxed problem PR y is defined, for a given y, as the problem of minimizing L(x, y) with respect to x IR n. But this problem separates into one problem for each variable x j, namely We have that l j(x j ) = c j minimize l j (x j ) = c j x j + yx j subject to x j [.1, 1 ]. (.1) x j + y and l j (x j ) = c j that l j (x j ) is strictly convex on the interval [.1, 1 ]. x 3 j >, which implies If y = then l j (x j ) is strictly decreasing, and then x j = 1 is the unique optimal solution to the subproblem (.1). cj If y > the unique solution to l j (x j) = is x j = y to the interval [.1, 1 ] if and only if c j y 1c j. which belongs From this, we conclude that the unique optimal solution x j (y) to the subproblem (.1) is as follows: x j (y) = 1 if y c j, cj if c j y 1c j, y.1 if y 1c j, The dual objective function is then given by n c j ϕ(y) = L( x(y), y) = yv + ( x j (y) + y x j(y)). The dual problem consists of maximizing ϕ(y) with respect to y. 7
5.(b) Assume now that n =, V = 1.5, c 1 = 1 and c = 9. Then we get the following different expressions for ϕ(y), depending on which of five intervals y belongs to: 1.) y 1 x 1 (y) = 1, x (y) = 1, ϕ(y) = 1 +.5y, ϕ (y) =.5..) 1 y 9 x 1 (y) = 1 y, x (y) = 1, ϕ(y) = 9 + y.5y. ϕ (y) = 1 y.5. 3.) 9 y 1 x 1 (y) = 1 y, x (y) = 3 y, ϕ(y) = 8 y 1.5y. ϕ (y) = 4 y 1.5. 4.) 1 y 9 x 1 (y) =.1, x (y) = 3 y, ϕ(y) = 1 + 6 y 1.4y. ϕ (y) = 3 y 1.4. 5.) 9 y x 1 (y) =.1, x (y) =.1, ϕ(y) = 1 1.3y. ϕ (y) = 1.3. We note that ϕ and ϕ are continuous, and ϕ is decreasing. In particular, ϕ (1) =.5 > and ϕ (9) = 1/6 <. from which it follows that the optimal solution to the dual problem is to be found in the interval 1 y 9, where ϕ (y) = 1 y.5. The unique solution ŷ to ϕ (y) = is then given by ŷ = 4, with ϕ(ŷ) = 11. The corresponding primal solution is (ˆx 1, ˆx ) = ( x 1 (ŷ), x (ŷ)) = (.5, 1), which is a feasible solution to the primal problem with f(ˆx 1, ˆx ) = + 9 = 11 = ϕ(ŷ). Thus, (ˆx 1, ˆx ) = (.5, 1) is a global optimal solution to the primal problem. 8