# 3.2: Minimization Applications

Learning Objectives

In this section, you will learn to:

1. Formulate minimization linear programming problems
2. Graph feasibility regions for maximization linear programming problems
3. Determine optimal solutions for maximization linear programming problems.

Minimization linear programming problems are solved in much the same way as the maximization problems.

For the standard minimization linear program, the constraints are of the form (ax + by ≥ c), as opposed to the form (ax + by ≤ c) for the standard maximization problem. As a result, the feasible solution extends indefinitely to the upper right of the first quadrant, and is unbounded. But that is not a concern, since in order to minimize the objective function, the line associated with the objective function is moved towards the origin, and the critical point that minimizes the function is closest to the origin.

However, one should be aware that in the case of an unbounded feasibility region, the possibility of no optimal solution exists.

Example (PageIndex{1})

At a university, Professor Symons wishes to employ two people, John and Mary, to grade papers for his classes. John is a graduate student and can grade 20 papers per hour; John earns $15 per hour for grading papers. Mary is an post-doctoral associate and can grade 30 papers per hour; Mary earns$25 per hour for grading papers. Each must be employed at least one hour a week to justify their employment.
If Prof. Symons has at least 110 papers to be graded each week, how many hours per week should he employ each person to minimize the cost?

Solution

We choose the variables as follows:

Let (x) = The number of hours per week John is employed.

and (y) = The number of hours per week Mary is employed.

The objective function is

[C = 15x + 25y onumber ]

The fact that each must work at least one hour each week results in the following two constraints:

[egin{array}{l}
x geq 1
y geq 1
end{array} onumber ]

Since John can grade 20 papers per hour and Mary 30 papers per hour, and there are at least 110 papers to be graded per week, we get

[20x + 30y ≥ 110 onumber]

The fact that (x) and (y) are non-negative, we get

[x ≥ 0 ext{, and } y ≥ 0. onumber]

The problem has been formulated as follows.

[egin{array}{ll}
extbf { Minimize } & mathrm{C}=15 mathrm{x}+25 mathrm{y}
extbf { Subject to: } & mathrm{x} geq 1
& mathrm{y} geq 1
& 20 mathrm{x}+30 mathrm{y} geq 110
& mathrm{x} geq 0 ; mathrm{y} geq 0
end{array} onumber]

To solve the problem, we graph the constraints as follows: Again, we have shaded the feasibility region, where all constraints are satisfied.

If we used test point (0,0) that does not lie on any of the constraints, we observe that (0, 0) does not satisfy any of the constraints (x ≥ 1), (y ≥ 1), (20x + 30y ≥ 110). Thus all the shading for the feasibility region lies on the opposite side of the constraint lines from the point (0,0).

Alternatively we could use test point (4,6), which also does not lie on any of the constraint lines. We’d find that (4,6) does satisfy all of the inequality constraints. Consequently all the shading for the feasibility region lies on the same side of the constraint lines as the point (4,6).

Since the extreme value of the objective function always takes place at the vertices of the feasibility region, we identify the two critical points, (1, 3) and (4, 1). To minimize cost, we will substitute these points in the objective function to see which point gives us the minimum cost each week. The results are listed below.

 Critical points Income (1, 3) 15(1) + 25(3) = $90 (4, 1) 15(4) + 25(1) =$85

The point (4, 1) gives the least cost, and that cost is $85. Therefore, we conclude that in order to minimize grading costs, Professor Symons should employ John 4 hours a week, and Mary 1 hour a week at a cost of$85 per week.

Example (PageIndex{2})

Professor Hamer is on a low cholesterol diet. During lunch at the college cafeteria, he always chooses between two meals, Pasta or Tofu. The table below lists the amount of protein, carbohydrates, and vitamins each meal provides along with the amount of cholesterol he is trying to minimize. Mr. Hamer needs at least 200 grams of protein, 960 grams of carbohydrates, and 40 grams of vitamins for lunch each month. Over this time period, how many days should he have the Pasta meal, and how many days the Tofu meal so that he gets the adequate amount of protein, carbohydrates, and vitamins and at the same time minimizes his cholesterol intake?

 PASTA TOFU PROTEIN 8g 16g CARBOHYDRATES 60g 40g VITAMIN C 2g 2g CHOLESTEROL 60mg 50mg

Solution

We choose the variables as follows.

Let (x) = The number of days Mr. Hamer eats Pasta.

and (y) = The number of days Mr. Hamer eats Tofu.

Since he is trying to minimize his cholesterol intake, our objective function represents the total amount of cholesterol C provided by both meals.

[C = 60x + 50y onumber ]

The constraint associated with the total amount of protein provided by both meals is

[8x + 16y ≥ 200 onumber ]

Similarly, the two constraints associated with the total amount of carbohydrates and vitamins are obtained, and they are

[egin{array}{l}
60 x+40 y geq 960
2 x+2 y geq 40
end{array} onumber]

The constraints that state that x and y are non-negative are

[x ≥ 0 ext{, and } y ≥ 0 onumber.]

We summarize all information as follows:

[egin{array}{ll}
extbf { Minimize } & mathrm{C}=60 mathrm{x}+50 mathrm{y}
extbf { Subject to: } & 8 mathrm{x}+16 mathrm{y} geq 200
& 60 mathrm{x}+40 mathrm{y} geq 960
& 2 mathrm{x}+2 mathrm{y} geq 40
& mathrm{x} geq 0 ; mathrm{y} geq 0
end{array} onumber]

To solve the problem, we graph the constraints and shade the feasibility region. We have shaded the unbounded feasibility region, where all constraints are satisfied.

To minimize the objective function, we find the vertices of the feasibility region. These vertices are (0, 24), (8, 12), (15, 5) and (25, 0). To minimize cholesterol, we will substitute these points in the objective function to see which point gives us the smallest value. The results are listed below.

 Critical points Income (0, 24) 60(0) + 50(24) = 1200 (8, 12) 60(8) + 50(12) = 1080 (15, 5) 60(15) + 50(5) = 1150 (25, 0) 60(25) + 50(0) = 1500

The point (8, 12) gives the least cholesterol, which is 1080 mg. This states that for every 20 meals, Professor Hamer should eat Pasta 8 days, and Tofu 12 days.

We must be aware that in some cases, a linear program may not have an optimal solution.

• A linear program can fail to have an optimal solution is if there is not a feasibility region. If the inequality constraints are not compatible, there may not be a region in the graph that satisfies all the constraints. If the linear program does not have a feasible solution satisfying all constraints, then it can not have an optimal solution.
• A linear program can fail to have an optimal solution if the feasibility region is unbounded.
• The two minimization linear programs we examined had unbounded feasibility regions. The feasibility region was bounded by constraints on some sides but was not entirely enclosed by the constraints. Both of the minimization problems had optimal solutions.
• However, if we were to consider a maximization problem with a similar unbounded feasibility region, the linear program would have no optimal solution. No matter what values of x and y were selected, we could always find other values of (x) and (y) that would produce a higher value for the objective function. In other words, if the value of the objective function can be increased without bound in a linear program with an unbounded feasible region, there is no optimal maximum solution.

Although the method of solving minimization problems is similar to that of the maximization problems, we still feel that we should summarize the steps involved.

Minimization Linear Programming Problems

1. Write the objective function.
2. Write the constraints.
1. For standard minimization linear programming problems, constraints are of the form: (ax + by ≥ c)
2. Since the variables are non-negative, include the constraints: (x ≥ 0); (y ≥ 0).
3. Graph the constraints.
5. Find the corner points.
6. Determine the corner point that gives the minimum value.
1. This can be done by finding the value of the objective function at each corner point.
2. This can also be done by moving the line associated with the objective function.
3. There is the possibility that the problem has no solution

## Schedulability analysis and stack size minimization with preemption thresholds and mixed-criticality scheduling

Mixed-Criticality Scheduling (MCS) is an effective approach to addressing certification requirements of safety-critical Cyber-Physical Systems that integrate multiple subsystems with different levels of criticality in application domains such as avionics and automotive systems. Although MCS was originally proposed in the context of safety-critical avionics applications, it is also finding its way into the automotive domain which faces intense cost-cutting pressure in today’s hyper-competitive market, so it is important to minimize hardware costs by adopting low-cost processors with limited processing and memory resources. Preemption Threshold Scheduling (PTS) is a well-known technique for controlling the degree of preemption in real-time scheduling, with benefits of reduced stack size and reduced number of preemptions compared to fully-preemptive scheduling. We present schedulability analysis to enable integration of PTS with MCS, including two variants PT-rtb and PT-max, in order to reduce application stack space requirement, and enable efficient implementation of MCS on resource-constrained embedded platforms. We also integrate our schedulability tests with priority and preemption threshold assignment algorithms, to have a complete solution for analysis and synthesis of mixed-criticality systems. Performance evaluation illustrates the benefits of our approach in terms of increased schedulability and reduced stack requirement.

## Two Phase Method: Minimization Example 1

If the objective function is in minimization form, then convert it into maximization form.

Changing the sense of the optimization

Any linear minimization problem can be viewed as an equivalent linear maximization problem, and vice versa. Specifically:

If z is the optimal value of the left-hand expression, then -z is the optimal value of the right-hand expression.

Converting inequalities to equalities

Where:
x4 is a slack variable
x5 is a surplus variable

The surplus variable x5 represents the extra units.

Now, if we let x1, x2 and x3 equal to zero in the initial solution, we will have x4 = 5 and x5 = -2, which is not possible because a surplus variable cannot be negative. Therefore, we need artificial variables .

Where A1 and A2 are artificial variables.

#### Phase 1 of Two Phase Method

In this phase, we remove the artificial variables and find an initial feasible solution of the original problem. Now the objective function can be expressed as

Initial basic feasible solution

The intial basic feasible solution is obtained by setting
x1 = x2 = x3 = x5 = 0

Then we shall have A1 = 2 , A2 = 5, x4 = 5

##### Two Phase Method: Table 1

On small screens, scroll horizontally to view full calculation

cj 0 0 0 0 0 -1 -1
cB Basic variables
B
x1 x2 x3 x4 x5 A1 A2 Solution values
b (= XB)
0 x4 1 3 1 1 0 0 0 5
-1 A1 2 -1 1 0 -1 1 0 2
-1 A2 4 3 -2 0 0 0 1 5
zj–cj -6 -2 1 0 1 0 0

Key column = x1 column
Minimum (5/1, 2/2, 5/4) = 1
Key row = A1 row
Pivot element = 2
A1 departs and x1 enters.

A2 departs and x2 enters.
Here, Phase 1 terminates because both the artificial variables have been removed from the basis.

#### Phase 2 of Two Phase Method

The basic feasible solution at the end of Phase 1 computation is used as the initial basic feasible solution of the problem. The original objective function is introduced in Phase 2 computation and the usual simplex procedure is used to solve the problem.

Use horizontal scrollbar to view full calculation

cj 3 -1 2 0 0
cB Basic variables
B
x1 x2 x3 x4 x5 Solution values
b (= XB)
0 x4 0 0 33/10 1 -9/10 33/10
3 x1 1 0 1/10 0 -3/10 11/10
-1 x2 0 1 -4/5 0 2/5 1/5
zj-cj 0 0 -9/10 0 -13/10
cj 3 -1 2 0 0
cB Basic variables
B
x1 x2 x3 x4 x5 Solution values
b (= XB)
0 x4 0 9/4 3/2 1 0 15/4
3 x1 1 3/4 -1/2 0 0 5/4
0 x5 0 5/2 -2 0 1 1/2
zj-cj 0 13/4 -7/2 0 0

Don't be impatient. The next table is the last table.

"The most useful virtue is patience" - John Dewey

#### Two Phase Method: Final Optimal Table

cj 3 -1 2 0 0
cB Basic variables
B
x1 x2 x3 x4 x5 Solution values
b (= XB)
2 x3 0 3/2 1 2/3 0 5/2
3 x1 1 3/2 0 1/3 0 5/2
0 x5 0 11/2 0 4/3 1 11/2
zj-cj 0 17/2 0 7/3 0

An optimal policy is x1 = 5/2, x2 = 0, x3 = 5/2. The associated optimal value of the objective function is z = 3 X (5/2) – 0 + 2 X (5/2) = 25/2.

## Scipy.optimize.minimize¶

Minimization of scalar function of one or more variables.

Parameters fun callable

The objective function to be minimized.

where x is an 1-D array with shape (n,) and args is a tuple of the fixed parameters needed to completely specify the function.

x0 ndarray, shape (n,)

Initial guess. Array of real elements of size (n,), where ‘n’ is the number of independent variables.

args tuple, optional

Extra arguments passed to the objective function and its derivatives (fun, jac and hess functions).

method str or callable, optional

Type of solver. Should be one of

• ‘Powell’ (see here)

• ‘CG’ (see here)

• ‘BFGS’ (see here)

• ‘Newton-CG’ (see here)

• ‘L-BFGS-B’ (see here)

• ‘TNC’ (see here)

• ‘COBYLA’ (see here)

• ‘SLSQP’ (see here)

• ‘trust-constr’ (see here)

• ‘dogleg’ (see here)

• ‘trust-ncg’ (see here)

• ‘trust-exact’ (see here)

• ‘trust-krylov’ (see here)

• custom - a callable object (added in version 0.14.0), see below for description.

If not given, chosen to be one of BFGS , L-BFGS-B , SLSQP , depending if the problem has constraints or bounds.

Method for computing the gradient vector. Only for CG, BFGS, Newton-CG, L-BFGS-B, TNC, SLSQP, dogleg, trust-ncg, trust-krylov, trust-exact and trust-constr. If it is a callable, it should be a function that returns the gradient vector:

jac(x, *args) -> array_like, shape (n,)

where x is an array with shape (n,) and args is a tuple with the fixed parameters. Alternatively, the keywords <‘2-point’, ‘3-point’, ‘cs’>select a finite difference scheme for numerical estimation of the gradient. Options ‘3-point’ and ‘cs’ are available only to ‘trust-constr’. If jac is a Boolean and is True, fun is assumed to return the gradient along with the objective function. If False, the gradient will be estimated using ‘2-point’ finite difference estimation.

Method for computing the Hessian matrix. Only for Newton-CG, dogleg, trust-ncg, trust-krylov, trust-exact and trust-constr. If it is callable, it should return the Hessian matrix:

where x is a (n,) ndarray and args is a tuple with the fixed parameters. LinearOperator and sparse matrix returns are allowed only for ‘trust-constr’ method. Alternatively, the keywords <‘2-point’, ‘3-point’, ‘cs’>select a finite difference scheme for numerical estimation. Or, objects implementing HessianUpdateStrategy interface can be used to approximate the Hessian. Available quasi-Newton methods implementing this interface are:

Whenever the gradient is estimated via finite-differences, the Hessian cannot be estimated with options <‘2-point’, ‘3-point’, ‘cs’>and needs to be estimated using one of the quasi-Newton strategies. Finite-difference options <‘2-point’, ‘3-point’, ‘cs’>and HessianUpdateStrategy are available only for ‘trust-constr’ method.

hessp callable, optional

Hessian of objective function times an arbitrary vector p. Only for Newton-CG, trust-ncg, trust-krylov, trust-constr. Only one of hessp or hess needs to be given. If hess is provided, then hessp will be ignored. hessp must compute the Hessian times an arbitrary vector:

hessp(x, p, *args) ->   ndarray shape (n,)

where x is a (n,) ndarray, p is an arbitrary vector with dimension (n,) and args is a tuple with the fixed parameters.

bounds sequence or Bounds , optional

Bounds on variables for L-BFGS-B, TNC, SLSQP and trust-constr methods. There are two ways to specify the bounds:

1. Instance of Bounds class.

2. Sequence of (min, max) pairs for each element in x. None is used to specify no bound.

Constraints definition (only for COBYLA, SLSQP and trust-constr). Constraints for ‘trust-constr’ are defined as a single object or a list of objects specifying constraints to the optimization problem. Available constraints are:

Constraints for COBYLA, SLSQP are defined as a list of dictionaries. Each dictionary with fields:

type str

Constraint type: ‘eq’ for equality, ‘ineq’ for inequality.

fun callable

The function defining the constraint.

jac callable, optional

The Jacobian of fun (only for SLSQP).

args sequence, optional

Extra arguments to be passed to the function and Jacobian.

Equality constraint means that the constraint function result is to be zero whereas inequality means that it is to be non-negative. Note that COBYLA only supports inequality constraints.

tol float, optional

Tolerance for termination. For detailed control, use solver-specific options.

options dict, optional

A dictionary of solver options. All methods accept the following generic options:

maxiter int

Maximum number of iterations to perform.

disp bool

Set to True to print convergence messages.

For method-specific options, see show_options .

callback callable, optional

Called after each iteration. For ‘trust-constr’ it is a callable with the signature:

callback(xk, OptimizeResult state) -> bool

where xk is the current parameter vector. and state is an OptimizeResult object, with the same fields as the ones from the return. If callback returns True the algorithm execution is terminated. For all the other methods, the signature is:

where xk is the current parameter vector.

Returns res OptimizeResult

The optimization result represented as a OptimizeResult object. Important attributes are: x the solution array, success a Boolean flag indicating if the optimizer exited successfully and message which describes the cause of the termination. See OptimizeResult for a description of other attributes.

Interface to minimization algorithms for scalar univariate functions

Additional options accepted by the solvers

This section describes the available solvers that can be selected by the ‘method’ parameter. The default method is BFGS.

Unconstrained minimization

Method Nelder-Mead uses the Simplex algorithm , . This algorithm is robust in many applications. However, if numerical computation of derivative can be trusted, other algorithms using the first and/or second derivatives information might be preferred for their better performance in general.

Method Powell is a modification of Powell’s method ,  which is a conjugate direction method. It performs sequential one-dimensional minimizations along each vector of the directions set (direc field in options and info), which is updated at each iteration of the main minimization loop. The function need not be differentiable, and no derivatives are taken.

Method CG uses a nonlinear conjugate gradient algorithm by Polak and Ribiere, a variant of the Fletcher-Reeves method described in  pp. 120-122. Only the first derivatives are used.

Method BFGS uses the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS)  pp. 136. It uses the first derivatives only. BFGS has proven good performance even for non-smooth optimizations. This method also returns an approximation of the Hessian inverse, stored as hess_inv in the OptimizeResult object.

Method Newton-CG uses a Newton-CG algorithm  pp. 168 (also known as the truncated Newton method). It uses a CG method to the compute the search direction. See also TNC method for a box-constrained minimization with a similar algorithm. Suitable for large-scale problems.

Method dogleg uses the dog-leg trust-region algorithm  for unconstrained minimization. This algorithm requires the gradient and Hessian furthermore the Hessian is required to be positive definite.

Method trust-ncg uses the Newton conjugate gradient trust-region algorithm  for unconstrained minimization. This algorithm requires the gradient and either the Hessian or a function that computes the product of the Hessian with a given vector. Suitable for large-scale problems.

Method trust-krylov uses the Newton GLTR trust-region algorithm ,  for unconstrained minimization. This algorithm requires the gradient and either the Hessian or a function that computes the product of the Hessian with a given vector. Suitable for large-scale problems. On indefinite problems it requires usually less iterations than the trust-ncg method and is recommended for medium and large-scale problems.

Method trust-exact is a trust-region method for unconstrained minimization in which quadratic subproblems are solved almost exactly . This algorithm requires the gradient and the Hessian (which is not required to be positive definite). It is, in many situations, the Newton method to converge in fewer iteraction and the most recommended for small and medium-size problems.

Bound-Constrained minimization

Method L-BFGS-B uses the L-BFGS-B algorithm ,  for bound constrained minimization.

Method TNC uses a truncated Newton algorithm ,  to minimize a function with variables subject to bounds. This algorithm uses gradient information it is also called Newton Conjugate-Gradient. It differs from the Newton-CG method described above as it wraps a C implementation and allows each variable to be given upper and lower bounds.

Constrained Minimization

Method COBYLA uses the Constrained Optimization BY Linear Approximation (COBYLA) method , , . The algorithm is based on linear approximations to the objective function and each constraint. The method wraps a FORTRAN implementation of the algorithm. The constraints functions ‘fun’ may return either a single number or an array or list of numbers.

Method SLSQP uses Sequential Least SQuares Programming to minimize a function of several variables with any combination of bounds, equality and inequality constraints. The method wraps the SLSQP Optimization subroutine originally implemented by Dieter Kraft . Note that the wrapper handles infinite values in bounds by converting them into large floating values.

Method trust-constr is a trust-region algorithm for constrained optimization. It swiches between two implementations depending on the problem definition. It is the most versatile constrained minimization algorithm implemented in SciPy and the most appropriate for large-scale problems. For equality constrained problems it is an implementation of Byrd-Omojokun Trust-Region SQP method described in  and in , p. 549. When inequality constraints are imposed as well, it swiches to the trust-region interior point method described in . This interior point algorithm, in turn, solves inequality constraints by introducing slack variables and solving a sequence of equality-constrained barrier problems for progressively smaller values of the barrier parameter. The previously described equality constrained SQP method is used to solve the subproblems with increasing levels of accuracy as the iterate gets closer to a solution.

Finite-Difference Options

For Method trust-constr the gradient and the Hessian may be approximated using three finite-difference schemes: <‘2-point’, ‘3-point’, ‘cs’>. The scheme ‘cs’ is, potentially, the most accurate but it requires the function to correctly handles complex inputs and to be differentiable in the complex plane. The scheme ‘3-point’ is more accurate than ‘2-point’ but requires twice as much operations.

Custom minimizers

It may be useful to pass a custom minimization method, for example when using a frontend to this method such as scipy.optimize.basinhopping or a different library. You can simply pass a callable as the method parameter.

The callable is called as method(fun, x0, args, **kwargs, **options) where kwargs corresponds to any other parameters passed to minimize (such as callback, hess, etc.), except the options dict, which has its contents also passed as method parameters pair by pair. Also, if jac has been passed as a bool type, jac and fun are mangled so that fun returns just the function values and jac is converted to a function returning the Jacobian. The method shall return an OptimizeResult object.

The provided method callable must be able to accept (and possibly ignore) arbitrary parameters the set of parameters accepted by minimize may expand in future versions and then these parameters will be passed to the method. You can find an example in the scipy.optimize tutorial.

Nelder, J A, and R Mead. 1965. A Simplex Method for Function Minimization. The Computer Journal 7: 308-13.

Wright M H. 1996. Direct search methods: Once scorned, now respectable, in Numerical Analysis 1995: Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis (Eds. D F Griffiths and G A Watson). Addison Wesley Longman, Harlow, UK. 191-208.

Powell, M J D. 1964. An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal 7: 155-162.

Press W, S A Teukolsky, W T Vetterling and B P Flannery. Numerical Recipes (any edition), Cambridge University Press.

Nocedal, J, and S J Wright. 2006. Numerical Optimization. Springer New York.

Byrd, R H and P Lu and J. Nocedal. 1995. A Limited Memory Algorithm for Bound Constrained Optimization. SIAM Journal on Scientific and Statistical Computing 16 (5): 1190-1208.

Zhu, C and R H Byrd and J Nocedal. 1997. L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization. ACM Transactions on Mathematical Software 23 (4): 550-560.

Nash, S G. Newton-Type Minimization Via the Lanczos Method. 1984. SIAM Journal of Numerical Analysis 21: 770-778.

Powell, M J D. A direct search optimization method that models the objective and constraint functions by linear interpolation. 1994. Advances in Optimization and Numerical Analysis, eds. S. Gomez and J-P Hennart, Kluwer Academic (Dordrecht), 51-67.

Powell M J D. Direct search algorithms for optimization calculations. 1998. Acta Numerica 7: 287-336.

Powell M J D. A view of algorithms for optimization without derivatives. 2007.Cambridge University Technical Report DAMTP 2007/NA03

Kraft, D. A software package for sequential quadratic programming. 1988. Tech. Rep. DFVLR-FB 88-28, DLR German Aerospace Center – Institute for Flight Mechanics, Koln, Germany.

Conn, A. R., Gould, N. I., and Toint, P. L. Trust region methods. 2000. Siam. pp. 169-200.

F. Lenders, C. Kirches, A. Potschka: “trlib: A vector-free implementation of the GLTR method for iterative solution of the trust region problem”, https://arxiv.org/abs/1611.04718

N. Gould, S. Lucidi, M. Roma, P. Toint: “Solving the Trust-Region Subproblem using the Lanczos Method”, SIAM J. Optim., 9(2), 504–525, (1999).

Byrd, Richard H., Mary E. Hribar, and Jorge Nocedal. 1999. An interior point algorithm for large-scale nonlinear programming. SIAM Journal on Optimization 9.4: 877-900.

Lalee, Marucha, Jorge Nocedal, and Todd Plantega. 1998. On the implementation of an algorithm for large-scale equality constrained optimization. SIAM Journal on Optimization 8.3: 682-706.

Let us consider the problem of minimizing the Rosenbrock function. This function (and its respective derivatives) is implemented in rosen (resp. rosen_der , rosen_hess ) in the scipy.optimize .

A simple application of the Nelder-Mead method is:

Now using the BFGS algorithm, using the first derivative and a few options:

Next, consider a minimization problem with several constraints (namely Example 16.4 from ). The objective function is:

## 4. The QAOA algorithm The Quantum approximate optimization algorithm (QAOA) by Farhi, Goldstone and Gutmann 3 is an example of a heuristic algorithm. Unlike Goemans-Williamson algorithm, QAOA does not come with performance guarantees. QAOA takes the approach of classical approximate algorithms and looks for a quantum analogue that will likewise produce a classical bit string $x^*$ that with high probability is expected to have a good approximation ratio $alpha$. Before discussing the details, let us first present the general idea of this approach.

### 4.1 Overview:

We want to find a quantum state $|psi_p(vec,vec<eta>) angle$, that depends on some real parameters $vec,vec <eta>in mathbb^p$, which has the property that it maximizes the expectation value with respect to the problem Hamiltonian $H$. Given this trial state we search for parameters $vec^*,vec<eta>^*$ that maximize $F_p(vec,vec<eta>) = langle psi_p(vec,vec<eta>)|H|psi_p(vec,vec<eta>) angle$.

Once we have such a state and the corresponding parameters we prepare the state $|psi_p(vec^*,vec<eta>^*) angle$ on a quantum computer and measure the state in the $Z$ basis $|x angle = |x_1,ldots x_n angle$ to obtain a random outcome $x^*$.

We will see that this random $x^*$ is going to be a bit string that is with high probability close to the expected value $M_p = F_p(vec^*,vec<eta>^*)$. Hence, if $M_p$ is close to $C_$ so is $C(x^*)$.

### 4.2.1 The QAOA trial state Central to QAOA is the trial state $|psi_p(vec,vec) angle$ that will be prepared on the quantum computer. Ideally we want this state to give rise to a large expectation value $F_p(vec,vec) = langle psi_p(vec,vec)|H|psi_p(vec,vec) angle$ with respect to the problem Hamiltonian $H$. In Farhi 3, the trial states $|psi_p(vec,vec) angle$ are constructed from the problem Hamiltonian $H$ together with single qubit Pauli $X$ rotations. That means, given a problems Hamiltonian diagonal in the computational basis and a transverse field Hamiltonian the trial state is prepared by applying $p$ alternating unitaries to the product state $|+ angle^n$ with $X |+ angle = |+ angle$. This particular ansatz has the advantage that there exists an explicit choice for the vectors $vec^*,vec^*$ such that for $M_p = F_p(vec^*,vec^*)$ when we take the limit $lim_ M_p = C_$. This follows by viewing the trial state $|psi_p(vec,vec) angle$ as the state that follows from trotterizing the adiabatic evolution with respect to $H$ and the transverse field Hamiltonian $B$, c.f. Ref 3. Conversely the disadvantage of this trial state is one would typically want a state that has been generated from a quantum circuit that is not too deep. Here depth is measured with respect to the gates that can be applied directly on the quantum chip. Hence there are other proposals that suggest using Ansatz trial state that are more tailored to the Hardware of the quantum chip Ref. 4, Ref. 5. 4.2.2 Computing the expectation value An important component of this approach is that we will have to compute or estimate the expectation value so we can optimize the parameters $vec,vec$. We will be considering two scenarios here.

#### Classical evaluation

Note that when the circuit to prepare $|psi_p(vec,vec<eta>) angle$ is not too deep it may be possible to evaluate the expectation value $F_p$ classically.

This happens for instance when one considers $MAXCUT$ for graphs with bounded degree and one considers a circuit with $p=1$. We will see an example of this in the Qiskit implementation below (section 5.2) and provide an exercise to compute the expectation value.

To illustrate the idea, recall that the Hamiltonian can be written as a sum of individual terms $H = sum_^m hat_k$. Due to the linearity of the expectation value, it is sufficient to consider the expectation values of the individual summands. For $p = 1$ one has that

Observe that with $B = sum_^n X_i$ the unitary $e^< -ieta_1 B >$ is actually a product of single qubit rotations about $X$ with an angle $eta$ for which we will write $X(eta)_k = exp(ieta X_k)$.

All the individual rotations that don't act on the qubits where $hat_k$ is supported commute with $hat_k$ and therefore cancel. This does not increase the support of the operator $hat_k$. This means that the second set of unitary gates $e^ < -igamma_1 H >= prod_^m U_l(gamma)$ have a large set of gates $U_l(gamma) = e^< -igamma_1 hat_l >$ that commute with the operator $e^ < ieta_1 B >hat_k e^< -ieta_1 B >$. The only gates $U_l(gamma) = e^< -igamma_1 hat_l >$ that contribute to the expectation value are those which involve qubits in the support of the original $hat_k$.

Hence, for bounded degree interaction the support of $e^ < igamma_1 H >e^ < ieta_1 B >hat_k e^ < -ieta_1 B >e^< -igamma_1 H >$ only expands by an amount given by the degree of the interaction in $H$ and is therefore independent of the system size. This means that for these smaller sub problems the expectation values are independent of $n$ and can be evaluated classically. The case of a general degree $3$ is considered in 3.

This is a general observation, which means that if we have a problem where the circuit used for the trial state preparation only increases the support of each term in the Hamiltonian by a constant amount the cost function can be directly evaluated.

When this is the case, and only a few parameters $eta, gamma$ are needed in the preparation of the trial state, these can be found easily by a simple grid search. Furthermore, an exact optimal value of $M_p$ can be used to bound the approximation ratio

to obtain an estimate of $alpha$. For this case the QAOA algorithm has the same characteristics as a conventional approximate optimization algorithm that comes with a guaranteed approximation ratio that can be obtained with polynomial efficiency in the problem size.

#### Evaluation on a quantum computer

When the quantum circuit becomes too deep to be evaluated classically, or when the connectivity of the Problem Hamiltonian is too high we can resort to other means of estimating the expectation value. This involves directly estimating $F_p(vec,vec<eta>)$ on the quantum computer. The approach here follows the path of the conventional expectation value estimation as used in VQE 4, where a trial state $| psi_p(vec,vec<eta>) angle$ is prepared directly on the quantum computer and the expectation value is obtained from sampling.

Since QAOA has a diagonal Hamiltonian $H$ it is actually straight forward to estimate the expectation value. We only need to obtain samples from the trial state in the computational basis. Recall that $H = sum_^n> C(x) |x anglelangle x|$ so that we can obtain the sampling estimate of

by repeated single qubit measurements of the state $| psi_p(vec,vec<eta>) angle$ in the $Z$ basis. For every bit string $x$ obtained from the distribution $|langle x| psi_p(vec,vec<eta>) angle |^2$ we evaluate the cost function $C(x)$ and average it over the total number of samples. The resulting empirical average approximates the expectation value up to an additive sampling error that lies within the variance of the state. The variance will be discussed below.

With access to the expectation value, we can now run a classical optimization algorithm, such as 6, to optimize the $F_p$.

While this approach does not lead to an a-priori approximation guarantee for $x^*$, the optimized function value can be used later to provide an estimate for the approximation ratio $alpha$.

### 4.3.3 Obtaining a solution with a given approximation ratio with high probability

The algorithm is probabilistic in nature and produces random bit strings from the distribution $|langle x| psi_p(vec,vec<eta>) angle |^2$. So how can we be sure that we will sample an approximation $x^*$ that is close to the value of the optimized expectation value $M_p$? Note that this question is also relevant to the estimation of $M_p$ on a quantum computer in the first place. If the samples drawn from $|langle x| psi_p(vec,vec<eta>) angle |^2$ have too much variance, many samples are necessary to determine the mean.

We will draw a bit string $x^*$ that is close to the mean $M_p$ with high probability when the energy as variable has little variance.

Note that the number of terms in the Hamiltonian $H = sum_^m hat_k$ are bounded by $m$. Say each individual summand $hat_k$ has an operator norm that can be bounded by a universal constant $|hat_k| leq ilde$ for all $k = 1ldots m$. Then consider

where we have used that $langle psi_p(vec,vec<eta>)|hat_k hat_l |psi_p(vec,vec<eta>) angle leq ilde^2$.

This means that the variance of any expectation $F_p(vec,vec<eta>)$ is bounded by $m^2 ilde^2$. Hence this in particular applies for $M_p$. Furthermore if $m$ only grows polynomially in the number of qubits $n$, we know that taking polynomially growing number of samples $s = Oleft(frac< ilde^2 m^2> ight)$ from $|langle x| psi_p(vec,vec<eta>) angle |^2$ will be sufficient to obtain a $x^*$ that leads to an $C(x^*)$ that will be close to $M_p$.

## Kmeans on Image Compression

In this part, we’ll implement kmeans to compress an image. The image that we’ll be working on is 396 x 396 x 3. Therefore, for each pixel location we would have 3 8-bit integers that specify the red, green, and blue intensity values. Our goal is to reduce the number of colors to 30 and represent (compress) the photo using those 30 colors only. To pick which colors to use, we’ll use kmeans algorithm on the image and treat every pixel as a data point. That means reshape the image from height x width x channels to (height * width) x channel, i,e we would have 396 x 396 = 156,816 data points in 3-dimensional space which are the intensity of RGB. Doing so will allow us to represent the image using the 30 centroids for each pixel and would significantly reduce the size of the image by a factor of 6. The original image size was 396 x 396 x 24 = 3,763,584 bits however, the new compressed image would be 30 x 24 + 396 x 396 x 4 = 627,984 bits. The huge difference comes from the fact that we’ll be using centroids as a lookup for pixels’ colors and that would reduce the size of each pixel location to 4-bit instead of 8-bit.

From now on we will be using sklearn implementation of kmeans. Few thing to note here:

• n_init is the number of times of running the kmeans with different centroid’s initialization. The result of the best one will be reported.
• tol is the within-cluster variation metric used to declare convergence.
• The default of init is k-means++ which is supposed to yield a better results than just random initialization of centroids. We can see the comparison between the original image and the compressed one. The compressed image looks close to the original one which means we’re able to retain the majority of the characteristics of the original image. With smaller number of clusters we would have higher compression rate at the expense of image quality. As a side note, this image compression method is called lossy data compression because we can’t reconstruct the original image from the compressed image.

## Section 5 - Other Clinical Trial-related Attachments

#### Who must complete "Section 5 - Other Clinical Trial-related Attachments:"

If you answered "Yes" to all the questions in the "Clinical Trial Questionnaire:" Include an attachment only if your FOA specifies that an attachment(s) is required or permitted otherwise, do not include any Other Clinical Trial-related attachments.

If you answered "No" to any question in the "Clinical Trial Questionnaire:" Do not provide information in this section. Inputting information in this section will result in errors and will prevent your application from being accepted.

R25 applicants who are proposing to provide clinical trial research experience for their participants (i.e., participants will not be leading an independent clinical trial): Do not provide information in "Section 5 - Other Clinical Trial-related Attachments." Inputting information in this section will result in errors and will prevent your application from being accepted.

R36 applicants who are proposing to gain clinical trial research experience under a mentor's supervision (i.e., you will not be leading an independent clinical trial): Do not provide information in "Section 5 - Other Clinical Trial-related Attachments." Inputting information in this section will result in errors and will prevent your application from being accepted.

###### Additional Instructions for Career Development:

CDA applicants who are proposing to gain clinical trial research experience under a mentor's supervision (i.e., you will not be leading an independent clinical trial): Do not provide information in "Section 5 - Other Clinical Trial-related Attachments." Inputting information in this section will result in errors and will prevent your application from being accepted.

K12 and D43 applicants who are proposing to provide clinical trial research experience for their Scholars/Trainees (i.e., Scholars/Trainees will not be leading an independent clinical trial): At the time of your application, do not provide information in "Section 5 - Other Clinical Trial-related Attachments." Inputting information in this section will result in errors and will prevent your application from being accepted. Post-award, while you will be required to fill out Study Records, you must still not provide information in "Section 5 - Other Clinical Trial-related Attachments."

Fellowship applicants proposing to gain clinical trial research experience under a sponsor's supervision (i.e., you will not be leading an independent clinical trial): Do not provide information in "Section 5 - Other Clinical Trial-related Attachments." Inputting information in this section will result in errors and will prevent your application from being accepted.

### 5.1 Other Clinical Trial-related Attachments

#### Format:

Attach this information as a PDF file. See NIH's Format Attachments page.

A maximum of 10 PDF attachments is allowed in the "Other Clinical Trial-related Attachments" section.

#### Content:

Provide additional trial-related information only if your FOA specifically requests it. Include only attachments requested in the FOA, and use requested filenames. If a specific filename is not given in the FOA, use a meaningful filename since it will become a bookmark in the assembled application image.

## 50 U.S. Code § 1881b - Certain acquisitions inside the United States targeting United States persons outside the United States

The Foreign Intelligence Surveillance Court shall have jurisdiction to review an application and to enter an order approving the targeting of a United States person reasonably believed to be located outside the United States to acquire foreign intelligence information, if the acquisition constitutes electronic surveillance or the acquisition of stored electronic communications or stored electronic data that requires an order under this chapter, and such acquisition is conducted within the United States.

If a United States person targeted under this subsection is reasonably believed to be located in the United States during the effective period of an order issued pursuant to subsection (c), an acquisition targeting such United States person under this section shall cease unless the targeted United States person is again reasonably believed to be located outside the United States while an order issued pursuant to subsection (c) is in effect. Nothing in this section shall be construed to limit the authority of the Government to seek an order or authorization under, or otherwise engage in any activity that is authorized under, any other subchapter of this chapter.

The Attorney General may require any other affidavit or certification from any other officer in connection with the application.

The judge may require the applicant to furnish such other information as may be necessary to make the findings required by subsection (c)(1).

In determining whether or not probable cause exists for purposes of paragraph (1)(B), a judge having jurisdiction under subsection (a)(1) may consider past activities of the target and facts and circumstances relating to current or future activities of the target. No United States person may be considered a foreign power, agent of a foreign power, or officer or employee of a foreign power solely upon the basis of activities protected by the first amendment to the Constitution of the United States.

Review by a judge having jurisdiction under subsection (a)(1) shall be limited to that required to make the findings described in paragraph (1).

If the judge determines that the facts submitted under subsection (b) are insufficient to establish probable cause under paragraph (1)(B), the judge shall enter an order so stating and provide a written statement for the record of the reasons for the determination. The Government may appeal an order under this subparagraph pursuant to subsection (f).

If the judge determines that the proposed minimization procedures referred to in paragraph (1)(C) do not meet the definition of minimization procedures under section 1801(h) or 1821(4) of this title, as appropriate, the judge shall enter an order so stating and provide a written statement for the record of the reasons for the determination. The Government may appeal an order under this subparagraph pursuant to subsection (f).

If the judge determines that an application pursuant to subsection (b) does not contain all of the required elements, or that the certification or certifications are clearly erroneous on the basis of the statement made under subsection (b)(1)(F)(v) and any other information furnished under subsection (b)(3), the judge shall enter an order so stating and provide a written statement for the record of the reasons for the determination. The Government may appeal an order under this subparagraph pursuant to subsection (f).

An order approved under this subsection shall be effective for a period not to exceed 90 days and such order may be renewed for additional 90-day periods upon submission of renewal applications meeting the requirements of subsection (b).

At or prior to the end of the period of time for which an acquisition is approved by an order or extension under this section, the judge may assess compliance with the minimization procedures referred to in paragraph (1)(C) by reviewing the circumstances under which information concerning United States persons was acquired, retained, or disseminated.

If the Attorney General authorizes an acquisition under paragraph (1), the Attorney General shall require that the minimization procedures referred to in subsection (c)(1)(C) for the issuance of a judicial order be followed.

In the absence of a judicial order approving an acquisition under paragraph (1), such acquisition shall terminate when the information sought is obtained, when the application for the order is denied, or after the expiration of 7 days from the time of authorization by the Attorney General, whichever is earliest.

If an application for approval submitted pursuant to paragraph (1) is denied, or in any other case where the acquisition is terminated and no order is issued approving the acquisition, no information obtained or evidence derived from such acquisition, except under circumstances in which the target of the acquisition is determined not to be a United States person, shall be received in evidence or otherwise disclosed in any trial, hearing, or other proceeding in or before any court, grand jury, department, office, agency, regulatory body, legislative committee, or other authority of the United States, a State, or political subdivision thereof, and no information concerning any United States person acquired from such acquisition shall subsequently be used or disclosed in any other manner by Federal officers or employees without the consent of such person, except with the approval of the Attorney General if the information indicates a threat of death or serious bodily harm to any person.

No cause of action shall lie in any court against any electronic communication service provider for providing any information, facilities, or assistance in accordance with an order or request for emergency assistance issued pursuant to subsection (c) or (d), respectively.

The Government may file a petition with the Foreign Intelligence Surveillance Court of Review for review of an order issued pursuant to subsection (c). The Court of Review shall have jurisdiction to consider such petition and shall provide a written statement for the record of the reasons for a decision under this paragraph.

The Government may file a petition for a writ of certiorari for review of a decision of the Court of Review issued under paragraph (1). The record for such review shall be transmitted under seal to the Supreme Court of the United States , which shall have jurisdiction to review such decision.

Except as provided in this section, nothing in this chapter shall be construed to require an application for a court order for an acquisition that is targeted in accordance with this section at a United States person reasonably believed to be located outside the United States.

Pub. L. 110–261, title IV, § 403(b)(1), July 10, 2008 , 122 Stat. 2474, as amended by Pub. L. 112–238, § 2(a)(1), Dec. 30, 2012 , 126 Stat. 1631 Pub. L. 115–118, title II, § 201(a)(1), Jan. 19, 2018 , 132 Stat. 19, provided that, except as provided in section 404 of Pub. L. 110–261, set out as a note under section 1801 of this title, effective Dec. 31, 2023 , this section is repealed.

This chapter, referred to in subsecs. (a), (d)(1), and (g), was in the original “this Act”, meaning Pub. L. 95–511, Oct. 25, 1978 , 92 Stat. 1783, which is classified principally to this chapter. For complete classification of this Act to the Code, see Short Title note set out under section 1801 of this title and Tables.

Pub. L. 110–261, title IV, § 403(b)(1), July 10, 2008 , 122 Stat. 2474, as amended by Pub. L. 112–238, § 2(a)(1), Dec. 30, 2012 , 126 Stat. 1631 Pub. L. 115–118, title II, § 201(a)(1), Jan. 19, 2018 , 132 Stat. 19, provided that, except as provided in section 404 of Pub. L. 110–261, set out as a Transition Procedures note under section 1801 of this title, the repeals made by section 403(b)(1) are effective Dec. 31, 2023 .

## Introduction

Nonlinear optical (NLO) materials, which can halve the wavelength of light (or double the frequency) by second-harmonic generation (SHG) process, are of current interest and great importance in laser science and technology 1,2,3 . Over the past decades, continuous intensive studies 4,5,6,7,8,9,10,11 have resulted in the development of various commercial NLO materials, such as β-BaB2O4 (BBO) 12 , LiB3O5 (ref. 13), AgGaS2 (ref. 14) and ZnGeP2 (ref. 15), which are applicable for the generation of coherent light from ultraviolet region to infrared region. However, there is still lack of commercially available NLO materials for the generation of deep-ultraviolet (wavelength below 200 nm) coherent light, limited by the fundamental but conflicting requirements on the structure-directing optical properties 16 : a wide transparency window down to the deep-ultraviolet spectral region, a large SHG response and a sufficient birefringence to achieve phase matchability. Traditionally, the search for deep-ultraviolet NLO materials mainly focused on beryllium borate systems owing to their deep-ultraviolet transparency, thus leading to the discovery of a number of beryllium borates, such as KBe2BO3F2 (KBBF) 17,18 , SrBe2B2O7 (ref. 19), Na2CsBe6B5O15 (ref. 20), NaCaBe2B2O6F (ref. 21), Na3Sr3Be3B3O9F4 (ref. 22), NaBeB3O6 and ABe2B3O7 (A=K, Rb) 23 . Nevertheless, till now KBBF is the sole material that can practically generate deep-ultraviolet coherent light by direct SHG process. In the structure of KBBF, the NLO-active [BO3] 3− groups in the [Be2BO3F2] layers are coplanar and aligned, giving rise to a relatively large SHG response and a sufficient birefringence for the generation of deep-ultraviolet coherent light. Unfortunately, KBBF suffers a strong layering tendency that originates from the weak F − –K + ionic interactions between the adjacent [Be2BO3F2] layers, which causes a great difficulty in the growth of thick crystals and thereby severely hinders the NLO performance of KBBF. Moreover, the containing beryllium can cause pneumonia-like symptoms and cancer if inhaled in this sense, KBBF is not environmentally friendly. Owing to these obstacles, the production of KBBF is still at the stage of laboratory. Therefore, it is urgently demanded to develop the next generation of deep-ultraviolet NLO materials that preserve the merits of KBBF while overcoming the demerits.

Here we report a beryllium-free borate, Li4Sr(BO3)2, whose structure features [SrBO3] layers bridged by NLO-active [BO3] 3− groups. The [SrBO3] layers afford [BO3] 3− groups arranged in a manner similar to that in the case of the [Be2BO3F2] layers in KBBF, conferring Li4Sr(BO3)2 the optical merits of KBBF. Furthermore, the NLO-active [BO3] 3− groups serving as layer connectors greatly mitigate the layering tendency and, simultaneously, help to enhance the SHG efficiency by more than half as compared with that of KBBF.

## 3.2: Minimization Applications

HPE will buy Zerto for $374 million and use its continuous data protection capabilities to bring disaster recovery, backup and . Business continuity managers must be ready for anything, and that includes interviewing for the role. Here are some common . A BCDR training program teaches employees key skills, such as how to conduct risk assessments, coordinate emergency response with. Micron's$900 million deal to sell its Utah semiconductor factory to Texas Instruments will end the plant's 3D XPoint supplies .

Explore a group of Optane-ready products to improve storage performance. In addition, there are other options beyond self-managed.

Nvidia GPUDirect Storage's driver hit 1.0 status to enable direct memory access between GPU and storage and boost the performance.

Dell VxRail hardware and software updates improve performance and ease deployment and management, while enabling discrete compute.

The VMware vSAN storage update aims to help enterprises start with a small HCI deployment. Customers gain the option to connect .

SimpliVity has added integration with HPE Cloud Volumes Backup and HPE StoreOnce to enable easier backup at the edge, as well as .

Dell Technology's segment of Global Alliances partners stood out in the vendor's first-quarter channel sales more IT channel .

Cybersecurity industry standards can help MSPs mitigate internal security risks. Here are the benefits and challenges of adopting.

At its annual global partner conference, Microsoft will reveal its key priorities for the upcoming year. Read on for news and .