Rosenbrock example

In this example we consider a python implementation of the following optimization problem:

\[\begin{split}\begin{align} \text{min} \qquad & 100(x_2 - x_1^2)^2 + (1 - x_1)^2 \\ \text{with respect to} \qquad & -2 \le x_{i} \le 2 \qquad i = 1, 2\\ \text{subject to} \qquad & x_{1} + x_{2} + 5 \ge 0 \\ \end{align}\end{split}\]

Python implementation

The python implementation of this problem is as follows

# Import some utilities
import numpy as np
import mpi4py.MPI as MPI
import matplotlib.pyplot as plt

# Import ParOpt
from paropt import ParOpt

# Create the rosenbrock function class
class Rosenbrock(ParOpt.Problem):
    def __init__(self):
        # Set the communicator pointer
        self.comm = MPI.COMM_WORLD
        self.nvars = 2
        self.ncon = 1

        # The design history file
        self.x_hist = []

        # Initialize the base class
        super(Rosenbrock, self).__init__(self.comm, self.nvars, self.ncon)

        return

    def getVarsAndBounds(self, x, lb, ub):
        '''Set the values of the bounds'''
        x[:] = -1.0
        lb[:] = -2.0
        ub[:] = 2.0
        return

    def evalObjCon(self, x):
        '''Evaluate the objective and constraint'''
        # Append the point to the solution history
        self.x_hist.append(np.array(x))

        # Evaluate the objective and constraints
        fail = 0
        con = np.zeros(1)
        fobj = 100*(x[1]-x[0]**2)**2 + (1-x[0])**2
        con[0] = x[0] + x[1] + 5.0
        return fail, fobj, con

    def evalObjConGradient(self, x, g, A):
        '''Evaluate the objective and constraint gradient'''
        fail = 0

        # The objective gradient
        g[0] = 200*(x[1]-x[0]**2)*(-2*x[0]) - 2*(1-x[0])
        g[1] = 200*(x[1]-x[0]**2)

        # The constraint gradient
        A[0][0] = 1.0
        A[0][1] = 1.0
        return fail

problem = Rosenbrock()

options = {'algorithm': 'ip'}
opt = ParOpt.Optimizer(problem, options)
opt.optimize()

This code produces the following output in the default file "paropt.out":

ParOptInteriorPoint Parameter Summary:
algorithm                                ip

iter nobj ngrd nhvc   alpha   alphx   alphz         fobj   |opt| |infes|  |dual|      mu    comp   dmerit     rho info
   0    1    1    0      --      --      --  4.04000e+02 8.5e+02 9.4e+02 8.4e+05 1.7e+05 1.7e+05       --      --
   1    3    2    0 3.5e-01 1.0e-03 1.0e-01  9.02442e+00 1.6e+02 9.4e+02 8.7e+05 4.2e+04 1.6e+05 -5.0e+05 0.0e+00
   2    4    3    0 1.0e+00 8.3e-01 1.0e+00  1.11146e+03 1.8e+03 1.6e+02 2.8e+05 1.0e+04 9.1e+04 -7.2e+05 0.0e+00
   3    5    4    0 1.0e+00 1.0e+00 1.0e+00  2.84010e+02 1.0e+03 3.7e-01 1.7e+04 2.6e+03 1.1e+04 -4.2e+05 0.0e+00
   4    6    5    0 1.0e+00 1.0e+00 1.0e+00  6.92825e+01 5.1e+02 5.3e-15 3.5e+03 6.5e+02 2.7e+03 -1.9e+04 0.0e+00
   5    7    6    0 1.0e+00 1.0e+00 1.0e+00  2.07229e+01 1.0e+02 8.9e-16 9.2e+02 1.6e+02 6.5e+02 -2.4e+03 0.0e+00
   6    8    7    0 1.0e+00 1.0e+00 1.0e+00  2.68808e+00 4.8e+01 2.8e-17 1.5e+02 4.1e+01 1.6e+02 -8.5e+01 0.0e+00
   7    9    8    0 1.0e+00 1.0e+00 1.0e+00  2.34024e-01 1.4e+01 2.8e-16 3.4e+01 1.0e+01 4.1e+01 -8.6e+01 0.0e+00
   8   10    9    0 1.0e+00 1.0e+00 1.0e+00  1.12205e-01 1.9e+00 1.4e-15 7.9e+00 2.6e+00 1.0e+01 -2.1e+01 0.0e+00
   9   11   10    0 1.0e+00 1.0e+00 1.0e+00  1.12286e-01 4.5e-02 2.5e-16 1.9e+00 6.4e-01 2.6e+00 -5.6e+00 0.0e+00

iter nobj ngrd nhvc   alpha   alphx   alphz         fobj   |opt| |infes|  |dual|      mu    comp   dmerit     rho info
  10   12   11    0 1.0e+00 1.0e+00 1.0e+00  1.11649e-01 2.8e-01 1.2e-15 4.8e-01 1.6e-01 6.4e-01 -1.4e+00 0.0e+00
  11   13   12    0 1.0e+00 1.0e+00 1.0e+00  1.09062e-01 8.3e-01 7.2e-16 1.2e-01 4.0e-02 1.6e-01 -3.6e-01 0.0e+00
  12   14   13    0 1.0e+00 1.0e+00 1.0e+00  1.04593e-01 1.5e+00 1.4e-15 1.2e-03 4.0e-02 4.0e-02 -9.5e-02 0.0e+00
  13   15   14    0 1.0e+00 1.0e+00 1.0e+00  9.69024e-02 2.5e+00 2.8e-17 7.7e-06 4.0e-02 4.0e-02 -9.8e-03 0.0e+00
  14   16   15    0 1.0e+00 1.0e+00 1.0e+00  8.39219e-02 3.4e+00 1.7e-16 3.7e-05 4.0e-02 4.0e-02 -1.7e-02 0.0e+00
  15   17   16    0 1.0e+00 1.0e+00 1.0e+00  6.08092e-02 3.3e+00 3.2e-16 8.3e-05 4.0e-02 4.0e-02 -2.9e-02 0.0e+00
  16   18   17    0 1.0e+00 1.0e+00 1.0e+00  3.10043e-02 7.5e-01 2.3e-16 1.3e-04 4.0e-02 4.0e-02 -4.2e-02 0.0e+00
  17   19   18    0 1.0e+00 1.0e+00 1.0e+00  1.78352e-02 2.5e+00 2.6e-16 2.5e-04 4.0e-02 4.0e-02 -2.0e-02 0.0e+00
  18   20   19    0 1.0e+00 1.0e+00 1.0e+00  8.31785e-03 4.6e-01 4.6e-16 5.9e-05 4.0e-02 4.0e-02 -1.3e-02 0.0e+00
  19   21   20    0 1.0e+00 1.0e+00 1.0e+00  3.40797e-03 1.2e+00 8.2e-16 1.7e-04 4.0e-02 4.0e-02 -5.7e-03 0.0e+00

iter nobj ngrd nhvc   alpha   alphx   alphz         fobj   |opt| |infes|  |dual|      mu    comp   dmerit     rho info
  20   22   21    0 1.0e+00 1.0e+00 1.0e+00  1.73702e-03 1.3e-02 6.3e-16 3.0e-02 1.0e-02 4.0e-02 -2.4e-03 0.0e+00
  21   23   22    0 1.0e+00 1.0e+00 1.0e+00  3.31850e-04 3.3e-01 1.4e-15 1.3e-03 1.0e-02 1.0e-02 -2.4e-02 0.0e+00
  22   24   23    0 1.0e+00 1.0e+00 1.0e+00  1.17623e-04 8.6e-02 9.6e-17 7.5e-03 2.5e-03 1.0e-02 -2.1e-04 0.0e+00
  23   25   24    0 1.0e+00 1.0e+00 1.0e+00  8.89746e-06 8.3e-02 1.4e-16 1.2e-04 2.5e-03 2.5e-03 -5.8e-03 0.0e+00
  24   26   25    0 1.0e+00 1.0e+00 1.0e+00  4.74120e-06 1.9e-02 2.7e-16 1.9e-03 6.2e-04 2.5e-03 -1.1e-05 0.0e+00
  25   27   26    0 1.0e+00 1.0e+00 1.0e+00  3.36109e-07 8.3e-04 3.0e-16 4.7e-04 1.6e-04 6.2e-04 -1.4e-03 0.0e+00
  26   28   27    0 1.0e+00 1.0e+00 1.0e+00  1.52109e-08 4.4e-04 6.8e-17 1.2e-04 3.9e-05 1.6e-04 -3.5e-04 0.0e+00
  27   29   28    0 1.0e+00 1.0e+00 1.0e+00  9.47900e-10 2.3e-05 1.2e-15 2.9e-05 9.7e-06 3.9e-05 -8.8e-05 0.0e+00
  28   30   29    0 1.0e+00 1.0e+00 1.0e+00  5.87613e-11 5.4e-07 4.8e-17 7.3e-06 2.4e-06 9.7e-06 -2.2e-05 0.0e+00
  29   31   30    0 1.0e+00 1.0e+00 1.0e+00  3.66518e-12 3.6e-08 4.5e-16 1.8e-06 6.1e-07 2.4e-06 -5.5e-06 0.0e+00

iter nobj ngrd nhvc   alpha   alphx   alphz         fobj   |opt| |infes|  |dual|      mu    comp   dmerit     rho info
  30   32   31    0 1.0e+00 1.0e+00 1.0e+00  2.29012e-13 3.6e-09 5.6e-16 4.6e-07 1.5e-07 6.1e-07 -1.4e-06 0.0e+00
  31   33   32    0 1.0e+00 1.0e+00 1.0e+00  1.30833e-14 2.4e-10 9.5e-16 4.6e-08 1.0e-07 1.5e-07 -3.5e-07 0.0e+00

ParOpt: Successfully converged to requested tolerance The output from ParOptInteriorPoint consists of a summary of the non-default parameter values. Note that the "ParOptOptimizer" interface is used to run this example. This class provides a common interface for all optimizers in ParOpt and therefore the file lists all options for all the optimizers, including the trust region method and the method of moving asymptotes. Following the option settings, the output consists of a summary of the iteration history with the following columns:

  • iter the current iteration

  • nobj the number of objective and constraint evaluations

  • ngrd the number of objective and constraint gradient evaluations

  • nhvc the number of Hessian-vector products

  • alpha the step size

  • alphax the scaling factor applied to the primal components of the step

  • alphaz the scaling factor applied to the dual components of the step

  • fobj the objective value

  • |opt| the optimality error in the specified norm

  • |infeas| the infeasibility in the specified norm

  • |dual| the error in the complementarity equation

  • mu the barrier parameter

  • comp the complementarity

  • dmerit the derivative of the merit function

  • rho the value of the penalty parameter in the line search

  • info additional information, usually about the quasi-Newton Hessian update or line search

The file can be visualized using the example in examples/plot_history/plot_history.py. This output can be unpacked from the file using:

values = paropt.unpack_output('paropt.out')

The trust region variant of the algorithm, that is used as the default setting, can be run by modifying the options:

options = {'algorithm': tr}

This produces the following result in the output in "paropt.tr":

ParOptTrustRegion Parameter Summary:
algorithm                                tr
use_quasi_newton_update                  False
write_output_frequency                   0

iter         fobj    infeas        l1    linfty  |x - xk|        tr       rho  mod red.     avg z     max z  avg pen.  max pen. info
   0  4.04000e+02  0.00e+00  9.61e+02  6.19e+02  1.00e-01  1.50e-01  8.97e-01  1.20e+02  4.40e-08  4.40e-08  1.00e+03  1.00e+03 21/14
   1  2.96020e+02  0.00e+00  6.60e+02  3.97e+02  1.50e-01  2.25e-01  1.03e+00  1.17e+02  4.00e-08  4.00e-08  1.00e+03  1.00e+03 21/14
   2  1.75328e+02  0.00e+00  3.31e+02  1.71e+02  2.25e-01  3.38e-01  1.12e+00  9.75e+01  3.56e-08  3.56e-08  1.00e+03  1.00e+03 21/14
   3  6.64257e+01  0.00e+00  1.10e+02  6.40e+01  3.21e-01  5.06e-01  1.31e+00  4.15e+01  3.16e-08  3.16e-08  1.00e+03  1.00e+03 21/14
   4  1.20441e+01  0.00e+00  2.32e+01  1.34e+01  2.09e-01  7.59e-01  1.19e+00  8.36e+00  2.99e-08  2.99e-08  1.00e+03  1.00e+03 21/14
   5  2.05970e+00  0.00e+00  3.00e+00  2.67e+00  5.55e-02  1.00e+00  1.07e+00  4.66e-01  2.94e-08  2.94e-08  1.00e+03  1.00e+03 21/14
   6  1.56179e+00  0.00e+00  2.83e+00  2.16e+00  6.60e-03  1.00e+00  1.72e+00  9.09e-03  2.94e-08  2.94e-08  1.00e+03  1.00e+03 21/14
   7  1.54617e+00  0.00e+00  4.12e+00  2.91e+00  3.53e-02  1.00e+00  1.67e+00  3.97e-02  2.92e-08  2.92e-08  1.00e+03  1.00e+03 21/14
   8  1.47972e+00  0.00e+00  4.75e+00  2.97e+00  1.52e-01  1.00e+00  2.30e+00  1.50e-01  2.45e-08  2.45e-08  1.00e+03  1.00e+03 skipH 21/14
   9  1.13633e+00  0.00e+00  1.21e+01  1.07e+01  2.00e-01  1.00e+00  4.55e-01  2.53e-01  2.51e-08  2.51e-08  1.00e+03  1.00e+03 21/14

iter         fobj    infeas        l1    linfty  |x - xk|        tr       rho  mod red.     avg z     max z  avg pen.  max pen. info
  10  1.02106e+00  0.00e+00  2.03e+00  1.94e+00  1.16e-01  1.00e+00  2.96e-01  2.59e-01  2.82e-08  2.82e-08  1.00e+03  1.00e+03 21/14
  11  9.44283e-01  0.00e+00  3.86e+00  2.35e+00  4.52e-02  1.00e+00  1.66e+00  4.35e-02  2.80e-08  2.80e-08  1.00e+03  1.00e+03 21/14
  12  8.72328e-01  0.00e+00  8.91e+00  7.93e+00  9.37e-02  1.00e+00  3.37e-01  6.44e-02  2.75e-08  2.75e-08  1.00e+03  1.00e+03 21/14
  13  8.50592e-01  0.00e+00  4.77e+00  4.13e+00  3.48e-02  1.00e+00  1.08e+00  5.10e-02  2.77e-08  2.77e-08  1.00e+03  1.00e+03 21/14
  14  7.95434e-01  0.00e+00  3.84e+00  3.02e+00  1.42e-02  1.00e+00  1.83e+00  2.42e-02  2.75e-08  2.75e-08  1.00e+03  1.00e+03 21/14
  15  7.51180e-01  0.00e+00  2.77e+00  2.70e+00  1.32e-01  1.00e+00  1.50e+00  1.42e-01  2.66e-08  2.66e-08  1.00e+03  1.00e+03 21/14
  16  5.38497e-01  0.00e+00  1.29e+01  7.66e+00  1.40e-01  1.00e+00  5.75e-01  9.45e-02  2.55e-08  2.55e-08  1.00e+03  1.00e+03 21/15
  17  4.84154e-01  0.00e+00  1.33e+00  8.31e-01  2.75e-02  1.00e+00  1.07e+00  1.26e-01  2.55e-08  2.55e-08  1.00e+03  1.00e+03 21/14
  18  3.48437e-01  0.00e+00  1.59e+00  1.31e+00  8.28e-02  1.00e+00  1.74e+00  5.06e-02  2.48e-08  2.48e-08  1.00e+03  1.00e+03 21/14
  19  2.60394e-01  0.00e+00  1.59e+00  1.31e+00  0.00e+00  2.50e-01 -2.47e+00  1.03e-01  2.31e-08  2.31e-08  1.00e+03  1.00e+03 21/14

iter         fobj    infeas        l1    linfty  |x - xk|        tr       rho  mod red.     avg z     max z  avg pen.  max pen. info
  20  2.60394e-01  0.00e+00  1.95e+00  1.45e+00  6.85e-02  3.75e-01  1.57e+00  3.69e-02  2.43e-08  2.43e-08  1.00e+03  1.00e+03 21/14
  21  2.02340e-01  0.00e+00  1.08e+01  6.16e+00  1.87e-01  5.62e-01  9.16e-01  7.48e-02  2.29e-08  2.29e-08  1.00e+03  1.00e+03 21/14
  22  1.33863e-01  0.00e+00  1.08e+01  6.16e+00  0.00e+00  1.41e-01  2.00e-01  4.59e-02  2.11e-08  2.11e-08  1.00e+03  1.00e+03 21/14
  23  1.33863e-01  0.00e+00  1.95e+00  1.08e+00  3.01e-02  2.11e-01  9.51e-01  3.57e-02  2.08e-08  2.08e-08  1.00e+03  1.00e+03 21/14
  24  9.99659e-02  0.00e+00  1.35e+00  7.99e-01  2.92e-02  3.16e-01  1.86e+00  7.19e-03  2.29e-08  2.29e-08  1.00e+03  1.00e+03 21/14
  25  8.66158e-02  0.00e+00  1.20e+01  7.47e+00  2.05e-01  3.16e-01  2.99e-01  4.26e-02  2.17e-08  2.17e-08  1.00e+03  1.00e+03 21/14
  26  7.38950e-02  0.00e+00  9.08e-01  7.20e-01  8.04e-02  3.16e-01  5.39e-01  5.39e-02  1.82e-08  1.82e-08  1.00e+03  1.00e+03 21/14
  27  4.48191e-02  0.00e+00  4.32e-01  2.96e-01  5.48e-02  4.75e-01  1.79e+00  7.61e-03  2.19e-08  2.19e-08  1.00e+03  1.00e+03 21/14
  28  3.12023e-02  0.00e+00  4.32e-01  2.96e-01  0.00e+00  1.19e-01 -2.00e+00  2.31e-02  2.08e-08  2.08e-08  1.00e+03  1.00e+03 21/14
  29  3.12023e-02  0.00e+00  2.26e-01  1.39e-01  5.46e-02  1.78e-01  1.76e+00  5.91e-03  2.16e-08  2.16e-08  1.00e+03  1.00e+03 21/14

iter         fobj    infeas        l1    linfty  |x - xk|        tr       rho  mod red.     avg z     max z  avg pen.  max pen. info
  30  2.08028e-02  0.00e+00  2.26e-01  1.39e-01  0.00e+00  4.45e-02  1.73e-01  1.64e-02  2.07e-08  2.07e-08  1.00e+03  1.00e+03 21/14
  31  2.08028e-02  0.00e+00  2.12e-01  1.79e-01  4.45e-02  6.67e-02  1.25e+00  5.38e-03  2.14e-08  2.14e-08  1.00e+03  1.00e+03 21/14
  32  1.40946e-02  0.00e+00  1.25e+00  7.53e-01  6.67e-02  1.00e-01  9.35e-01  7.50e-03  2.11e-08  2.11e-08  1.00e+03  1.00e+03 21/14
  33  7.07496e-03  0.00e+00  3.65e+00  2.37e+00  7.35e-02  1.00e-01  5.32e-01  2.76e-03  2.06e-08  2.06e-08  1.00e+03  1.00e+03 21/14
  34  5.60569e-03  0.00e+00  1.67e+00  1.06e+00  1.58e-02  1.50e-01  1.26e+00  1.79e-03  1.94e-08  1.94e-08  1.00e+03  1.00e+03 21/14
  35  3.35667e-03  0.00e+00  5.38e-01  3.26e-01  1.84e-02  2.25e-01  1.51e+00  1.05e-03  2.03e-08  2.03e-08  1.00e+03  1.00e+03 21/14
  36  1.76414e-03  0.00e+00  6.54e-02  3.54e-02  4.14e-02  3.38e-01  1.39e+00  9.90e-04  2.07e-08  2.07e-08  1.00e+03  1.00e+03 21/14
  37  3.92289e-04  0.00e+00  3.63e-01  2.40e-01  3.34e-02  5.07e-01  1.04e+00  3.35e-04  2.04e-08  2.04e-08  1.00e+03  1.00e+03 21/14
  38  4.46965e-05  0.00e+00  1.70e-02  1.05e-02  3.49e-03  7.60e-01  1.09e+00  3.97e-05  2.05e-08  2.05e-08  1.00e+03  1.00e+03 21/14
  39  1.49333e-06  0.00e+00  3.00e-03  2.00e-03  2.38e-03  1.00e+00  1.01e+00  1.48e-06  2.04e-08  2.04e-08  1.00e+03  1.00e+03 21/14

iter         fobj    infeas        l1    linfty  |x - xk|        tr       rho  mod red.     avg z     max z  avg pen.  max pen. info
  40  2.52590e-09  0.00e+00  7.23e-05  4.88e-05  1.12e-05  1.00e+00  9.78e-01  2.58e-09  2.04e-08  2.04e-08  1.00e+03  1.00e+03 21/14
  41  2.10651e-12  0.00e+00  2.52e-09  1.56e-09  1.64e-06  1.00e+00  1.00e+00  2.11e-12  2.04e-08  2.04e-08  1.00e+03  1.00e+03 21/14

The trust region output again outputs the parameter values. Next, the file contains the iteration history with the following columns

  • iter the current iteration

  • fobj the objective value

  • infeas the l1 norm of the constraint violation

  • l1 the l1 norm of the optimality conditions

  • linfty the l-infinity norm of the optimality conditions

  • |x - xk| the step lenth

  • tr the trust region radius

  • rho the ratio of the actual improvement to the expected improvement used in the acceptance criteria

  • mod red. the predicted model reduction

  • avg z the average multiplier

  • max z the maximum multiplier

  • avg pen. the average penalty value

  • max pen. the maximum penalty value

  • info information about the interior-point subproblem solution (iteration for the step/iteration for the penalty problem)