1. Standard Form Optimization

Standard Form

A standard optimization problem is typically stated as follows:
\begin{aligned} p^* = \min_\limits{x} \quad & f_{0}(x) \\ \text{subject to:} \quad & f_{i}(x) \leq 0, \quad i=1,\ldots,m, \end{aligned}
where $p^{\ast }$ is the optimal value, $x\in {\mathbb{R}}^n$ is the decision variable, $f_0:{\mathbb{R}}^n\to \mathbb{R}$ is the objective function (or cost), and $f_i:{\mathbb{R}}^n\to \mathbb{R},i=1,\dots ,k$ are the $k$ optimization constraints.

Other than inequality constraints, an optimization problem may have equality constraints $h_i=0$ (equivalently represented via a pair of inequalities $h_i\ge 0$ and $h_i\le 0$), and set constraints $x\in \mathcal{X}$.

Furthermore, maximization problems can be readily recast as minimization problems, such that
$p^{\ast }=\underset{x}{\max }{f}_0(x)=-\underset{x}{\min }-f_0(x).$

"Subject To" Notation $\text{subject to:}$” is sometimes replaced with the shorthand “$\text{s.t.:}$,” or simply by colon notation “$\text{:}$”.

In the above, the term “

Properties

(Feasible Set) $\mathcal{X}=\{x\in {\mathbb{R}}^n:f_i(x)\le 0\forall i=1,\dots ,k\}$.

The feasible set is the set of points that satisfy the constraints of an optimization problem. If $\mathcal{X}=\varnothing$, the problem is said to be infeasible, such that $p^{\ast }=\{\begin{array}{ll}+\infty & \text{if }{p}^{\ast }=\underset{x}{\min }{f}_0(x),\\ -\infty & \text{if }{p}^{\ast }=\underset{x}{\max }{f}_0(x).\end{array}$

(Optimal Set) ${\mathcal{X}}_{\text{opt}}=\{x\in \mathcal{X}:f_0(x)=p^{\ast }\}=\arg \underset{x\in \mathcal{X}}{\min }{f}_0(x)$.

If $\mathcal{X}=\varnothing$, then, naturally ${\mathcal{X}}_{\text{opt}}=\varnothing$. However, the converse isn’t true. There may be feasible solutions to an optimization problem, but the optimal set is empty because:

1. The optimal solution is at a limit (i.e., $f_0$ diverges at $x^{\ast }$).
2. The constraints of the optimization problem are not satisfied.

(Sub-optimality) If $x\in \mathcal{X}$ and $p^{\ast }<f_0(x)\le p^{\ast }+ϵ$, then $x$ is $ϵ$-suboptimal.