Optimization Methods: Introduction and Basic Concepts 1

Module – 1 Lecture Notes – 3

Classification of Optimization Problems

Introduction

In the previous lecture we studied the basics of an optimization problem and its formulation
as a mathematical programming problem. In this lecture we look at the various criteria for
classification of optimization problems.

Optimization problems can be classified based on the type of constraints, nature of design
variables, physical structure of the problem, nature of the equations involved, deterministic
nature of the variables, permissible value of the design variables, separability of the functions
and number of objective functions. These classifications are briefly discussed below.

Classification based on existence of constraints.

Under this category optimizations problems can be classified into two groups as follows:

Constrained optimization problems: which are subject to one or more constraints.

Unconstrained optimization problems: in which no constraints exist.

Classification based on the nature of the design variables.

There are two broad categories in this classification.

(i) In the first category the objective is to find a set of design parameters that makes a
prescribed function of these parameters minimum or maximum subject to certain constraints.
For example to find the minimum weight design of a strip footing with two loads shown in
Fig 1 (a) subject to a limitation on the maximum settlement of the structure can be stated as
follows.

⎧b ⎫
Find X = ⎨ ⎬ which minimizes
⎩d ⎭

f(X) = h(b,d)

Subject to the constraints δ s ( X ) ≤ δ max ; b ≥ 0 ; d ≥ 0

where δ s is the settlement of the footing. Such problems are called parameter or static

optimization problems.

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 2

It may be noted that, for this particular example, the length of the footing (l), the loads P1 and
P2 and the distance between the loads are assumed to be constant and the required
optimization is achieved by varying b and d.

(ii) In the second category of problems, the objective is to find a set of design parameters,
which are all continuous functions of some other parameter that minimizes an objective
function subject to a set of constraints. If the cross sectional dimensions of the rectangular
footings are allowed to vary along its length as shown in Fig 1 (b), the optimization problem
can be stated as :

⎧ b(t ) ⎫
Find X(t) = ⎨ ⎬ which minimizes
⎩d (t )⎭

f(X) = g( b(t), d(t) )

Subject to the constraints

δ s ( X(t) ) ≤ δ max 0≤t≤l

b(t) ≥ 0 0≤t≤l

d(t) ≥ 0 0≤t≤l

The length of the footing (l) the loads P1 and P2 , the distance between the loads are assumed
to be constant and the required optimization is achieved by varying b and d along the length l.

Here the design variables are functions of the length parameter t. this type of problem, where
each design variable is a function of one or more parameters, is known as trajectory or
dynamic optimization problem.

P1 P1 t
P2 P2

b b(t)

d d(t)

l l

(a) (b)

Fig 1

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 3

Classification based on the physical structure of the problem

Based on the physical structure, optimization problems are classified as optimal control and
non-optimal control problems.

(i) Optimal control problems

An optimal control (OC) problem is a mathematical programming problem involving a

number of stages, where each stage evolves from the preceding stage in a prescribed manner.
It is defined by two types of variables: the control or design and state variables. The control
variables define the system and controls how one stage evolves into the next. The state
variables describe the behavior or status of the system at any stage. The problem is to find a
set of control variables such that the total objective function (also known as the performance
index, PI) over all stages is minimized, subject to a set of constraints on the control and state
variables. An OC problem can be stated as follows:
l
Find X which minimizes f(X) = ∑ f (x , y )
i =1
i i i

Subject to the constraints

qi ( xi , y i ) + y i = y i +1 i = 1, 2, …., l

g j (x j ) ≤ 0 , j = 1, 2, …., l

hk ( y k ) ≤ 0 , k = 1, 2, …., l

Where xi is the ith control variable, yi is the ith state variable, and fi is the contribution of the
ith stage to the total objective function. gj, hk, and qi are the functions of xj, yj ; xk, yk and xi and
yi, respectively, and l is the total number of states. The control and state variables xi and yi
can be vectors in some cases.

(ii) Problems which are not optimal control problems are called non-optimal control
problems.

Classification based on the nature of the equations involved

Based on the nature of equations for the objective function and the constraints, optimization
problems can be classified as linear, nonlinear, geometric and quadratic programming
problems. The classification is very useful from a computational point of view since many

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 4

predefined special methods are available for effective solution of a particular type of
problem.

(i) Linear programming problem

If the objective function and all the constraints are ‘linear’ functions of the design variables,
the optimization problem is called a linear programming problem (LPP). A linear
programming problem is often stated in the standard form :

⎧ x1 ⎫
⎪x ⎪
⎪⎪ 2 ⎪⎪
Find X = ⎨ . ⎬
⎪.⎪
⎪ ⎪
⎪⎩ x n ⎪⎭

n
Which maximizes f(X) = ∑c x
i =1
i i

Subject to the constraints

n

∑a
i =1
ij xi = b j , j = 1, 2, . . . , m

xi ≥ 0 , j = 1, 2, . . . , m

where ci, aij, and bj are constants.

(ii) Nonlinear programming problem

If any of the functions among the objectives and constraint functions is nonlinear, the
problem is called a nonlinear programming (NLP) problem. This is the most general form of
a programming problem and all other problems can be considered as special cases of the NLP
problem.

(iii) Geometric programming problem

A geometric programming (GMP) problem is one in which the objective function and
constraints are expressed as polynomials in X. A function h(X) is called a polynomial (with
m terms) if h can be expressed as

h( X ) = c1 x1a11 x 2a21 L x nan1 + c 2 x1a12 x 2a22 L x nan 2 + L + c m x1a1m x 2a2 m L x nanm

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 5

where cj ( j = 1, L , m ) and aij ( i = 1, L , n and j = 1, L , m ) are constants with c j ≥ 0 and

xi ≥ 0 .

Thus GMP problems can be posed as follows:

Find X which minimizes

N0
⎛ n aij ⎞
f(X) = ∑
j =1
c j⎜
⎜ C xi
⎝ i =1
⎟⎟,
⎠
cj > 0, xi > 0

subject to
Nk
⎛ n
⎞
∑ a ⎜⎜⎝ C x ⎟⎟ > 0,
qijk
gk(X) = jk i ajk > 0, xi > 0, k = 1,2,…..,m
j =1 i =1 ⎠

where N0 and Nk denote the number of terms in the objective function and in the kth constraint
function, respectively.

(iv) Quadratic programming problem

A quadratic programming problem is the best behaved nonlinear programming problem with
a quadratic objective function and linear constraints and is concave (for maximization
problems). It can be solved by suitably modifying the linear programming techniques. It is
usually formulated as follows:

n n n
F(X) = c + ∑ qi xi + ∑∑ Qij xi x j
i =1 i =1 j =1

Subject to
n

∑a
i =1
ij xi = b j , j = 1,2,….,m

xi ≥ 0 , i = 1,2,….,n

where c, qi, Qij, aij, and bj are constants.

Classification based on the permissible values of the decision variables

Under this classification, objective functions can be classified as integer and real-valued
programming problems.

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 6

(i) Integer programming problem

If some or all of the design variables of an optimization problem are restricted to take only
integer (or discrete) values, the problem is called an integer programming problem. For
example, the optimization is to find number of articles needed for an operation with least
effort. Thus, minimization of the effort required for the operation being the objective, the
decision variables, i.e. the number of articles used can take only integer values. Other
restrictions on minimum and maximum number of usable resources may be imposed.

(ii) Real-valued programming problem

A real-valued problem is that in which it is sought to minimize or maximize a real function

by systematically choosing the values of real variables from within an allowed set. When the
allowed set contains only real values, it is called a real-valued programming problem.

Classification based on deterministic nature of the variables

Under this classification, optimization problems can be classified as deterministic or

stochastic programming problems.

(i) Deterministic programming problem

In a deterministic system, for a same input, the system will produce the same output always.
In this type of problems all the design variables are deterministic.

(ii) Stochastic programming problem

In this type of an optimization problem, some or all the design variables are expressed
probabilistically (non-deterministic or stochastic). For example estimates of life span of
structures which have probabilistic inputs of the concrete strength and load capacity is a
stochastic programming problem as one can only estimate stochastically the life span of the
structure.

Classification based on separability of the functions

Based on this classification, optimization problems can be classified as separable and non-
separable programming problems based on the separability of the objective and constraint
functions.

(i) Separable programming problems

D Nagesh Kumar, IISc, Bangalore M1L3

Optimization Methods: Introduction and Basic Concepts 7

In this type of a problem the objective function and the constraints are separable. A function
is said to be separable if it can be expressed as the sum of n single-variable functions,
f1 ( xi ), f 2 (x 2 ),... f n ( x n ) , i.e.

n
f ( X ) = ∑ f i ( xi )
i =1

and separable programming problem can be expressed in standard form as :

n
Find X which minimizes f ( X ) = ∑ f i ( xi )
i =1

subject to
n
g j ( X ) = ∑ g ij ( xi ) ≤ b j , j = 1,2,. . . , m
i =1

where bj is a constant.

Classification based on the number of objective functions

Under this classification, objective functions can be classified as single-objective and multi-
objective programming problems.

(i) Single-objective programming problem in which there is only a single objective function.

(ii) Multi-objective programming problem

A multiobjective programming problem can be stated as follows:

Find X which minimizes f1 ( X ), f 2 ( X ),... f k ( X )

Subject to

gj(X) ≤ 0 , j = 1, 2, . . . , m

where f1, f2, . . . fk denote the objective functions to be minimized simultaneously.

For example in some design problems one might have to minimize the cost and weight of the
structural member for economy and, at the same time, maximize the load carrying capacity
under the given constraints.

D Nagesh Kumar, IISc, Bangalore M1L3