Relative Entropy (Kullback–Leibler divergence). Relative entropy where probabilities are variables

 

Syntax

 
Parameters

matrix        is a PSG matrix:

 

where the header row contains names of variables. The second row contains numerical data, .

 

Mathematical Definition

Relative Entropy is calculated as follows:

,

where

is an argument of Relative Entropy function.

This function is usually used with the additional constraint:

.

 

Remarks

The Relative Entropy function can be used in linear combination with any other function that do not belong to the probability group. However, if you want to accelerate optimization process with Relative Entropy function in objective, this function should be a stand alone function and it should be linearized (see Objective in General (Text) Format). In this case, the number of decision variables may go up to 1,000,000, and the BULDOZER solver is recommended (see Solver).

 

Example

 

Case Studies with Relative Entropy

 

See also

Polynomial Absolute, CVaR Component Positive, CVaR Component Negative, CVaR Component Absolute, VaR Component Positive, VaR Component Negative, Maximum Component Positive, Maximum Component Negative, Maximum Component Absolute, Quadratic Function, Squareroot Quadratic, Logarithms Sum