The set of all decisions is called the operating policy or, more simply, the policy.
An optimal policy is one which in some sense gets the best out of the process as a whole by maximizing the value of the product.
There are thus three components to an optimal design problem: (1)

The specification of the state of the process stream;
(2)

The specification of the operating variables and the transformation they effect;
(3)

The specification of the objective function of which the optimization is desired.
For a chemical process the first of these might involve the concentrations of the different chemical species, and the temperature or pressure of the stream.
For the second we might have to choose the volume of reactor or amount of cooling to be supplied;
the way in which the transformation of state depends on the operating variables for the main types of reactors is discussed in the next chapter.
The objective function is some measure of the increase in value of the stream by processing;
it is the subject of Chapter 4.

The essential characteristic of an optimal policy when the state of the stream is transformed in a sequence of stages with no feedback was first isolated by Bellman.
He recognized that whatever transformation may be effected in the first stage of an R-stage process, the remaining stages must use an optimal Af-stage policy with respect to the state resulting from the first stage, if there is to be any chance of optimizing the complete process.
Moreover, by systematically varying the operating conditions in the first stage and always using the optimal Af-stage policy for the remaining stages, we shall eventually find the optimal policy for all R stages.
Proceeding in this way, from one to two and from two to three stages, we may gradually build up the policy for any number.
At each step of the calculation the operating variables of only one stage need be varied.

To see how important this economy is, let us suppose that there are M operating variables at each stage and that the state is specified by N variables;
then the search for the maximum at any one stage will require a number of operations of order Af (where a is some number not unreasonably large).
To proceed from one stage to the next a sufficient number of feed states must be investigated to allow for interpolation;
this number will be of the order of Af.
If, however, we are seeking the optimal R-stage policy for a given feed state, only one search for a maximum is required at the final step.
Thus a number of operations of the order of Af are required.
If all the operating variables were varied simultaneously, Af operations would be required to do the same job, and as R increases this increases very much more rapidly than the number of operations required by the dynamic program.
But even more important than this is the fact that the direct search by simultaneously varying all operating conditions has produced only one optimal policy, namely, that for the given feed state and R stages.
In contrast, the dynamic program produces this policy and a whole family of policies for any smaller number of stages.
If the problem is enlarged to require a complete coverage of feed states, Af operations are needed by the dynamic program and Af by the direct search.
But Af is vastly larger than R.
No optimism is more baseless than that which believes that the high speed of modern digital computers allows for use of the crudest of methods in searching out a result.
Suppose that Af, and that the average operation requires only Af sec.
Then the dynamic program would require about a minute whereas the direct search would take more than three millennia!

The principle of optimality thus brings a vital organization into the search for the optimal policy of a multistage decision process.
Bellman (1957) has annunciated in the following terms:

"An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with respect to the state resulting from the first decision".

This is the principle which we will invoke in every case to set up a functional equation.
It appears in a form that is admirably suited to the powers of the digital computer.
At the same time, every device that can be employed to reduce the number of variables is of the greatest value, and it is one of the attractive features of dynamic programming that room is left for ingenuity in using the special features of the problem to this end.

2.2 the discrete deterministic process
Consider the process illustrated in Fig. 2.1, consisting of R distinct stages.
These will be numbered in the direction opposite to the flow of the process stream, so that stage R is the T stage from the end.
Let the state of the stream leaving stage R be denoted by a vector Af and the operating variables of stage R by Af.
Thus Af denotes the state of the feed to the R-stage process, and Af the state of the product from the last stage.
Each stage transforms the state Af of its feed to the state Af in a way that depends on the operating variables Af.
We write this Af.
This transformation is uniquely determined by Af and we therefore speak of the process as deterministic.
In practical situations there will be restrictions on the admissible operating conditions, and we regard the vectors as belonging to a fixed and bounded set S.
The set of vectors Af constitutes the operating policy or, more briefly, the policy, and a policy is admissible if all the Af belong to S.
When the policy has been chosen, the state of the product can be obtained from the state of the feed by repeated application of the transformation (1);
thus Af.
The objective function, which is to be maximized, is some function, usually piecewise continuous, of the product state.
Let this be denoted by Af.

An optimal policy is an admissible policy Af which maximizes the objective function P.
The policy may not be unique but the maximum value of P certainly is, and once the policy is specified this maximum can be calculated by (2) and (3) as a function of the feed state Af.
Let Af where the maximization is over all admissible policies Af.
When it is necessary to be specific we say that the optimal policy is an optimal R-stage policy with respect to the feed state Af.

For any choice of admissible policy Af in the first stage, the state of the stream leaving this stage is given by Af.
This is the feed state of the subsequent Af stages which, according to the principle of optimality, must use an optimal Af-stage policy with respect to this state.
This will result in a value Af of the objective function, and when Af is chosen correctly this will give Af, the maximum of the objective function.
Thus Af where the maximization is over all admissible policies Af, and Af is related to Af by (5).
The sequence of equations (6) can be solved for Af when Af is known, and clearly Af, the maximization being over all admissible Af.

The set of equations (5), (6), and the starting equation (7) is of a recursive type well suited to programming on the digital computer.
In finding the optimal R-stage policy from that of Af stages, only the function Af is needed.
When Af has been found it may be transferred into the storage location of Af and the whole calculation repeated.
We also see how the results may be presented, although if n, the number of state variables, is large any tabulation will become cumbersome.
A table or set of tables may be set out as in Table 2.1.

To extract the optimal R-stage policy with respect to the feed state Af, we enter section R of this table at the state Af and find immediately from the last column the maximum value of the objective function.
In the third column is given the optimal policy for stage R, and in the fourth, the resulting state of the stream when this policy is used.
Since by the principle of optimality the remaining stages use an optimal Af-stage policy with respect to Af, we may enter section Af of the table at this state Af and read off the optimal policy for stage Af and the resulting state Af.
Proceeding in this way up the table we extract the complete optimal policy and, if it is desired, we can check on Af by evaluating Af at the last stage.

It may be that the objective function depends not only on Af but also on Af, as when the cost of the operating policy is considered.
A moment's reflection shows that the above algorithm and presentation work equally well in this case.
A form of objective function that we shall often have occasion to consider is Af.
Here P represents the value of the stream in state P and Q the cost of operating the stage with conditions Q.
Hence P is the increase in value of the stream minus the cost of operation, that is, the net profit.
If Af denotes the net profit from stage R and Af, then the principle of optimality gives Af.
This sequence of equations may be started with the remark that with no process Af there is no profit, i.e., Af.

2.3 the discrete stochastic process
The process in which the outcome of any one stage is known only statistically is also of interest, although for chemical reactor design it is not as important as the deterministic process.
In this case the stage R operating with conditions Af transforms the state of the stream from Af to Af, but only the probability distribution of Af is known.
This is specified by a distribution function Af such that the probability that Af lies in some region D of the stage space is Af.

We cannot now speak of maximizing the value of the objective function, since this function is now known only in a probabilistic sense.
We can, however, maximize its expected value.
For a single stage we may define Af where the maximization is by choice of Af.
We thus have an optimal policy which maximizes the expected value of the objective function for a given Af.
If we consider a process in which the outcome of one stage is known before passage to the next, then the principle of optimality shows that the policy in subsequent stages should be optimal with respect to the outcome of the first.
Then Af, the maximization being over all admissible Af and the integration over the whole of stage space.

The type of presentation of results used in the deterministic process may be used here, except that now the fourth column is redundant.
The third column gives the optimal policy, but we must wait to see the outcome of stage R and enter the preceding section of the table at this state.
The discussion of the optimal policy when the outcome of one stage is not known before passing to the next is a very much more difficult matter.

2.4 the continuous deterministic process
In many cases it is not possible to divide the process into a finite number of discrete stages, since the state of the stream is transformed in a continuous manner through the process.
We replace r, the number of the stage from the end of the process, by t, a continuous variable which measures the "distance" of the point considered from the end of the process.
The word distance is used here in a rather general sense;
it may in fact be the time that will elapse before the end of the process.
If T is the total "length" of the process, its feed state may be denoted by a vector p(T) and the product state by p(Q).
P denotes the state at any point T and Q the vector of operating variables there.