A HYBRID NEURAL MODEL FOR THE OPTIMIZATION OF FED-BATCH FERMENTATIONS

A HYBRID NEURAL MODEL FOR THE OPTIMIZATION OF FED-BATCH FERMENTATIONS^*

A.C. COSTA ¹, A.S.W. HENRIQUES ¹, T.L.M. ALVES ¹, R. MACIEL FILHO ² and E.L. LIMA ¹

PEQ/COPPE/UFRJ - Cx. Postal 68502, CEP 21945-970, Rio de Janeiro, RJ, Brazil.¹

DPQ/FEQ/UNICAMP - Cx. Postal 6066, CEP 13081-970, Campinas, SP, Brazil. ²

e-mail: [e-mail]

(Received: July 10, 1998; Accepted: March 2, 1999)

Abstract - In this work a hybrid neural modelling methodology, which combines mass balance equations with functional link networks (FLNs), used to represent kinetic rates, is developed for bioprocesses. The simple structure of the FLNs allows the easy and rapid estimation of network weights and, consequently, the use of the hybrid model in an adaptive form. As the proposed model is able to adjust to kinetic and environmental changes, it is suitable for use in the development of optimization strategies for fed-batch bioreactors. The proposed methodology is used to model the processes for penicillin and ethanol production, and the development of an adaptive optimal control scheme is discussed using ethanol fermentation as an example.

Keywords: fed-batch fermentation, optimal control, hybrid neural modelling.

INTRODUCTION

Many industrially important fermentation processes are carried out in fed-batch mode, in which a feed stream containing substrate and/or nutrients is fed into the fermentor during the course of batch fermentation. In the optimization of such processes, the objective, expressed in the form of a performance index, is to maximize the product of interest at the end of the batch cycle. This is an optimal control problem, and there are various methods proposed for its solution (Modak and Lim, 1987; Palanki et al., 1993; Costa, 1996). The optimal solution consists in determination of the temporal profiles of control variables which maximize the performance index.

One of the most severe problems in the control and optimization of biotechnological processes is the construction of reliable system models. This difficulty is associated with the complex nature of microbial metabolism and the highly non-linear nature of its kinetics. Besides, fed-batch processes are transient and the process variables undergo significant changes with time. Hence, a model adopted for optimization and control must be able to describe the behavior of the system over a wide range of operational conditions.

Because of the problems discussed above, in most cases, detailed models based on fundamental principles and detailed kinetic studies are not readily available, due to economic and time constraints. Thus, it would be of great advantage to find some simple and rapid way of describing fermentation processes accurately enough for optimization and control.

In recent years many methods have been proposed to achieve this goal. One of them is the use of neural networks (Di Massimo et al., 1992; Simutis et al., 1993; Glassey et al., 1994), which offers a tool for direct use of process data to generate input-output relationships. Another approach has been the use of tendency models (Marchal-Brassely et al., 1992; Fotopoulos et al., 1994).

In this work a hybrid neural modelling procedure will be developed. It combines a priori knowledge of the process, through mass balance equations, with neural networks, which describe the unknown kinetics. Other authors have applied this procedure to biochemical processes (Psichogios and Ungar, 1992; Schubert et al., 1994; Fu and Barford, 1994). The great advantage of the hybrid model proposed here is the use of functional link networks (FLNs) (Chen and Billings, 1992), in which network weights appear linearly. This makes on-line reestimation of weights very easy, which enables on-line calculation of kinetic rates. As these rates are calculated based on real data, the proposed model is able to adjust to possible changes in the process and account for uncertainties.

A modification is applied to the FLNs to increase their non-linear approximation ability, and an orthogonal least-squares estimator is used to calculate network weights and to eliminate non-significant nodes during the training of the network (Henrique and Lima, 1996). This significantly reduces the size and complexity of the network and avoids overfitting the data.

The proposed modelling procedure is applied to the penicillin fermentation (Bajpai and Reuss, 1981) and alcohol production (Aiba et al., 1968) processes, and the development of an adaptive optimal control scheme in which the optimal trajectory of the control variable is corrected at each sampling time is discussed.

HYBRID NEURAL MODELLING

As noted by Psichogios and Ungar (1992), it is straightforward to derive an approximate model of a bioreactor from simple first principles considerations such as mass balance of the process variables. The critical factor in determining the dynamic behavior of the process is the unknown kinetics. Thus, in the hybrid neural model, the aspects of the problem whose quantitative behavior is well understood are described by deterministic mathematical equations, while neural networks describe the kinetics.

Mass Balance Equations

A fed-batch stirred bioreactor can be described by the following mass balance equations:

(1)

(2)

(3)

(4)

where X, S and P are the biomass, substrate and ethanol concentrations; S_F is the feed substrate concentration, D is the dilution rate, defined as D=F/V; F is the volumetric substrate feed rate; V is the volume of the reactor and m , s and p are the specific rates of growth, substrate consumption and ethanol formation, respectively, which are described by functional link networks.

Functional Link Networks

A neural network typically consists of many simple computational elements or nodes arranged in layers and operating in parallel. Weights, which define the strength of connections between the nodes, are estimated to yield a good performance. Usually, in the training of neural networks, the inputs to a node are linearly weighted before the sum is passed through some non-linear activation function that ultimately gives the network its non-linear approximation ability. The same non-linearity, however, creates problems in learning the network weights, as non-linear learning rules must be used, the learning rate is often unacceptably slow and local minima may cause problems (Chen and Billings, 1992).

One way of avoiding non-linear learning is to use functional link networks. In these networks, a non-linear functional transform or expansion of the network inputs is initially performed and the resulting terms are combined linearly. The structure obtained has a good non-linear approximation capability, and the estimation of network weights is linear.

The general structure of an FLN is shown in Fig. 1, where x_e is the input vector and y_i(x_e) is an output. The hidden layer performs a functional expansion on the input data which maps the input space of dimension n₁ onto a new space of increased dimension, M (M > n₁). The output layer consists of m nodes, each one, in fact, a linear combiner. The input-output relationship of the FLN is

, 1 £ i £ m (5)

Figure 1:
General structure of an FLN.

Henrique and Lima (1996) proposed a modification of the structure of the FLNs, where the output given by eq. (5) is transformed by an invertible non-linear activation function. The new output is

, 1 £ i £ m (6)

where f_i is an invertible non-linear function such as, for example

(7)

The proposed modification increases the non-linear approximation ability of the network, while estimation of the parameters remains a linear problem.

Training of the Neural Network

The network inputs (x_e) are the process state variables: biomass, substrate and product concentrations. The output is the kinetic rate to be calculated. In this work, the state variables are considered accessible either by direct measurement or by the use of estimators such as the extended Kalman filter. The outputs are not measurable but can be estimated from measured experimental data by the discretization of the mass balance equations (eqs. 1, 2 and 3). As this is a simulation, the inputs to the network are not noisy. As noted by Simutis et al. (1993), however, in real situations this consideration may not be true and some kind of smoothing algorithm has to be applied to produce reliable rate values from noisy concentration data. Another approach is to use an integrator to calculate the concentrations of biomass, substrate and product from the rate values produced by the network. Thus, during the training process, the error to be minimized is the difference between the calculated concentrations and their real measurements. As the integration is numerically more stable than the differentiation of noisy data sets, greater stability of the training process can be achieved.

In this work, the real inputs, x_e, were first transformed into a greater number (n_z) of auxiliary inputs, z, with expressions that are known to be common in bioprocesses kinetics

(xe1, xe2,...,,...).

Then a polynomial expansion of degree six was performed on these new inputs. The generated monomials (h_j(z)) are given in Fig. 2.

Figure 2: Polynomial expansion of degree six.

Once the monomials are determined, the orthogonal least-squares estimator proposed by Billings et al. (1989) is used to calculate the network weights (w_ij) and to eliminate the monomials which are not significant in explaining the output variance.

APPLICATION OF HYBRID NEURAL MODELLING TO BIOPROCESSES

The application of the proposed hybrid neural modelling procedure to the penicillin and alcohol fermentation processes is shown in the following examples. In these examples, only the specific growth rate is considered unknown and described by an FLN, but the same training methodology can be used to obtain the other kinetic rates. The hybrid model would then be formed by the mass balance equations and three FLNs, one for each kinetic rate.

It is assumed that the "true" but unknown specific growth rate is described by the models given by Bajpai and Reuss (1981) for penicillin fermentation and by Aiba et al. (1968) for alcohol production. These models are only used to simulate the "true" process data.

Penicillin Fermentation

The "true" specific growth rate is given by

(8)

As it is known beforehand that the specific growth rate is only a function of the biomass and substrate concentrations, the network input is

(9)

The network output is

(10)

In this case, vector z was chosen as

(11)

The activation function used was

(12)

The activation function and vector z were chosen after tests with different options.

The number of monomials generated is a function of the size of vector z and the degree of polynomial expansion, in this case six. The number of monomials generated was 210. The orthogonal least-squares estimator was used to eliminate the non-significant monomials and to estimate the weights. The resulting expression is

(13)

It can be noticed that this expression corresponds exactly to the model equation (eq. 8), which shows that the orthogonal estimator makes a good choice of significant terms. This happened because, in this example, the specific growth rate is a function of only the auxiliary inputs. It would be interesting to determine if the use of this kind of inputs to describe a more complex function, such as the specific growth rate in the case of ethanol fermentation, leads to good results.

Ethanol Fermentation

The "true" growth rate is given by

(14)

The network input is

(15)

The output is given by eq. 10.

Vector z is defined as

(16)

and the activation function used is described by eq. 12.

The resulting expression is

(17)

Performance is measured by (Milton and Arnold, 1990)

(18)

In the equation above, , , y_e(k) is the k experimental output, y(k) is the corresponding network output, is the mean value of the experimental outputs and N is the number of experimental data.

The FLN describes the data generated by the model with a correlation of 99.97%.

Fig. 3 shows the results obtained when the FLN is implemented in a batch ethanol fermentation. The model equation (eq. 14) was used to simulate the process data.

Figure 3: Specific growth rate versus time for batch ethanol fermentation.

Adaptive Scheme

After the neural network part of the hybrid model is trained, based on already available data, its weights can continue being updated during the real-time application. In this way, the model is able to adapt itself to deviations from the information contained in the training data set. This adaptive characteristic is of extreme importance in biochemical processes as it takes into account kinetic changes that may occur during the course of fermentation.

In the use of the hybrid model in an adaptive scheme, the neural network structure is considered defined (during the training process) and the network weights are reestimated in real time based on the on-line measured values, which are added to a part of the original training set. The size of the original data set used at this stage defines the importance attributed to the on-line measured values, that is, the smaller the training data set, the greater the importance attributed to the measured data.

At each sampling time, the following algorithm is used:

(a) The values of the state variables are measured or estimated.

(b) These values are used to calculate an approximate value for the kinetic rate from the discretization of Eqs. 1, 2 or 3 (depending on which kinetic rate is represented by the neural network).

(c) The values of the state variables and of the approximate kinetic rate are added to a part of the original training set, from which the values of the state variables and the kinetic rate corresponding to the oldest time are eliminated.

(d) The new training set is used to reestimate the network weights using the algorithm proposed by Billings et al. (1989).

When the hybrid model is used in an adaptive scheme, the importance of choosing the functional link structure for the neural networks is shown. As, for these networks, estimation of the weights is linear, the on-line calculation of the new weights is rapid and does not require much computational effort.

USE OF THE HYBRID NEURAL MODEL FOR THE OPTIMAL CONTROL OF FED-BATCH FERMENTATIONS

The objective of the optimization problem for fed-batch fermentations is to determine the substrate feed policy which maximizes the product of interest at the end of the batch cycle. Open-loop implementation of the precalculated solution, however, is only appropriate in situations where the process model is accurately known and there are no external disturbances to the process.

In most cases, models used to determine the optimal feed policy for fed-batch biochemical processes are developed based on steady state data and give only approximate results under transient conditions. Besides, strain modification due to microbial adaptation or a change in the quality of nutrient medium can cause large variations in the values of parameters used to model a system (Agrawal et al., 1989). Thus, it is desirable that the precalculated solution be corrected on-line. However, in most real-time applications this is infeasible due to prohibitive computing-power requirements. The great difficulty is usually the on-line estimation of kinetic parameters, as they lead to the non-linearity of the model equations.

The hybrid neural model developed in this work deals easily with the on-line estimation of kinetic parameters and is appropriate for use in an adaptive optimal control scheme. The development of such a scheme will be discussed in this work using the ethanol fermentation process as an example.

Implementation of an adaptive optimal control scheme requires that the optimal profile of the control variable be known as a function of only the process state variables. The methodology used in this work to calculate the optimal profile was proposed by Costa (1996) and is valid for processes described by four or less mass balance equations. The objective is to maximize product concentration at the end of the batch cycle. The control variable is the dilution rate.

For an ethanol fermentation with initial conditions of X(0)=0.2 g/l, S(0)=100 g/l P(0)=0 g/l, V(0)=5 l, S_F=100 g/l and V_max=20 l, the solution of the optimization problem obtained using the methodology proposed by Costa (1996) is shown in Fig. 4. The bioreactor is operated in batch mode until the singular arc expression is satisfied. At this point, it is fed at the singular dilution rate until the volume reaches its maximum. Another batch period follows until the stop condition is attained, in this case, until the substrate concentration is equal to zero. Both the singular dilution rate and the singular arc expressions are obtained as functions of the state variables, the kinetic rates and their derivatives. The difficulty in on-line implementation is the calculation of the singular arc and the singular dilution rate expressions. The adaptive scheme based on the hybrid neural model calculates these expressions at each sampling time (15 minutes in this case) using the measured values of the state variables and the values of the kinetic rates calculated by the FLNs with updated weights. The derivatives of the kinetic rates are approximated by the derivatives of the corresponding FLNs.

Figure 4: Optimal operation of a fed-batch bioreactor for ethanol fermentation.

The ethanol fermentation example is used to evaluate the performance of the adaptive optimal scheme. Some modelling errors are simulated: the real specific growth rate is considered to be 10% higher or lower than the model predicts. The results obtained are compared to the open-loop implementation of the profile, calculated based on the imperfect model, and to the results that would be obtained if the perfect model were used to calculate the optimal profile.

Figures 5 and 6 show the temporal profiles of the volumetric substrate feed rate, which is easily obtained from the dilution rate (F=D.V), and of product concentration, when the real specific growth rate is 10% higher than the model predicts. In these figures it can be seen that the adaptive scheme leads to results much closer to the optimal ones (obtained based on the perfect model). As the real specific growth rate is higher than the model predicts, the substrate consumption calculated by the imperfect model is lower than the real one. Thus, the feed rate profile, calculated based on the imperfect model, is lower than the optimal one and its open-loop implementation leads to a low final product concentration. In the adaptive scheme, the reestimation of network weights leads to better results, which are closer to the optimum.

Figure 5: Substrate feed rate versus time. Real specific growth rate 10% higher than the model predicts.

Figure 6: Product versus time. Real specific growth rate 10% higher than the model predicts.

Figures 7 and 8 show the volumetric substrate feed rate and the product concentration versus time when the real specific growth rate is 10% lower than the model predicts. In this case the feed rate profile, calculated based on the imperfect model, is higher than the optimum, as the model predicts a substrate consumption higher than the real one. The adaptive scheme leads to results near the optimum.

Figure 7: Substrate feed rate versus time. Real specific growth rate 10% lower than the model predicts.

Figure 8: Product versus time. Real specific growth rate 10% lower than the model predicts.

CONCLUSIONS

Bioreactors operating in fed-batch mode are quite difficult to model, since their operation involves microbial growth under constantly changing conditions. The development of models from a basic understanding of fermentation is often extremely time consuming for such processes. Besides, the development of optimization strategies is heavily dependent upon the reliability of the process model, and uncertainties in the kinetic parameters may lead to deterioration in the optimization scheme.

The problems discussed above point to a need for the development of simple though realistic mathematical descriptions of fed-batch processes which can be used to develop optimal operating strategies. Hybrid neural models are suitable for this use, since, as pointed out by Psichogios and Ungar (1992), they are expected to perform better than "black-box" neural network models. This is due to the fact that generalization and extrapolation are confined only to the uncertain part of the process, while the basic model is always consistent with first principles. Besides, significantly less data are required for training hybrid neural models.

In this work a hybrid neural modelling methodology, which combines the mass balance equations with functional link networks, was developed for bioprocesses. The hybrid model was shown to describe the dynamics of bioprocesses accurately.

An efficient optimization strategy is of considerable importance nowadays. The development time of the optimization scheme must be significantly shorter than the development time of a new strain. Thus, estimation and adaptive tools are required. In this work it was shown that the choice of a simple structure for the neural network part of the hybrid model enables its use for the development of adaptive optimal control strategies, as the reestimation of network parameters is easy and rapid.

NOTATION

D Dilution rate (h^-1) D_sing Singular dilution rate F Volumetric feed rate (L/h) f Activation function of the FLNs h Monomials generated by the functional expansion M Number of monomials generated by the functional expansion m Number of outputs of the FLNs n₁ Size of the input vector P Product concentration (g/L) S Substrate concentration (g/L) S_F Feed substrate concentration (g/L) t Time (h) V Volume of the reactor (L) V_max Maximum volume of the reactor w FLN weights X Biomass concentration (g/L) x_e Input of the FLN y Output of the FLN z Auxiliary inputs vector

Greek Letters

m Specific growth rate (h^-1) p Specific product formation rate (h^-1) s Specific substrate consumption rate (h^-1)

REFERENCES

Agrawal, P., Koshy, G. and Ramseier, M., An Algorithm for Operating a Fed-Batch Fermentor at Optimum Specific-Growth Rate. Biotechnol. Bioeng. 33, 115 (1989).

Aiba, S., Shoda, N. and Nagatani, N., Kinetics of Product Inhibition in Alcohol Fermentation. Biotechnol. Bioeng. 10, 845 (1968).

Bajpai, R.K. and Reuss, M., Evaluation of Feeding Strategies in Carbon-Regulated Secondary Metabolite Product through Mathematical Modelling, Biotechnol. Bioeng. 23, 714 (1981).

Billings, S.A., Chen, S. and Korenberg, M.J., Identification of MIMO Non-linear Systems Using a Forward-Regression Orthogonal Estimator, Int. J. Control. 49, 2157 (1989).

Chen, S. and Billings, S.A., Neural Networks for Nonlinear System Modelling and Identification, Int. J. Control. 56, 319 (1992).

Costa, A.C., Controle Singular em Bioreatores. M.Sc. Thesis, PEQ/COPPE/UFRJ, Rio de Janeiro, Brazil (1996).

Di Massimo, C., Montague, G.A., Willis, M.J., Tham, M.T. and Morris, A.J., Towards Improved Penicillin Fermentation via Artificial Neural Networks, Comp. Chem. Engng. 16, 283 (1992).

Fotopoulos, J., Georgakis, C. and Stenger Jr, H.G., Uncertainty Issues in the Modelling and Optimization of Batch Reactors with Tendency Models, Chem. Engng. Sci. 49, 5533 (1994).

Fu, P. and Barford, J.P., Hybrid Modelling Combining a Detailed Metabolic Simulation and Neural Network Approaches to Complex Biochemical Processes, Proc. of PSE’94, 571 (1994).

Glassey, J., Montague, G.A., Ward, A.C. and Kara, B.V., Artificial Neural Network Based Experimental Design Procedures for Enhancing Fermentation Development, Biotechnol. Bioeng. 44, 397 (1994).

Henrique, H.M. and Lima, E.L., Model Structure Determination in Neural Network Models, submitted, 1996.

Marchal-Brassely, S., Villermaux, J., Houzelot, J.L. and Barnay, J.L., Optimal Operation of a Semi-Batch Reactor by Self-adaptive Models for Temperature and Feed-Rate Profiles, Chem. Engng. Sci. 47, 2445 (1992).

Milton, J. S. and Arnold, J. C., Introduction to Probability and Statistics, McGraw Hill, New York (1990).

Modak, J.M. and Lim, H.C., Feedback Optimization of Fed-Batch Fermentation, Biotechnol. Bioeng. 30, 528 (1987).

Palanki, S., Kravaris, C. and Wang, H.Y., Synthesis of State Feedback Laws for End-point Optimization in Batch Processes, Chem. Engng. Sci. 48, 135 (1993).

Psichogios, D.C. and Ungar, L.H., A Hybrid Neural Network - First Principles Approach to Process Modelling, AIChE J. 38, 1499 (1992).

Schubert, J., Simutis, R., Dors, M., Havlik, I. and Lübbert, A., Bioprocess Optimization and Control: Application of Hybrid Modelling, J. Biotechnol. 35, 51 (1994).

Simutis, R., Havlik, I. and Lübbert, A., Fuzzy-aided Neural Network for Real-Time State Estimation and Process Prediction in the Alcohol Formation Step of Production-scale Beer Brewing, J. Biotechnol. 27, 203 (1993).

Publication Dates

• Publication in this collection
  23 Apr 1999
• Date of issue
  Mar 1999

History

• Accepted
  02 Mar 1999
• Received
  10 July 1998