4.2 Model order reduction via proper orthogonal decomposition
4.2.5 Robustness of POD model order reduction
In the next test, we study the behavior of reduced-order modeling in case of systems which are badly scaled. For this purpose, we select a cost parameter σu which is close to zero, a discontinuous desired state, and generous control bounds which allow strong oscillations of the resulting bang-bang type control.
Run 4. LetyQ∈L2(Θ×Ω)be defined asyQ(t, x) =−2x−2t+ 2tx+ 2x2 for t < xand yQ(t, x) = 0otherwise fort, x∈[0,1]and lety◦(x) =−2x(1−x) =yQ(0, x). We choose the control boundsua=−100,ub = 100and the cost parameterσu=5.0e-10. Fig. 4.24 illustrates the desired state and the optimal control-state pair of the full-order model.
The problem setting is challenging not only because of the jump inyQ, but also because of the information transport: The shape of the initial value both has to be shifted to the left boundary of Ω though the PDE includes no transport term and the diffusion effect has to be eliminated by the control. In the control term, we recognize not only the regularizedδ-distribution part required to enforce the jump in the PDE solution, but also strong oscillations in the neighborhood of the line t=x. Carried over to the state solution, we see that these oscillations cause a fold by multiple changes of the first-order derivative in the direction(τ, ξ) = (1,1)along the critical line.
0 optimal state y¯ = ¯y(t, x)and the optimal control termB¯u= (B¯u)(t, x).
According to the small regularization parameterσu, the control term gets singular on the crack t= x where yQ jumps. The optimal state solution sub-stantially resembles the discontinuous shape of the desired state, but we still recognize a smoothing of the edge and a slight shift of the jump.
0
Fig. 4.25 presents the reduced order model solution gained by OS-POD. We see that a good approximation of the state solution is provided while the ROM control strongly dif-fers from the high-fidelity optimal control. Of course, the strict convexity of the objective function J and hence the uniqueness of the control solution gets lost due to the slight consideration of control costs. Accordingly, the control residual and the corresponding a-posteriori error bound do not provide practicable measures for the ROM efficiency, one should regard the residual of the state instead which dominates the value J in this configuration.
Figure 4.25:The optimal state and the optimal control of the rank-12reduced order model. Here, the mutual updating of the POD basis and solving of the coupled reduced optimality system proposed in Alg. 3.1a is required to obtain good POD bases. The ROM optimization process stops after five OS-POD steps, each initialized by four gradient steps, reaching a state residual of k¯y−¯y`kL2 < 1.0e-02. The corresponding objective residual
|J(¯u)−J(¯u`)|decays below the squared level 1.0e-04 since the control residual which ammounts to the value k¯u−u¯`k2U =3.40e+05 is compensated by the cost parameter σu =5.0e-10 and hence has no impact on the magnitude ofJthis time.
In Fig. 4.26 we illustrate the improving quality of the optimal reduced-order statey¯` for increasing ROM ranks `. We recognize that all state solutions are covering uniformly distributed oscillations; for increasing`, the frequency of the disturbance increases while its amplitude decreases, improving the L2 approximation ofy¯in total.
0
Fig. 4.27 shows that the ROM control and state errors can be reduced on the level 1.0e-04 both with iterative basis updates and different numbers of OS-POD gradient steps. The qualitative difference is the number of POD basis elements required to reach this level:
The more effort is invested in the basis construction, the less POD functions are required to build up an accurate reduced-order model.
With an adaptive basis update x, at least 24 POD elements are needed to reach the desired exactness. If the basis construction is provided by a gradient step •, one can do with nine basis elements less; when performing two OS-POD steps ?, twelve POD elements are sufficient and with four OS-POD steps, the exactness of the rank-10 model is satisfactory already.
Since the effort needed to solve the reduced-order optimality system is reduced for smaller basis ranks while the effort of getting an accurate, short basis of the requested quality by an increasing number of OS-POD steps grows, we have a closer look at these competitive effects.
5 10 15 20
OS-POD vs. iterative POD updates Iterative Updates
OS-POD vs. iterative POD updates Iterative Updates One Gradient Step Two Gradient Steps Four Gradient Steps
Figure 4.27:The control residuals (left) and state residuals (right) for different POD bases.
Fig. 4.28 shows how the expense of the basis construction and the resulting effort of the ROM solving balance; in the example concerned here, the four OS-POD steps pay at the end: Compared to the adaptive updating, the calculation time for the basis construction nearly increases by the factor five (from 23.40 sec to 112.56 sec), but the computational effort of the ROM solving decreases to 26.34% in return (from 394.38 sec to 103.89 sec).
1 2 3 4
OS-POD vs. iterative POD updates Calculate error bound Solve optimality system Solve eigenvalue problem Provide gradient step Calculate snapshots
Figure 4.28: The different calculation times needed to reduce the ROM error in the state variable on the level 1.0e-04.
Comparing the iterative updates with the four OS-POD initialization steps, the computational effort is distributed as follows across the single ROM steps:
Process Alg. 3.2a Alg. 3.1a
Determine snapshots 3.46 sec 1.37 sec Provide gradient step 51.54 sec 0.00 sec Solve eigenvalue problem 57.56 sec 22.03 sec Solve reduced system 103.89 sec 394.38 sec Evaluate error estimator 2.31 sec 4.55 sec
Total 222.48 sec 427.69 sec
Up to now, we have not considered problems where control and (penalized) state con-straints are active at the same time. We have seen that for any regularization parameter ε >0, this problem type can be transformed to an optimization task with pure control constraints. Numerically, however, problems similar to those appearing for pure state constraints may arise if the time-space grid is chosen too coarse in comparison with the smallness ofε.
In the following, we consider a larger control cost parameter σu = 1.0e-04, the penal-ization parameter σw = 1.0e-04 and the regularization parameter ε = 1.0e-05 of the penalized state constraints. We take into account pointwise state bounds y(t, x)≤0 on the whole domainΩ, i.e.I =Id.
Fig. 4.29 shows the desired state, the optimal state, the optimal control and the optimal penalty. We observe that the smoothing and shifting of the jump inyQ is stronger here;
a close approximationy¯≈yQ as before is not possible since control costs are taken into account this time. Similar to Fig. 4.4, the activity of w essentially is limited to the boundary between the active and inactive domains of the state. Of course, w= 0holds where y takes negative values. The magnitude of the maximal pointwise state bound violations is limited by ε· kwk∞=3.57e-06.
0
Figure 4.29:The desired stateyQ=yQ(t, x), the optimal state solutiony¯= ¯y(t, x), the optimal control term B¯u= (B¯u)(t, x)and the optimal penalty variablew¯= (Iw)(t, x).¯
In [59], Run 2, we observed an appropriate behavior for transport dominated convection-diffusion equations with active control and state constraints of the type discribed in Ex.
1.16.1. Fig. 4.30 shows the performance and calculation times of different ROM models.
10 20 30 40
Control errors for different Pod bases Error(Update0)
Figure 4.30:We observe that in contrast to the previous test, the control residuals corresponding to the initial basisxstagnate independent of the chosen basis rank: The model definitely requires a basis update. The different basis update strategies – one adaptive step•, two adaptive steps?or one OS-POD step– result in decay orders comparable to the optimal one.
Concerning the computational effort, the initial basis, the OS-POD update and the optimal POD elements lead to nearly identical calculation times. However, the optimal model is not available in practice and the initial one is not accurate. The adaptive updates cause up to twice as much calculation time as OS-POD; with model rank
`= 40, this effort reaches the one of the full-order model while OS-POD can save 50% of the calculation time.
We finish our numerical test runs with a state constrained optimal control problem in 2D spatial dimensions. Here, the POD model reduction is more efficient than in 1D due to the so-called “curse of dimensionality“: Balancing the time steps and the spatial cell diameters, the dG(0)cG(1) discretization suggests to choose ∆t ∼ ∆x2, so a uniform discretization of the intervals Ω = [0,1], Θ = [0,1] leading to an accuracy of the order ε = 1.0e-04 requires 10000 time points and 100 spatial points; a rank-10 POD basis reduces the computational effort of solving the reduced-order optimality system down to 10%. In contrast, a twodimensional domain Ω = [0,1]×[0,1]requires 10000 spatial points, so a rank-50 POD basis still reduces the calculation time down to 0.5%.
Run 5. LetΩ = [0, π]×[0, π], Θ = [0,π2]and yΩ(x1, x2) = min(sin(x1) sin(x2),12). Fig.
4.31 shows the desired state and the optimal state, control and penalty.
0
0 1
2 3 0
2 0 1 2
directionx1 optimal control
directionx2 Bu(T,x1,x2)
0
2 0
2
−0.2
−0.1 0
directionx1
optimal penalty
direction x2
Iw(T,x1,x2)
Figure 4.31:The desired stateyQ=yQ(t, x), the optimal state solutiony¯= ¯y(T , x), the optimal control term B¯u= (B¯u)(T , x)and the optimal penalty variablew¯= (Iw)(T , x).¯
Fig. 4.32 presents the behavior of adaptive vs. OS-POD basis updates for this example.
To avoid the effort of solving the full-order optimization problem, we compare the a-posteriori error estimates instead fo the control errors themselves this time. One observes that the adaptive basis construction fails here: The POD errors of the initial control guess u˜ ≡ 1 x stagnate on the level 3.0e+00, one adaptive basis update • leads to no improvement and a second update ? even results in divergence of the optimizer, destabilizing the ROM system components for ` = 30; with further updates, this effect occurs for all ranks `. Notice that a failure of the optimization process prevents to continue with the adaptive ROM routine since no control output is available to generate new snapshots and new POD elements. On the other hand, the OS-POD strategy works here, iterative gradient steps continuously improve the reduced-order model and induce a slow reduction of the POD error.
10 20 30 40
10−1 100 101 102
Pod basis rankℓ
controlerrors
Ospod gradient steps vs. iterative basis updates No Update
One Update Two Updates Ospod
Figure 4.32: The a-posteriori error bounds for the adaptive and the OS-POD basis construction strategy.
In [61] we already studied this example on a coarse grid; there, one adaptive basis update managed to de-crease the ROM error below 1.0e-01. Furthermore, the a-posteriori error bounds were close to the actual con-trol error; the overestimation was of a factor smaller than 10.
Fig. 4.33 shows the improvement of the third POD element after one, two, four and eight OS-POD steps.
0 1
2 3 0
2
−1 0 1
directionx1
3rd pod basis element
directionx2
ψ12(x1,x2)
0 1
2 3 0
2
−1 0 1
directionx1
3rd pod basis element
directionx2
ψ12(x1,x2)
0 1
2 3 0
2
−1 0 1
directionx1 3rd pod basis element
directionx2 ψ12(x1,x2)
0 1
2 3 0
2
−1 0 1
directionx1 3rd pod basis element
directionx2 ψ12(x1,x2)
Figure 4.33:The third POD element on different update levels.
Matching the numerical results we presented here, we demonstrated in [59], Run 1, that also for boundary control problems of the type presented in Ex. 1.16.2, the POD elements generated by several OS-POD steps quickly converge towards the elements of the optimal POD basis. Further, the OS-POD model indeed matches the approximation quality of the optimal POD model there.
Conclusion
Several aspects of reduced order modeling adressed in this work can be continued: While we fixed the regularization and cost parameters in our analytical investigations, a connec-tion of the recent convergence analysis postulated in the menconnec-tioned references with POD and OS-POD is motivated by our numerical tests. Another challenging concept is the extension of the presented convergence analysis for the iterative solving of the OS-POD problem on the more efficient techniques we used in our modified algorithms, providing improved bounds and needing less restrictive assumptions on the data.
Up to now, the performance of OS-POD for nonlinear partial differential equations to-gether with control and state constraints has not been considered. A generalization of the presented results on the field of nonlinear fluid dynamics is desirable [30], concerning the actual activity of this field on one hand and the mathematical challenges arising for this problem type on the other hand.
Further, different requirements arise in the context of closed-loop control with POD model order reduction for infinite horizon control problems: Determining a feedback law via dynamic programming can be provided by solving a corresponding Hamilton-Jacobi-Bellman equation which in practice is not possible for more than, let’s say, five equations [2]. Gaining accurate POD models of such a small rank may exceed the linear subspace methods we discussed here even if OS-POD is applied; a combination with a nonlinear approximation method such as the so-called parametrized manifolds [37] could be an appropriate way to deal with these demands.
Bibliography
[1] Afanasiev, K. & Hinze, M.: Adaptive control of a wake flow using proper orthogonal decomposition. Lecture Notes Pure Appl. Math.,vol. 216: pp. 317–332, 2001.
[2] Alla, A., Falcone, M. & Volkwein, S.: Error analysis for POD approximations of infinite horizon problems via the dynamic programming approach. Konstanzer Schriften in Mathematik,vol. 339: pp. 1–25, 2015.
[3] Allain, P., Courty, N. & Corpetti, T.: Crowd Flow Characterization with Optimal Control Theory. Computer Vision ACCV 2009,vol. 5995: pp. 279–290, 2010.
[4] Alt, H. W.: Lineare Funktionalanalysis. Springer Lehrbuch, 6th ed., 2012.
[5] Altmüller, N.: Model Predictive Control for Partial Differential Equations. Ph.D.
thesis, Universität Bayreuth, 2014.
[6] Amann, H.: Gewöhnliche Differentialgleichungen. de Gruyter Lehrbuch, 2nd ed., 1995.
[7] Amann, H.: Analysis 1. Birkhäuser, 3rd ed., 2006.
[8] Amann, H.: Analysis 2. Birkhäuser, 3rd ed., 2008.
[9] Antoulas, A. C.: Approximation of Large-Scale Dynamical Systems. SIAM Ad-vances in Design and Control, 2005.
[10] Arada, N., Casas, E. & Tröltzsch, F.: Error Estimates for the Numerical Approxi-mation of a Semilinear Elliptic Control Problem. Comp. Opt. Appl.,vol. 23: pp.
201–229, 2002.
[11] Arada, N. & Raymond, J. P.: Optimal control problems with mixed control-state constraints. SIAM J. Contr. Optim.,vol. 39, no. 5: pp. 1391–1407, 2000.
[12] Arian, E., Fahl, M. & Sachs, E.: Trust-region proper orthogonal decomposition for flow control. Technical Report 2000, ICASE,vol. 25: pp. 1–15, 2000.
[13] Astolfi, A.: Model reduction by moment matching for linear and nonlinear systems.
IEEE Trans. Auto. Contr.,vol. 55, no. 10: pp. 2321–2336, 2010.
[14] Baker, G., Bramble, J. & Thomeé, V.: Single step Galerkin approximations for parabolic problems. Math. Comp.,vol. 31: pp. 818–847, 1977.
[15] Barrault, M., Maday, Y., Nguyen, N. C. et al.: An empirical interpolation method:
application to efficient reduced-basis discretization of partial differential equations.
Comp. Rend. Math.,vol. 139, no. 9: pp. 667–672, 2004.
[16] Bartee, J., Noll, P., Axelrud, C. et al.: Industrial Application of Nonlinear Model Predictive Control Technology for Fuel Ethanol Fermentation Process. Amer. Contr.
Conf.,vol. 09: pp. 2290–2294, 2009.
[17] Bartels, S.: Numerical Methods for Nonlinear Partial Differential Equations, vol. 47. Springer Series in Computational Mathematics, 2010.
[18] Bärwolff, G. & Hinze, M.: Optimization of semiconductor melts. ZAMM, vol. 86, no. 6: pp. 423–437, 2006.
[19] Belgacem, F. B., Fekih, H. E. & Raymond, J.: A penalized Robin approach for solving a parabolic equation with nonsmooth Dirichlet boundary conditions. Asymp.
Anal.,vol. 34, no. 2: pp. 121–136, 2003.
[20] Berggren, M.: Approximations of very weak solutions to boundary-value problems.
SIAM J. Numer. Anal.,vol. 42, no. 2: pp. 860–877, 2004.
[21] Bergounioux, M., Haddou, M., Hintermüller, M. et al.: A comparison of a Moreau-Yosida based active set strategy and interior point methods for constrained optimal control problems. SIAM J. Optim.,vol. 11, no. 2: pp. 495–521, 2000.
[22] Bergounioux, M., Ito, K. & Kunisch, K.: Primal-Dual Strategy for constrained optimal control problems. SIAM J. Contr. Optim., vol. 37, no. 4: pp. 1176–1194, 1999.
[23] Bergounioux, M. & Kunisch, K.: Primal-Dual Strategy for State-Constrained Op-timal Control Problems. Comp. Opt. Appl.,vol. 20: pp. 193–224, 2002.
[24] Berkooz, G., Holmes, P. & Lumley, J. L.: The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows. Annu. Rev. Fluid Mech., vol. 25: pp. 539–575, 1993.
[25] Borwein, J. M., Bradley, D. M. & Crandall, R. E.: Computational strategies for the Riemann zeta function. Comp. Appl. Math., vol. 121: pp. 247–296, 2000.
[26] Botkin, N. D.: Identification of unknown parameters for heat conductivity equa-tions. Num. Funct. Anal. Opt., vol. 16, no. 5 & 6: pp. 583–599, 1995.
[27] Braess, D.: Finite Elemente. Springer-Lehrbuch, 5th ed., 2013.
[28] Bramble, J. H. & Hilbert, S. R.: Estimation of linear functionals on Sobolev spaces with applications to Fourier transforms and spline interpolation. SIAM J. Numer.
Anal.,vol. 7: pp. 112–124, 1970.
[29] Buffa, A., Maday, Y., Patera, A. T. et al.: A priori convergence of the Greedy algorithm for the parametrized reduced basis method. ESAIM M2AN, vol. 46, no. 3: pp. 595–603, 2012.
[30] Burkardt, J., Gunzburger, M. & Lee, H.-C.: POD and CVT-based reduced-order modeling of Navier-Stokes flows. Comp. Appl. Mech.,vol. 196: pp. 337–355, 2006.
[31] Casas, E.: Pontryagin’s Principle for State-Constrained Boundary Control Prob-lems of Semilinear Parabolic Equations. SIAM J. Contr. Optim., vol. 35, no. 4:
pp. 1297–1327, 1997.
[32] Casas, E. & Tröltzsch, F.: Error Estimates for Linear-Quadratic Elliptic Control Problems. IFIP Analysis and Optimization of Differential Systems,vol. 121: pp.
89–100, 2003.
[33] Chapelle, D., Gariah, A. & Sainte-Marie, J.: Galerkin approximation with Proper Orthogonal Decomposition: New error estimates and illustrative examples. ESAIM:
M2AN,vol. 46, no. 4: pp. 731–757, 2011.
[34] Chatelin, F.: Spectral Approximation of Linear Operators. Computer Science and Applied Mathematics. Academic Press, 1983.
[35] Chatterjee, A.: An introduction to the proper orthogonal decomposition. Current Sciences,vol. 78, no. 7: pp. 808–817, 2000.
[36] Chaturantabut, S. & Sorensen, D. C.: Nonlinear model reduction via discrete em-pirical interpolation. SIAM J. Sci. Comp., vol. 32, no. 5: p. 2737â2764, 2010.
[37] Chekroun, M. & Liu, H.: Finite-Horizon Parameterizing Manifolds, and Applica-tions to Suboptimal Control of Nonlinear Parabolic PDEs. Acta Appl Math, vol.
135: pp. 81–144, 2015.
[38] Ciarlet, P. G.: The Finite Element Method for Elliptic Problems. SIAM Society for Industrial and Applied Mathematics, 2nd ed., 2002.
[39] Condon, M. & Ivanov, R.: Empirical Balanced Truncation of Nonlinear Systems.
J. Nonlin. Sci.,vol. 14: pp. 405–414, 2004.
[40] Conway, J. B.: A Course in Functional Analysis, vol. 96. Springer Graduate texts in mathematics, 1985.
[41] Dahmen, W. & Reusken, A.: Numerik für Ingenieure und Naturwissenschaftler.
Springer-Lehrbuch, 2nd ed., 2007.
[42] De Asmundis, R., di Serafino, D., Hager, W. W. et al.: An efficient gradient method using the Yuan steplength. Comp. Opt. Appl.,vol. 59: pp. 541–563, 2014.
[43] Domahidi, A., Zeilinger, M. N., Morari, M. et al.: Learning a feasible and stabilizing explicit model predictive control law by robust optimization. IEEE Dec. Contr., vol. 50: pp. 513–519, 2011.
[44] Dones, I., Skogestad, S. & Preisig, H. A.: Application of Balanced Truncation to Nonlinear Systems. Ind. Eng. Chem. Res.,vol. 50, no. 17: pp. 10093–10101, 2011.
[45] Dunford, N. & Schwartz, J. T.: Spectral theory, self adjoint operators in Hilbert space. 7, Pure and Applied Mathematics, 1963.
[46] Edwards, H.: Riemann’s Zeta function,Pure Appl. Math., vol. 58. Academic Press, 1974.
[47] Eriksson, K., Johnson, C. & Thomée, V.: Time discretization of parabolic problems by the discontinuous Galerkin method. M2AN,vol. 19, no. 4: pp. 611–643, 1985.
[48] Evans, L. C.: Partial Differential Equations, Graduate Studies in Mathematics, vol. 19. American Mathematical Society, 4th ed., 1998.
[49] Fisher, M., Nocedal, J., Trémolet, Y. et al.: Data Assimilation in Weather
Fore-casting: A Case Study in PDE-Constrained Optimization. Optim. Eng., vol. 10:
pp. 409–426, 2009.
[50] Forster, O.: Analysis 1. Vieweg+Teubner Verlag, 10th ed., 2011.
[51] Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, New York, 1990.
[52] Gautschi, W.: Leonhard Eulers Umgang mit langsam konvergenten Reihen. Elem.
Math.,vol. 62, no. 07: pp. 1–10, 2007.
[53] Gerdts, M.: Optimal Control of ODEs and DAEs. De Gruyter Textbook, 2012.
[54] Ghattas, O. & Bark, J.-H.: Optimal Control of Two- and Three-Dimensional In-compressible Navier-Stokes Flows. J. Comp. Phys., vol. 136: pp. 231–244, 1997.
[55] Gong, W., Hinze, M. & Zhou, Z.: Space-time finite element approximation of parabolic optimal control problems. Numer. Math., vol. 20, no. 2: pp. 111–146, 2012.
[56] Gong, W., Hinze, M. & Zhou., Z.: A finite element method for Dirichlet bound-ary control problems governed by parabolic PDEs. Hamburger Beiträge zur Ange-wandten Mathematik,vol. 2014-19: pp. 1–21, 2014.
[57] Grepl, M. A.: Reduced-Basis Approximation and A Posteriori Error Estimation for Parabolic Differential Equations. Ph.D. thesis, PhD thesis, Massachusetts Institute for Technology, 2005.
[58] Griepentrog, J. A.: Maximal regularity for nonsmooth parabolic problems in Sobolev-Morrey spaces. Adv. Diff. Equ.,vol. 12: pp. 1031–1078, 2007.
[59] Grimm, E., Gubisch, M. & Volkwein, S.: A-Posteriori Error Analysis and Optimality-System POD for Constrained Optimal Control. Comp. Sci. Eng., vol.
105: pp. 297–317, 2015.
[60] Gubisch, M., Neitzel, I. & Volkwein, S.: A-Posteriori Error Estimates for POD Model Order Reduction Respecting High-order Discretitation Errors.
preprint to appear in: Konstanzer Online-Publikationssystem (KOPS), Jan. 2016, http://kops.uni-konstanz.de/.
[61] Gubisch, M. & Volkwein, S.: POD a-posteriori error analysis for optimal control Problems with mixed control-state constraints. Comp. Opt. Appl.,vol. 58, no. 3:
pp. 619–644, 2014.
[62] Gubisch, M. & Volkwein, S.: Proper Orthogonal Decomposition for Linear-Quadratic Optimal Control. to appear in: Model Reduction and Approximation:
Theory and Algorithms. SIAM, Philadelphia, PA. preprint available: http://nbn-resolving.de/urn:nbn:de:bsz:352-250378, pp. 1–63, 2016.
[63] Gugercin, S. & Athanasios, A. C.: A survey of model reduction by balanced
[63] Gugercin, S. & Athanasios, A. C.: A survey of model reduction by balanced