×

Data-driven switching modeling for MPC using regression trees and random forests. (English) Zbl 1441.93078

Summary: Model predictive control is a well consolidated technique to design optimal control strategies, leveraging the capability of a mathematical model to predict a system’s behavior over a time horizon. However, building physics-based models for complex large-scale systems can be cost and time prohibitive. To overcome this problem we propose a methodology to exploit machine learning techniques (i.e. regression trees and random forests) in order to build a switching affine dynamical model (deterministic and Markovian) of a large-scale system using historical data, and apply model predictive control. A comparison with an optimal benchmark and related techniques is provided on an energy management system to validate the performance of the proposed methodology.

MSC:

93B45 Model predictive control
93E03 Stochastic systems in control theory (general)
93C30 Control/observation systems governed by functional relations other than differential equations (such as hybrid and switching systems)
93A15 Large-scale systems

Software:

EnergyPlus
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Lun, Y. Z.; D’Innocenzo, A.; Smarra, F.; Malavolta, I.; Di Benedetto, M. D., State of the art of cyber-physical systems security: An automatic control perspective, J. Syst. Softw., 149, 174-216 (2019)
[2] Ma, Y.; Matusko, J.; Borrelli, F., Stochastic model predictive control for building HVAC systems: complexity and conservatism, IEEE Trans. Control Syst. Technol., 23, 1, 101-116 (2015)
[3] Oldewurtel, F.; Parisio, A.; Jones, C. N.; Gyalistras, D.; Gwerder, M.; Stauch, V.; Lehmann, B.; Morari, M., Use of model predictive control and weather forecasts for energy efficient building climate control, Energy Build., 45, 15-27 (2012)
[4] Maasoumy, M.; Razmara, M.; Shahbakhti, M.; Vincentelli, A. S., Handling model uncertainty in model predictive control for energy efficient buildings, Energy Build., 77, 377-392 (2014)
[5] Iovine, A.; Rigaut, T.; Damm, G.; De Santis, E.; Di Benedetto, M. D., Power management for a DC microgrid integrating renewables and storages, Control Eng. Pract., 85, 59-79 (2019)
[6] Venkat, A. N.; Hiskens, I. A.; Rawlings, J. B.; Wright, S. J., Distributed MPC strategies with application to power system automatic generation control, IEEE Trans. Control Syst. Technol., 16, 6, 1192-1206 (2008)
[7] Kennel, F.; Gorges, D.; Liu, S., Energy management for smart grids with electric vehicles based on hierarchical MPC, IEEE Trans. Ind. Inform., 9, 3, 1528-1537 (2013)
[8] Sturzenegger, D.; Gyalistras, D.; Morari, M.; Smith, R. S., Model predictive climate control of a swiss office building: Implementation, results, and cost-benefit analysis, IEEE Trans. Control Syst. Technol., 24, 1, 1-12 (2016)
[9] Žáčeková, E.; Váňa, Z.; Cigler, J., Towards the real-life implementation of MPC for an office building: Identification issues, Appl. Energy, 135, 53-62 (2014)
[10] Behl, M.; Smarra, F.; Mangharam, R., DR-Advisor: A data-driven demand response recommender system, Appl. Energy, 170, 30-46 (2016)
[11] Jain, A.; Smarra, F.; Mangharam, R., Data predictive control using regression trees and ensemble learning, (Decision and Control (CDC), 2017 IEEE 56th Annual Conference on (2017), IEEE), 4446-4451
[12] Di Girolamo, G.; Smarra, F.; Gattulli, V.; Potenza, F.; Graziosi, F.; D’Innocenzo, A., Data-driven optimal predictive control of seismic induced vibrations in frame structures, Struct. Control Health Monit. (2020)
[13] Smarra, F.; Jain, A.; Mangharam, R.; D’Innocenzo, A., Data-driven switched affine modeling for model predictive control, (IFAC Conference on Analysis and Design of Hybrid Systems, ADHS’18 (2018), IFAC), 199-204
[14] Smarra, F.; Jain, A.; de Rubeis, T.; Ambrosini, D.; D’Innocenzo, A.; Mangharam, R., Data-driven model predictive control using random forests for building energy optimization and climate control, Appl. Energy (2018)
[15] Costa, O. L.V.; Fragoso, M. D.; Marques, R. P., Discrete-Time Markov Jump Linear Systems (2006), Springer Science & Business Media · Zbl 1081.93001
[16] Bernardini, D.; Bemporad, A., Stabilizing model predictive control of stochastic constrained linear systems, IEEE Trans. Automat. Control, 57, 6, 1468-1480 (2012) · Zbl 1369.93687
[17] Patrinos, P.; Sopasakis, P.; Sarimveis, H.; Bemporad, A., Stochastic model predictive control for constrained discrete-time Markovian switching systems, Automatica, 50, 10, 2504-2514 (2014) · Zbl 1301.93173
[18] Müller, M. A.; Martius, P.; Allgöwer, F., Model predictive control of switched nonlinear systems under average dwell-time, J. Process Control, 22, 9, 1702-1710 (2012)
[19] Zhang, L.; Zhuang, S.; Braatz, R. D., Switched model predictive control of switched linear systems: Feasibility, stability and robustness, Automatica, 67, 8-21 (2016) · Zbl 1335.93041
[20] L. Bridgeman, C. Danielson, S. Di Cairano, Stability and feasibility of MPC for switched linear systems with dwell-time constraints, in: American Control Conference, ACC, 2016.
[21] Hespanha, J. P., Uniform stability of switched linear systems: Extensions of LaSalle’s invariance principle, IEEE Trans. Automat. Control, 49, 4, 470-482 (2004) · Zbl 1365.93348
[22] Liberzon, D.; Hespanha, J. P.; Morse, A. S., Stability of switched systems: a Lie-algebraic condition, Systems Control Lett., 37, 3, 117-122 (1999) · Zbl 0948.93048
[23] Xie, G.; Zheng, D.; Wang, L., Controllability of switched linear systems, IEEE Trans. Automat. Control, 47, 8, 1401-1405 (2002) · Zbl 1364.93075
[24] Sun, Z.; Ge, S. S.; Lee, T. H., Controllability and reachability criteria for switched linear systems, Automatica, 38, 5, 775-786 (2002) · Zbl 1031.93041
[25] Lazar, M.; Heemels, W.; Weiland, S.; Bemporad, A., Stabilizing model predictive control of hybrid systems, IEEE Trans. Automat. Control, 51, 11, 1813-1818 (2006) · Zbl 1366.93502
[26] Bemporad, A.; Morari, M., Control of systems integrating logic, dynamics, and constraints, Automatica, 35, 3, 407-427 (1999) · Zbl 1049.93514
[27] Kenanian, J.; Balkan, A.; Jungers, R. M.; Tabuada, P., Data driven stability analysis of black-box switched linear systems, Automatica, 109, 108533 (2019) · Zbl 1429.93266
[28] Oldewurtel, F., Stochastic Model Predictive Control for Energy Efficient Building Climate Control (2011), ETH Zurich, (Ph.D. thesis)
[29] Lauer, F., Estimating the probability of success of a simple algorithm for switched linear regression, Nonlinear Anal. Hybrid Syst., 8, 31-47 (2013) · Zbl 1257.93105
[30] Ljung, L., Perspectives on system identification, Annu. Rev. Control, 34, 1, 1-12 (2010)
[31] Zadeh, L., On the identification problem, IRE Trans. Circuit Theory, 3, 4, 277-281 (1956)
[32] Ljung, L., System Identification: Theory for the User (1998), Pearson Education
[33] Katayama, T., Subspace Methods for System Identification (2006), Springer Science & Business Media
[34] Nelles, O., Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models (2013), Springer Science & Business Media
[35] Lauer, F.; Bloch, G., Hybrid System Identification: Theory and Algorithms for Learning Switching Models, vol. 478 (2018), Springer
[36] Chen, S.; Billings, S.; Grant, P., Non-linear system identification using neural networks, Int. J. Control, 51, 6, 1191-1214 (1990) · Zbl 0703.93071
[37] P.M.L. Drezet, R.F. Harrison, Support vector machines for system identification, in: UKACC International Conference on Control ’98 (Conf. Publ. No. 455), vol. 1, 1998, pp. 688-692.
[38] Martínez-Ramón, M.; Rojo-Alvarez, J. L.; Camps-Valls, G.; Muñoz-Marí, J.; Soria-Olivas, E.; Figueiras-Vidal, A. R., Support vector machines for nonlinear kernel ARMA system identification, IEEE Trans. Neural Netw., 17, 6, 1617-1622 (2006)
[39] Pillonetto, G.; Dinuzzo, F.; Chen, T.; De Nicolao, G.; Ljung, L., Kernel methods in system identification, machine learning and function estimation: A survey, Automatica, 50, 3, 657-682 (2014) · Zbl 1298.93342
[40] Lauer, F., On the complexity of piecewise affine system identification, Automatica, 62, 148-153 (2015) · Zbl 1329.93052
[41] Vidal, R.; Soatto, S.; Ma, Y.; Sastry, S., An algebraic geometric approach to the identification of a class of linear hybrid systems, (42nd IEEE International Conference on Decision and Control (IEEE Cat. No. 03CH37475), vol. 1 (2003), IEEE), 167-172
[42] Lauer, F.; Bloch, G.; Vidal, R., A continuous optimization framework for hybrid system identification, Automatica, 47, 3, 608-613 (2011) · Zbl 1219.93030
[43] Bemporad, A.; Garulli, A.; Paoletti, S.; Vicino, A., A bounded-error approach to piecewise affine system identification, IEEE Trans. Automat. Control, 50, 10, 1567-1580 (2005) · Zbl 1365.93100
[44] Garulli, A.; Paoletti, S.; Vicino, A., A survey on switched and piecewise affine system identification, IFAC Proc. Vol., 45, 16, 344-355 (2012)
[45] Pekpe, K. M.; Mourot, G.; Gasso, K.; Ragot, J., Identification of switching systems using change detection technique in the subspace framework, (2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No. 04CH37601) vol. 4 (2004), IEEE), 3720-3725
[46] Verdult, V.; Verhaegen, M., Subspace identification of piecewise linear systems, (2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No. 04CH37601), vol. 4 (2004), IEEE), 3838-3843
[47] Borges, J.; Verdult, V.; Verhaegen, M.; Botto, M. A., A switching detection method based on projected subspace classification, (Proceedings of the 44th IEEE Conference on Decision and Control (2005), IEEE), 344-349
[48] Lopes, R. V.; Borges, G. A.; Ishihara, J. Y., New algorithm for identification of discrete-time switched linear systems, (2013 American Control Conference (2013), IEEE), 6219-6224 · Zbl 0929.17017
[49] Lauer, F.; Bloch, G., Piecewise smooth system identification in reproducing kernel Hilbert space, (53rd IEEE Conference on Decision and Control (2014), IEEE), 6498-6503
[50] Tugnait, J., Adaptive estimation and identification for discrete systems with Markov jump parameters, IEEE Trans. Automat. Control, 27, 5, 1054-1065 (1982) · Zbl 0494.93045
[51] Jin, X.; Huang, B., Identification of switched Markov autoregressive exogenous systems with hidden switching state, Automatica, 48, 2, 436-441 (2012) · Zbl 1260.93166
[52] Krishnamurthy, V.; Moore, J. B., On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure, IEEE Trans. Signal Process., 41, 8, 2557-2573 (1993) · Zbl 0825.93742
[53] Orguner, U.; Demirekler, M., An online sequential algorithm for the estimation of transition probabilities for jump Markov linear systems, Automatica, 42, 10, 1735-1744 (2006) · Zbl 1114.93103
[54] Bemporad, A.; Breschi, V.; Piga, D.; Boyd, S. P., Fitting jump models, Automatica, 96, 11-21 (2018) · Zbl 1406.93315
[55] Ernst, D.; Glavic, M.; Capitanescu, F.; Wehenkel, L., Reinforcement learning versus model predictive control: A comparison on a power system problem, IEEE Trans. Syst. Man Cybern. B, 39, 2, 517-529 (2009)
[56] Lewis, F. L.; Vrabie, D., Reinforcement learning and adaptive dynamic programming for feedback control, IEEE Circuits Syst. Mag., 9, 3, 32-50 (2009)
[57] Recht, B., A tour of reinforcement learning: The view from continuous control, Ann. Rev. Control Robot. Auton. Syst., 2, 1, 253-279 (2019)
[58] Gorges, D., Relations between model predictive control and reinforcement learning, IFAC-PapersOnLine, 50, 1, 4920-4928 (2017), 20th IFAC World Congress
[59] Sutton, R. S.; Barto, A. G., Introduction to Reinforcement Learning (1998), MIT Press: MIT Press Cambridge, MA, USA
[60] Gheshlaghi Azar, M.; Munos, R.; Kappen, B., On the sample complexity of reinforcement learning with a generative model (2012), arXiv e-prints arXiv:1206.6461
[61] Sutton, R. S., Dyna, an integrated architecture for learning, planning, and reacting, SIGART Bull., 2, 4, 160-163 (1991)
[62] Peng, B.; Li, X.; Gao, J.; Liu, J.; Wong, K.-F.; Su, S.-Y., Deep dyna-Q: Integrating planning for task-completion dialogue policy learning (2018), arXiv e-prints arXiv:1801.06176
[63] Weber, T.; Racanière, S.; Reichert, D. P.; Buesing, L.; Guez, A.; Jimenez Rezende, D.; Puigdomènech Badia, A.; Vinyals, O.; Heess, N.; Li, Y.; Pascanu, R.; Battaglia, P.; Hassabis, D.; Silver, D.; Wierstra, D., Imagination-augmented agents for deep reinforcement learning (2017), arXiv e-prints arXiv:1707.06203
[64] Nagabandi, A.; Kahn, G.; Fearing, R. S.; Levine, S., Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning (2017), arXiv e-prints arXiv:1708.02596
[65] Pyeatt, L. D., Reinforcement learning with decision trees, (Applied Informatics (2003))
[66] Ernst, D.; Geurts, P.; Wehenkel, L., Tree-based batch mode reinforcement learning, J. Mach. Learn. Res. (2005) · Zbl 1222.68193
[67] Kuss, M.; Rasmussen, C. E., GaussIan processes in reinforcement learning, (Advances in Neural Information Processing Systems (2004)), 751-758
[68] Wang, T.; Bao, X.; Clavera, I.; Hoang, J.; Wen, Y.; Langlois, E.; Zhang, S.; Zhang, G.; Abbeel, P.; Ba, J., Benchmarking model-based reinforcement learning (2019), arXiv e-prints arXiv:1907.02057
[69] Costanzo, G.; Iacovella, S.; Ruelens, F.; Leurs, T.; Claessens, B., Experimental analysis of data-driven control for a building heating system, Sustain. Energy Grids Netw., 6, 81-90 (2016)
[70] L. Eller, L.C. Siafara, T. Sauter, Adaptive control for building energy management using reinforcement learning, in: 2018 IEEE International Conference on Industrial Technology, ICIT, 2018, pp. 1562-1567.
[71] Ferreira, P.; Ruano, A.; Silva, S.; Conceicao, E., Neural networks based predictive control for thermal comfort and energy savings in public buildings, Energy Build., 55, 238-251 (2012)
[72] Afram, A.; Janabi-Sharifi, F.; Fung, A. S.; Raahemifar, K., Artificial neural network (ANN) based model predictive control (MPC) and optimization of HVAC systems: A state of the art review and case study of a residential HVAC system, Energy Build., 141, 96-113 (2017)
[73] Breiman, L., Classification and Regression Trees (2017), Routledge
[74] Breiman, L., Random forests, Mach. Learn., 45, 1, 5-32 (2001) · Zbl 1007.68152
[75] Jain, A.; Smarra, F.; Behl, M.; Mangharam, R., Data-driven model predictive control with regression trees-An application to building energy management, ACM Trans. Cyber-Phys. Syst., 2, 1, 4 (2018)
[76] Gyalistras, D.; Gwerder, M., Use of weather and occupancy forecasts for optimal building climate control (opticontrol): Two years progress report main report, (Terrestrial Systems Ecology (2010), ETH Zurich R&D HVAC Products, Building Technologies Division, Siemens Switzerland Ltd, Zug: ETH Zurich R&D HVAC Products, Building Technologies Division, Siemens Switzerland Ltd, Zug Switzerland), 83
[77] Crawley, D. B.; Lawrie, L. K.; Winkelmann, F. C.; Buhl, W. F.; Huang, Y. J.; Pedersen, C. O.; Strand, R. K.; Liesen, R. J.; Fisher, D. E.; Witte, M. J., Energyplus: creating a new-generation building energy simulation program, Energy Build., 33, 4, 319-331 (2001)
[78] Merkblatt, S., 2024: Standard-Nutzungsbedingungen für die Energie-und Gebäudetechnik (2006), Swiss Society of Engineers and Architects: Swiss Society of Engineers and Architects Zürich
[79] Hastie, T.; Tibshirani, R.; Friedman, J.; Hastie, T.; Friedman, J.; Tibshirani, R., The Elements of Statistical Learning, vol. 2, no. 1 (2009), Springer · Zbl 1273.62005
[80] Ripley, B. D., Pattern Recognition and Neural Networks (1996), Cambridge University Press: Cambridge University Press Cambridge · Zbl 0853.62046
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.