×

An overview for Markov decision processes in queues and networks. (English) Zbl 1473.90175

Li, Quan-Lin (ed.) et al., Stochastic models in reliability, network security and system safety. Essays dedicated to Professor Jinhua Cao on the occasion of his 80th birthday. Singapore: Springer. Commun. Comput. Inf. Sci. 1102, 44-71 (2019).
This valuable survey paper provides a detailed overview of research on Markov Decision Processes (MDPs) in queues and networks in the last six decades. The survey is related to research problems such as: Markov processes, queueing systems, queueing networks, Markov decision processes, sensitivity-based optimization, stochastic optimization, fluid and diffusion control. It presents birth of main ideas in the listed above scientific areas and their evolution together with many practical applications scattered in a huge number of references. It also points out a number of interesting directions for potential future research such as:
development of effective and efficient algorithms for finding optimal polices and computing optimal performance measures, and also their possible links to AI and learning methods;
optimal policies in the MDPs of queueing methods under intelligent environment e.g. Internet of Things (IoT), big data, cloud service, blockchain, AI;
analyzing of optimal policy in the MDPs with processes related to various practical stochastic models (birth-and-death (QBD) process, Markov process);
application of the sensitivity-based optimization and the event-based optimization to dynamic control networks such as: production and inventory control, manufacturing control, transportation networks, healthcare, sharing economics, cloud service, blockchain, service management, energy-efficient management.

This survey paper is based on ample bibliography consisting of 344 items. This work can be recommended as a very good guide on bibliography concerning MDPs in queues and networks and research related to them as well as their practical applications.
For the entire collection see [Zbl 1427.68014].

MSC:

90C40 Markov and semi-Markov decision processes
90B22 Queues and service in operations research
90B10 Deterministic network models in operations research
90-02 Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming

Software:

UMDES
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Abdel-Gawad, E.F.: Optimal control of arrivals and routing in a network of queues. Ph.D. dissertation, North Carolina State University (1984)
[2] Adlakha, S., Lall, S., Goldsmith, A.: Networked Markov decision processes with delays. IEEE Trans. Autom. Control 57(4), 1013-1018 (2012) · Zbl 1369.90183 · doi:10.1109/TAC.2011.2168111
[3] Adusumilli, K.M., Hasenbein, J.J.: Dynamic admission and service rate control of a queue. Queueing Syst. 66(2), 131-154 (2010) · Zbl 1207.90038 · doi:10.1007/s11134-010-9192-z
[4] Ahmed, M.H.: Call admission control in wireless networks: a comprehensive survey. IEEE Commun. Surv. Tutorials 7(1), 49-68 (2005) · doi:10.1109/COMST.2005.1423334
[5] Ahn, H.S., Duenyas, I., Lewis, M.E.: Optimal control of a two-stage tandem queuing system with flexible servers. Probab. Eng. Inf. Sci. 16(4), 453-469 (2002) · Zbl 1038.90018 · doi:10.1017/S0269964802164047
[6] Alsheikh, M.A., Hoang, D.T., Niyato, D., Tan, H.P., Lin, S.: Markov decision processes with applications in wireless sensor networks: a survey, pp. 1-29. arXiv preprint arXiv:1501.00644 (2015)
[7] Altman, E., Nain, P.: Optimal control of the M/G/1 queue with repeated vacations of the server. IEEE Trans. Autom. Control 38(12), 1766-1775 (1993) · Zbl 0810.90039 · doi:10.1109/9.250556
[8] Altman, E.: A Markov game approach for optimal routing into a queuing network. Ph.D. dissertation, INRIA (Institut National de Recherche en Informatique et en Automatique) (1994)
[9] Altman, E.: Non zero-sum stochastic games in admission, service and routing control in queueing systems. Queueing Syst. 23(1-4), 259-279 (1996) · Zbl 0877.90097 · doi:10.1007/BF01206560
[10] Altman, E.: Constrained Markov Decision Processes. CRC Press (1999) · Zbl 0963.90068
[11] Altman, E.: A Markov game approach for optimal routing into a queuing network. In: Bardi, M., Raghavan, T.E.S., Parthasarathy, T. (eds.) Stochastic and Differential Games, vol. 4, pp. 359-375. Birkhäuser, Boston (1999). https://doi.org/10.1007/978-1-4612-1592-9_9 · Zbl 0940.91013 · doi:10.1007/978-1-4612-1592-9_9
[12] Altman, E.: Applications of Markov decision processes in communication networks. In: Feinberg, E.A., Shwartz, A. (eds.) Handbook of Markov Decision Processes, vol. 40, pp. 489-536. Springer, Boston (2002). https://doi.org/10.1007/978-1-4615-0805-2_16 · Zbl 1003.90049 · doi:10.1007/978-1-4615-0805-2_16
[13] Altman, E., Gaujal, B., Hordijk, A.: Discrete-Event Control of Stochastic Networks: Multimodularity and Regularity. Springer, Heidelberg (2003). https://doi.org/10.1007/b93837 · Zbl 1104.93004 · doi:10.1007/b93837
[14] Altman, E., Hordijk, A.: Zero-sum Markov games and worst-case optimal control of queueing systems. Queueing Syst. 21(3-4), 415-447 (1995) · Zbl 0859.90141 · doi:10.1007/BF01149169
[15] Altman, E., Jiménez, T.: Admission control to an M/M/1 queue with partial information. In: Dudin, A., De Turck, K. (eds.) ASMTA 2013. LNCS, vol. 7984, pp. 12-21. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39408-9_2 · Zbl 1395.90112 · doi:10.1007/978-3-642-39408-9_2
[16] Altman, E., Jiménez, T., Koole, G.: On optimal call admission control in resource-sharing system. IEEE Trans. Commun. 49(9), 1659-1668 (2001) · Zbl 1009.90123 · doi:10.1109/26.950352
[17] Altman, E., Jiménez, T., Nún̄ez Queija, R., Yechiali, U.: Optimal routing among \(\cdot \)/M/1 queues with partial information. Stoch. Models 20(2), 149-171 (2004) · Zbl 1060.60087
[18] Altman, E., Nain, P.: Optimality of a threshold policy in the M/M/1 queue with repeated vacations. Math. Methods Oper. Res. 44(1), 75-96 (1996) · Zbl 0863.90065 · doi:10.1007/BF01246330
[19] Altman, E., Stidham, S.: Optimality of monotonic policies for two-action Markovian decision processes, with applications to control of queues with delayed information. Queueing Syst. 21(3-4), 267-291 (1995) · Zbl 0859.90118 · doi:10.1007/BF01149165
[20] Anderson, M.Q.: Optimal admission pricing and service rate control of an \(M^{\text{ X }} \)/M/s queue with reneging. Naval Res. Logistics 30(2), 261-270 (1983) · Zbl 0535.60092 · doi:10.1002/nav.3800300207
[21] Anderson, W.J.: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York (1991). https://doi.org/10.1007/978-1-4612-3038-0 · Zbl 0731.60067 · doi:10.1007/978-1-4612-3038-0
[22] Argon, N.T., Tsai, Y.C.: Dynamic control of a flexible server in an assembly-type queue with setup costs. Queueing Syst. 70(3), 233-268 (2012) · Zbl 1275.90016 · doi:10.1007/s11134-011-9269-3
[23] Asmussen, S.: Applied Probability and Queues. Springer, New York (2003). https://doi.org/10.1007/b97236 · Zbl 1029.60001 · doi:10.1007/b97236
[24] Ata, B.: Dynamic control of a multiclass queue with thin arrival streams. Oper. Res. 54(5), 876-892 (2006) · Zbl 1167.90439 · doi:10.1287/opre.1060.0308
[25] Atan, S.T.: Solution methods for controlled queueing networks. Ph.D. dissertation, Iowa State University (1997)
[26] Atar, R., Mandelbaum, A., Reiman, M.I.: Scheduling a multi class queue with many exponential servers: asymptotic optimality in heavy traffic. Ann. Appl. Probab. 14(3), 1084-1134 (2004) · Zbl 1049.60079 · doi:10.1214/105051604000000233
[27] Balsamo, S., de Nitto Personé, V., Onvural, R.: Analysis of Queueing Networks with Blocking. Springer, Dordrecht (2001). https://doi.org/10.1007/978-1-4757-3345-7 · Zbl 0977.68007 · doi:10.1007/978-1-4757-3345-7
[28] Bartroli, M.: On the structure of optimal control policies for networks of queues. Ph.D. dissertation, University of North Carolina at Chapel Hill (1989)
[29] Baskett, F., Chandy, K.M., Muntz, R.R., Palacios, F.G.: Open, closed, and mixed networks of queues with different classes of customers. J. ACM 22(2), 248-260 (1975) · Zbl 0313.68055 · doi:10.1145/321879.321887
[30] Bäuerle, N.: Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab. 10(4), 1065-1083 (2000) · Zbl 1057.90003 · doi:10.1214/aoap/1019487605
[31] Bäuerle, N.: Optimal control of queueing networks: an approach via fluid models. Adv. Appl. Probab. 34(2), 313-328 (2002) · Zbl 1027.90007 · doi:10.1239/aap/1025131220
[32] Bellman, R., Kalaba, R.E.: Dynamic Programming and Modern Control Theory. Academic Press, New York (1965) · Zbl 0139.04502
[33] Bertsekas, D.P.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (1995) · Zbl 0904.90170
[34] Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996) · Zbl 0924.68163
[35] Bertsimas, D., Paschalidis, I.C., Tsitsiklis, J.N.: Optimization of multiclass queueing networks: polyhedral and nonlinear characterizations of achievable performance. Ann. Appl. Probab. 4(1), 43-75 (1994) · Zbl 0797.60079 · doi:10.1214/aoap/1177005200
[36] Besbes, O., Maglaras, C.: Revenue optimization for a make-to-order queue in an uncertain market environment. Oper. Res. 57(6), 1438-1450 (2009) · Zbl 1233.90111 · doi:10.1287/opre.1080.0645
[37] Bhulai, S.: Dynamic routing policies for multiskill call centers. Probab. Eng. Inf. Sci. 23(1), 101-119 (2009) · Zbl 1155.90351 · doi:10.1017/S0269964809000096
[38] Bhulai, S., Brooms, A.C., Spieksma, F.M.: On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue. Queueing Syst. 76(4), 425-446 (2014) · Zbl 1327.90365 · doi:10.1007/s11134-013-9371-9
[39] Blackburn, J.D.: Optimal control of a single-server queue with balking and reneging. Manage. Sci. 19(3), 297-313 (1972) · Zbl 0267.60096 · doi:10.1287/mnsc.19.3.297
[40] Blanc, J.P.C., de Waal, P.R., Nain, P., Towsley, D.: Optimal control of admission to a multiserver queue with two arrival streams. IEEE Trans. Autom. Control 37(6), 785-797 (1992) · Zbl 0767.90015 · doi:10.1109/9.256332
[41] Bolch, G., Greiner, S., De Meer, H., Trivedi, K.S.: Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications. Wiley, New York (2006) · Zbl 1099.60002 · doi:10.1002/0471791571
[42] Bonomi, F.: On job assignment for a parallel system of processor sharing queues. IEEE Trans. Comput. 39(7), 858-869 (1990) · doi:10.1109/12.55688
[43] Boucherie, R.J., Van Dijk, N.M. (eds.): Queueing Networks: A Fundamental Approach. Springer, Boston (2011). https://doi.org/10.1007/978-1-4419-6472-4 · Zbl 1203.90004 · doi:10.1007/978-1-4419-6472-4
[44] Boucherie, R.J., Van Dijk, N.M. (eds.): Markov Decision Processes in Practice. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-47766-4 · Zbl 1369.90001 · doi:10.1007/978-3-319-47766-4
[45] Bradford, R.M.: Pricing, routing, and incentive compatibility in multiserver queues. Eur. J. Oper. Res. 89(2), 226-236 (1996) · Zbl 0913.90103 · doi:10.1016/0377-2217(95)00265-0
[46] Bramson, M.: Stability of Queueing Networks. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68896-9 · Zbl 1152.60004 · doi:10.1007/978-3-540-68896-9
[47] Browne, S., Yechiali, U.: Dynamic priority rules for cyclic-type queues. Adv. Appl. Probab. 21(2), 432-450 (1989) · Zbl 0672.60092 · doi:10.2307/1427168
[48] Browne, S., Yechiali, U.: Dynamic routing in polling systems. Teletraffic Sci. ITC-12, 1455-1466 (1989)
[49] Browne, S., Yechiali, U.: Dynamic scheduling in single-server multiclass service systems with unit buffers. Naval Res. Logistics 38(3), 383-396 (1991) · Zbl 0725.90033 · doi:10.1002/1520-6750(199106)38:3<383::AID-NAV3220380308>3.0.CO;2-D
[50] Brouns, G.A.: Queueing models with admission andtermination control: monotonicity and threshold results. Technische Universiteit Eindhoven, pp. 1-198 (2003)
[51] Brouns, G.A., Van Der Wal, J.: Optimal threshold policies in a two-class preemptive priority queue with admission and termination control. Queueing Syst. 54(1), 21-33 (2006) · Zbl 1137.90416 · doi:10.1007/s11134-006-8307-z
[52] Buzacott, J.A., Shanthikumar, J.G.: Stochastic Models of Manufacturing Systems. Prentice Hall (1993) · Zbl 0769.90043
[53] Cao, F.: Event-based optimization for the continuous-time Markov systems. Doctoral dissertation, Hong Kong University of Science and Technology, Hong Kong (2008)
[54] Cao, P., Xie, J.: Optimal control of an inventory system with joint production and pricing decisions. IEEE Trans. Autom. Control 61(12), 4235-4240 (2016) · Zbl 1359.90008 · doi:10.1109/TAC.2015.2510162
[55] Cao, P., Xie, J.: Optimal control of a multiclass queueing system when customers can change types. Queueing Syst. 82(3-4), 285-313 (2016) · Zbl 1341.60110 · doi:10.1007/s11134-015-9466-6
[56] Cao, X.R.: Realization Probabilities: The Dynamics of Queuing Systems. Springer, Heidelberg (1994). https://doi.org/10.1007/BFb0035250 · Zbl 0798.60079 · doi:10.1007/BFb0035250
[57] Cao, X.R.: Basic ideas for event-based optimization of Markov systems. Discrete Event Dyn. Syst. 15(2), 169-197 (2005) · Zbl 1130.90054 · doi:10.1007/s10626-004-6211-4
[58] Cao, X.R.: Stochastic Learning and Optimization: A Sensitivity-Based Approach. Springer, Boston (2007). https://doi.org/10.1007/978-0-387-69082-7 · Zbl 1130.93057 · doi:10.1007/978-0-387-69082-7
[59] Cao, X.R.: Optimization of average rewards of time nonhomogeneous Markov chains. IEEE Trans. Autom. Control 60(7), 1841-1856 (2015) · Zbl 1360.90177 · doi:10.1109/TAC.2015.2394951
[60] Cao, X.R.: Optimality conditions for long-run average rewards with underselectivity and nonsmooth features. IEEE Trans. Autom. Control 62(9), 4318-4332 (2017) · Zbl 1390.93861 · doi:10.1109/TAC.2017.2655487
[61] Cao, X.R.: Semismooth potentials of stochastic systems with degenerate diffusions. IEEE Trans. Autom. Control 63(10), 3566-3572 (2018) · Zbl 1423.93412 · doi:10.1109/TAC.2018.2800528
[62] Cao, X.R.: State classification and multi-class optimization of continuous-time and continuous-state Markov processes. IEEE Trans. Autom. Control, 1-14 (2019)
[63] Cao, X.R.: Stochastic control of multi-dimensional systems with relative optimization. IEEE Tran Autom. Control, 1-15 (2019)
[64] Cao, X.R.: Relative Optimization of Continuous-time and Continuous-state Stochastic Systems. A complete draft of Cao’s new book by Springer (2019)
[65] Cao, X.R., Chen, H.F.: Perturbation realization, potentials, and sensitivity analysis of Markov processes. IEEE Trans. Autom. Control 42(10), 1382-1393 (1997) · Zbl 0889.93039 · doi:10.1109/9.633827
[66] Cao, X.R., Yuan, X.M., Qiu, L.: A single sample path-based performance sensitivity formula for Markov chains. IEEE Trans. Autom. Control 41, 1814-1817 (1996) · Zbl 1041.60501 · doi:10.1109/9.545747
[67] Cassandra, A.R.: Exact and Approximate Algorithms for Partially Observable Markov Decision Processes. Doctoral Dissertation, Brown University Providence (1998)
[68] Cassandras, C.G., Lafortune, S.: Introduction to Discrete Event Systems. Springer, Boston (2008). https://doi.org/10.1007/978-0-387-68612-7 · Zbl 1165.93001 · doi:10.1007/978-0-387-68612-7
[69] Çelik, S., Maglaras, C.: Dynamic pricing and lead-time quotation for a multiclass make-to-order queue. Manag. Sci. 54(6), 1132-1146 (2008) · Zbl 1232.90140 · doi:10.1287/mnsc.1070.0836
[70] Chang, C.S., Nelson, R., Yao, D.D.: Optimal task scheduling on distributed parallel processors. Perform. Eval. 20(1-3), 207-221 (1994) · doi:10.1016/0166-5316(94)90014-0
[71] Chang, C.S.: Performance Guarantees in Communication Networks. Springer, London (2000). https://doi.org/10.1007/978-1-4471-0459-9 · Zbl 0965.90007 · doi:10.1007/978-1-4471-0459-9
[72] Chao, X., Miyazawa, M., Pinedo, M.: Queueing Networks: Customers. Signals and Product Form Solutions. Wiley, New York (1999) · Zbl 0936.90010
[73] Chen, H.: Optimal intensity control of a multi-class queue. Queueing Syst. 5(4), 281-293 (1989) · Zbl 0684.60070 · doi:10.1007/BF01225320
[74] Chen, H., Frank, M.Z.: State dependent pricing with a queue. IIE Trans. 33(10), 847-860 (2001)
[75] Chen, H., Yang, P., Yao, D.D.: Control and scheduling in a two-station queueing network: optimal policies and heuristics. Queueing Syst. 18(3-4), 301-332 (1994) · Zbl 0836.90073 · doi:10.1007/BF01158766
[76] Chen, H., Yao, D.D.: Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization. Springer, New York (2001). https://doi.org/10.1007/978-1-4757-5301-1 · Zbl 0992.60003 · doi:10.1007/978-1-4757-5301-1
[77] Chen, M.: From Markov Chains to Non-equilibrium Particle Systems. World Scientific, Singapore (2004) · Zbl 1078.60003 · doi:10.1142/5513
[78] Chen, R.R., Meyn, S.: Value iteration and optimization of multiclass queueing networks. Queueing Syst. 32(1-3), 65-97 (1999) · Zbl 0949.90020 · doi:10.1023/A:1019182903300
[79] Chou, T.C.K., Abraham, J.A.: Load balancing in distributed systems. IEEE Trans. Software Eng. 4, 401-412 (1982) · doi:10.1109/TSE.1982.235574
[80] Chung, K.L.: Markov Chains. Springer, Boston (1967). https://doi.org/10.1007/1-4020-0611-X · doi:10.1007/1-4020-0611-X
[81] Cohen, J.W.: The Single Server Queue. North-Holland Publishing Company, Amsterdan (1969) · Zbl 0183.49204
[82] Crabill, T.B.: Optimal control of a service facility with variable exponential service times and constant arrival rate. Manage. Sci. 18(9), 560-566 (1972) · Zbl 0244.60064 · doi:10.1287/mnsc.18.9.560
[83] Crabill, T.B., Gross, D., Magazine, M.J.: A survey of research on optimal design and control of queues. No. Serial T-280, Washington DC program in Logistics, George Washington University (1973)
[84] Crabill, T.B., Gross, D., Magazine, M.J.: A classified bibliography of research on optimal design and control of queues. Oper. Res. 25(2), 219-232 (1977) · Zbl 0372.60143 · doi:10.1287/opre.25.2.219
[85] Daduna, H.: Queueing Networks with Discrete Time Scale: Explicit Expressions for the Steady State Behavior of Discrete Time Stochastic Networks. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44592-7 · Zbl 0988.60094 · doi:10.1007/3-540-44592-7
[86] Dai, J.G.: On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann. Appl. Probab. 5(1), 49-77 (1995) · Zbl 0822.60083 · doi:10.1214/aoap/1177004828
[87] De Serres, Y.: Simultaneous optimization of flow control and scheduling in a single server queue with two job classes. Oper. Res. Lett. 10(2), 103-112 (1991) · Zbl 0717.90030 · doi:10.1016/0167-6377(91)90095-7
[88] De Serres, Y.: Simultaneous optimization of flow-control and scheduling in a single server queue with two job classes: numerical results and approximation. Comput. Oper. Res. 18(4), 361-378 (1991) · Zbl 0717.90031 · doi:10.1016/0305-0548(91)90097-B
[89] Deb, R.K.: Optimal control of batch service queues with switching costs. Adv. Appl. Probab. 8(1), 177-194 (1976) · Zbl 0351.60082 · doi:10.2307/1426028
[90] Deb, R.K., Serfozo, R.F.: Optimal control of batch service queues. Adv. Appl. Probab. 5(2), 340-361 (1973) · Zbl 0264.60066 · doi:10.2307/1426040
[91] Delasay, M., Kolfal, B., Ingolfsson, A.: Maximizing throughput in finite-source parallel queue systems. Eur. J. Oper. Res. 217(3), 554-559 (2012) · Zbl 1244.90239 · doi:10.1016/j.ejor.2011.09.041
[92] Demirkan, H., Spohrer, J.C., Krishna, V. (eds.): Service Systems Implementation. Springer, Boston (2011). https://doi.org/10.1007/978-1-4419-7904-9 · doi:10.1007/978-1-4419-7904-9
[93] De Waal, P.: A constrained optimization problem for a processor sharing queue. Naval Res. Logistics 40(5), 719-731 (1993) · Zbl 0772.90046 · doi:10.1002/1520-6750(199308)40:5<719::AID-NAV3220400512>3.0.CO;2-C
[94] Dimitrakopoulos, Y., Burnetas, A.: The value of service rate flexibility in an M/M/1 queue with admission control. IISE Trans. 49(6), 603-621 (2017) · doi:10.1080/24725854.2016.1269976
[95] Dimitrakos, T.D., Kyriakidis, E.G.: A semi-Markov decision algorithm for the maintenance of a production system with buffer capacity and continuous repair times. Int. J. Prod. Econ. 111(2), 752-762 (2008) · doi:10.1016/j.ijpe.2007.03.010
[96] Dinh, H.T., Lee, C., Niyato, D., Wang, P.: A survey of mobile cloud computing: architecture, applications, and approaches. Wireless Commun. Mob. Comput. 13(18), 1587-1611 (2013) · doi:10.1002/wcm.1203
[97] Disney, R.L., König, D.: Queueing networks: a survey of their random processes. SIAM Rev. 27(3), 335-403 (1985) · Zbl 0581.60075 · doi:10.1137/1027109
[98] Dobrushin, R.L., Kelbert, M.Y., Rybko, A.N., Suhov, Y.M.: Qualitative methods of queueing network theory. In: Dobrushin, R.L., Kryukov, V.M., Toom, A.L. (eds.) Stochastic Cellular Systems: Ergodicity, Memory, Morphogenesis, pp. 183-224. University Press, Manchester (1990)
[99] Doob, J.L.: Stochastic Processes. Wiley, New York (1953) · Zbl 0053.26802
[100] Doshi, B.T.: Continuous time control of the arrival process in an M/G/1 queue. Stoch. Process. Their Appl. 5(3), 265-284 (1977) · Zbl 0369.60112 · doi:10.1016/0304-4149(77)90035-7
[101] Doshi, B.T.: Optimal control of the service rate in an M/G/1 queueing system. Adv. Appl. Probab. 10(3), 682-701 (1978) · Zbl 0381.60086 · doi:10.2307/1426641
[102] Down, D.G., Koole, G., Lewis, M.E.: Dynamic control of a single-server system with abandonments. Queueing Syst. 67(1), 63-90 (2011) · Zbl 1230.90106 · doi:10.1007/s11134-010-9201-2
[103] Down, D.G., Lewis, M.E.: Dynamic load balancing in parallel queueing systems: stability and optimal control. Eur. J. Oper. Res. 168(2), 509-519 (2006) · Zbl 1101.90017 · doi:10.1016/j.ejor.2004.04.041
[104] Dshalalow, J.H.: Advances in Queueing Theory, Methods, and Open Problems. CRC Press, Boca Raton (1995) · Zbl 0836.00013
[105] Dshalalow, J.H.: Frontiers in Queueing: Models and Applications in Science and Engineering. CRC Press, Boca Raton (1997) · Zbl 0857.00015
[106] Economou, A., Kanta, S.: Optimal balking strategies and pricing for the single server Markovian queue with compartmented waiting space. Queueing Syst. 59(3-4), 237 (2008) · Zbl 1162.90393 · doi:10.1007/s11134-008-9083-8
[107] Efrosinin, D.: Controlled queueing systems with heterogeneous servers. Ph.D. dissertation, Universitätsbibliothek (University of Trier) (2004) · Zbl 1061.90035
[108] Emmons, H.: The optimal admission policy to a multiserver queue with finite horizon. J. Appl. Probab. 9(1), 103-116 (1972) · Zbl 0227.60058 · doi:10.2307/3212640
[109] Erlang, A.K.: The theory of probabilities and telephone conversations. Nyt Tidsskrift for Matematik 20(B), 33-39 (1909) · JFM 40.0281.01
[110] Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, New York (2005) · Zbl 1089.60005
[111] Ettl, M., Feigin, G.E., Lin, G.Y., Yao, D.D.: A supply network model with base-stock control and service requirements. Oper. Res. 48(2), 216-232 (2000) · doi:10.1287/opre.48.2.216.12376
[112] Farrar, T.M.: Resource allocation in systems of queues. Ph.D. dissertation, University of Cambridge (1992)
[113] Farrar, T.M.: Optimal use of an extra server in a two station tandem queueing network. IEEE Trans. Autom. Control 38(8), 1296-1299 (1993) · Zbl 0825.93693 · doi:10.1109/9.233173
[114] Farrell, W.: Optimal switching policies in a non-homogeneous exponential queueing system. Ph.D. dissertation, University of California at Los Angeles (1976)
[115] Federgruen, A., So, K.C.: Optimality of threshold policies in single-server queueing systems with server vacations. Adv. Appl. Probab. 23(2), 388-405 (1991) · Zbl 0728.60088 · doi:10.2307/1427755
[116] Federgruen, A., Zheng, Y.S.: An efficient algorithm for computing an optimal \((r, Q)\) policy in continuous review stochastic inventory systems. Oper. Res. 40(4), 808-813 (1992) · Zbl 0758.90021 · doi:10.1287/opre.40.4.808
[117] Federgruen, A., Zipkin, P.: An efficient algorithm for computing optimal \((s, S)\) policies. Oper. Res. 32(6), 1268-1285 (1984) · Zbl 0553.90031 · doi:10.1287/opre.32.6.1268
[118] Feinberg, E.A.: Optimality conditions for inventory control. In: Optimization Challenges in Complex, Networked and Risky Systems, pp. 14-45. INFORMS TutORials in Operations Research (2016) · doi:10.1287/educ.2016.0145
[119] Feinberg, E.A., Kella, O.: Optimality of D-policies for an M/G/1 queue with a removable server. Queueing Syst. 42(4), 355-376 (2002) · Zbl 1013.90043 · doi:10.1023/A:1020989221378
[120] Feinberg, E.A., Kim, D.J.: Bicriterion optimization of an M/G/1 queue with a removable server. Probab. Eng. Inf. Sci. 10(1), 57-73 (1996) · Zbl 1093.90511 · doi:10.1017/S0269964800004174
[121] Feinberg, E.A., Liang, Y.: Structure of optimal policies to periodic-review inventory models with convex costs and backorders for all values of discount factors. Ann. Oper. Res., 1-17 (2017)
[122] Feinberg, E.A., Shwartz, A. (eds.): Handbook of Markov Decision Processes: Methods and Applications. Springer, New York (2002). https://doi.org/10.1007/978-1-4615-0805-2 · Zbl 0979.90001 · doi:10.1007/978-1-4615-0805-2
[123] Feinberg, E.A., Yang, F.: Optimality of trunk reservation for an M/M/k/N queue with several customer types and holding costs. Probab. Eng. Inf. Sci. 25(4), 537-560 (2011) · Zbl 1254.90044 · doi:10.1017/S0269964811000179
[124] Feinberg, E.A., Yang, F.: Dynamic price optimization for an M/M/k/N queue with several customer types. ACM SIGMETRICS Perform. Eval. Rev. 41(3), 25-27 (2014) · doi:10.1145/2567529.2567537
[125] Feinberg, E.A., Yang, F.: Optimal pricing for a GI/M/k/N queue with several customer types and holding costs. Queueing Syst. 82(1-2), 103-120 (2016) · Zbl 1334.60194 · doi:10.1007/s11134-015-9457-7
[126] Feinberg, E.A., Zhang, X.: Optimal switching on and off the entire service capacity of a parallel queue. Probab. Eng. Inf. Sci. 29(4), 483-506 (2015) · Zbl 1370.90284 · doi:10.1017/S0269964815000157
[127] Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, New York (2012). https://doi.org/10.1007/978-1-4612-4054-9 · Zbl 0934.91002 · doi:10.1007/978-1-4612-4054-9
[128] Franken, L.J., Haverkort, B.R.: Reconfiguring distributed systems using Markov-decision models. In: Proceedings of the Workshop on Trends in Distributed Systems, pp. 219-228 (1996)
[129] Gajrat, A., Hordijk, A., Ridder, A.: Large-deviations analysis of the fluid approximation for a controllable tandem queue. Ann. Appl. Probab. 13(4), 1423-1448 (2003) · Zbl 1038.60088 · doi:10.1214/aoap/1069786504
[130] Gallisch, E.: On monotone optimal policies in a queueing model of M/G/1 type with controllable service time distribution. Ann. Appl. Probab. 11(4), 870-887 (1979) · Zbl 0434.60095 · doi:10.2307/1426864
[131] Gandhi, A.D., Cassandras, C.G.: Optimal control of polling models for transportation applications. Math. Comput. Modell. 23(11-12), 1-23 (1996) · Zbl 0857.93097 · doi:10.1016/0895-7177(96)00062-3
[132] Gans, N., Koole, G., Mandelbaum, A.: Telephone call centers: tutorial, review, and research prospects. Manuf. Serv. Oper. Management. 5(2), 79-141 (2003) · doi:10.1287/msom.5.2.79.16071
[133] Garavello, M., Piccoli, B.: Traffic Flow on Networks. Springfield: American Institute of Mathematical Sciences (2006) · Zbl 1136.90012
[134] Gast, N., Gaujal, B.: A mean field approach for optimization in discrete time. Discrete Event Dyn. Syst. 21(1), 63-101 (2011) · Zbl 1233.90275 · doi:10.1007/s10626-010-0094-3
[135] Gast, N., Gaujal, B., Le Boudec, J.Y.: Mean field for Markov decision processes: from discrete to continuous optimization. IEEE Trans. Autom. Control 57(9), 2266-2280 (2012) · Zbl 1369.90186 · doi:10.1109/TAC.2012.2186176
[136] Gaujal, B., Hordijk, A., Van Der Laan, D.: On the optimal open-loop control policy for deterministic and exponential polling systems. Probab. Eng. Inf. Sci. 21(2), 157-187 (2007) · Zbl 1128.90018 · doi:10.1017/S0269964807070118
[137] Gayon, J.P., De Vericourt, F., Karaesmen, F.: Stock rationing in an M/\(E_{\text{ r }} /1\) multi-class make-to-stock queue with backorders. IIE Trans. 41(12), 1096-1109 (2009) · doi:10.1080/07408170902800279
[138] Gelenbe, E., Pujolle, G., Gelenbe, E., Pujolle, G.: Introduction to Queueing Networks. Wiley, New York (1998) · Zbl 0654.60079
[139] George, J.M., Harrison, J.M.: Dynamic control of a queue with adjustable service rate. Oper. Res. 49(5), 720-731 (2001) · Zbl 1163.90420 · doi:10.1287/opre.49.5.720.10605
[140] Ghoneim, H.A., Stidham, S.: Control of arrivals to two queues in series. Eur. J. Oper. Res. 21(3), 399-409 (1985) · Zbl 0569.60091 · doi:10.1016/0377-2217(85)90160-2
[141] Giovanidis, A., Wunder, G., Bühler, J.: Optimal control of a single queue with retransmissions: delay-dropping tradeoffs. IEEE Trans. Wireless Commun. 8(7), 3736-3746 (2009) · doi:10.1109/TWC.2009.080959
[142] Glasserman, P., Ho, Y.C.: Gradient Estimation via Perturbation Analysis. Springer, Boston (1991) · Zbl 0746.90024
[143] Glasserman, P., Yao, D.D.: Monotone Structure in Discrete-Event Systems. Wiley, New York (1994) · Zbl 0920.93003
[144] Groenevelt, R., Koole, G., Nain, P.: On the bias vector of a two-class preemptive priority queue. Math. Methods Oper. Res. 55(1), 107-120 (2002) · Zbl 1037.90006 · doi:10.1007/s001860200175
[145] Guo, X., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes. Springer, Heidelberg (2009) · Zbl 1209.90002 · doi:10.1007/978-3-642-02547-1
[146] Ha, A.Y.: Inventory rationing in a make-to-stock production system with several demand classes and lost sales. Manage. Sci. 43(8), 1093-1103 (1997) · Zbl 0887.90053 · doi:10.1287/mnsc.43.8.1093
[147] Ha, A.Y.: Stock-rationing policy for a make-to-stock production system with two priority classes and backordering. Naval Res. Logistics 44(5), 457-472 (1997) · Zbl 0890.90082 · doi:10.1002/(SICI)1520-6750(199708)44:5<457::AID-NAV4>3.0.CO;2-3
[148] Ha, A.Y.: Stock rationing in an M/\(E_{\text{ k }} /1\) make-to-stock queue. Manage. Sci. 46(1), 77-87 (2000) · Zbl 1231.90135 · doi:10.1287/mnsc.46.1.77.15135
[149] Hajek, B.: Optimal control of two interacting service stations. IEEE Trans. Autom. Control 29(6), 491-499 (1984) · Zbl 0555.90047 · doi:10.1109/TAC.1984.1103577
[150] Hariharan, R., Moustafa, M.S., Stidham, S.: Scheduling in a multi-class series of queues with deterministic service times. Queueing Syst. 24(1-4), 83-99 (1996) · Zbl 0873.90037 · doi:10.1007/BF01149081
[151] Harrison, J.M.: Dynamic scheduling of a multiclass queue: discount optimality. Oper. Res. 23(2), 270-282 (1975) · Zbl 0321.60073 · doi:10.1287/opre.23.2.270
[152] Harrison, J.M.: Brownian Motion and Stochastic Flow Systems. Wiley, New York (1985) · Zbl 0659.60112
[153] Haviv, M., Puterman, M.L.: Bias optimality in controlled queueing systems. J. Appl. Probab. 35(1), 136-150 (1998) · Zbl 0903.90176 · doi:10.1239/jap/1032192558
[154] Helm, W.E., Waldmann, K.H.: Optimal control of arrivals to multiserver queues in a random environment. J. Appl. Probab. 21(3), 602-615 (1984) · Zbl 0545.90044 · doi:10.2307/3213621
[155] Hernádez-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996). https://doi.org/10.1007/978-1-4612-0729-0 · doi:10.1007/978-1-4612-0729-0
[156] Hernádez-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999). https://doi.org/10.1007/978-1-4612-0561-6 · Zbl 0928.93002 · doi:10.1007/978-1-4612-0561-6
[157] Ho, Y.C., Cao, X.R.: Performance sensitivity to routing changes in queuing networks and flexible manufacturing systems using perturbation analysis. IEEE J. Rob. Autom. 1(4), 165-172 (1985) · doi:10.1109/JRA.1985.1087025
[158] Ho, Y.C., Cao, X.R.: Perturbation Analysis of Discrete-Event Dynamic Systems. Kluwer Academic Publisher, Boston (1991) · Zbl 0744.90036 · doi:10.1007/978-1-4615-4024-3
[159] Honhon, D., Seshadri, S.: Admission control with incomplete information to a finite buffer queue. Probab. Eng. Inf. Sci. 21(1), 19-46 (2007) · Zbl 1115.60090 · doi:10.1017/S0269964807070027
[160] Hordijk, A., Koole, G.: On the shortest queue policy for the tandem parallel queue. Probab. Eng. Inf. Sci. 6(1), 63-79 (1992) · Zbl 1134.60397 · doi:10.1017/S0269964800002321
[161] Hordijk, A., Koole, G.: On the optimality of LEPT and \(\mu\) c rules for parallel processors and dependent arrival processes. Adv. Appl. Probab. 25(4), 979-996 (1993) · Zbl 0804.90072 · doi:10.2307/1427802
[162] Hordijk, A., Passchier, O., Spieksma, F.: Optimal service control against worst case admission policies: a multichained stochastic game. Math. Methods Oper. Res. 45(2), 281-301 (1997) · Zbl 0882.90144 · doi:10.1007/BF01193866
[163] Hordijk, A., Van Der Duyn Schouten, F.A.: Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model. Adv. Appl. Probab. 15(2), 274-303 (1983) · Zbl 0517.90091 · doi:10.2307/1426437
[164] Hordijk, A., Spieksma, F.: Constrained admission control to a queueing system. Ann. Appl. Probab. 21(2), 409-431 (1989) · Zbl 0674.60086 · doi:10.2307/1427167
[165] Howard, R.A.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960) · Zbl 0091.16001
[166] Hu, Q.Y., Yue, W.Y.: Markov Decision Processes with Their Applications. Springer, Boston (2007). https://doi.org/10.1007/978-0-387-36951-8 · Zbl 1190.90261 · doi:10.1007/978-0-387-36951-8
[167] Iravani, S.M., Posner, M.J.M., Buzacott, J.A.: A two-stage tandem queue attended by a moving server with holding and switching costs. Queueing Syst. 26(3-4), 203-228 (1997) · Zbl 0892.90077 · doi:10.1023/A:1019181124314
[168] Jackson, J.R.: Networks of waiting lines. Oper. Res. 5(4), 518-521 (1957) · Zbl 1414.90067 · doi:10.1287/opre.5.4.518
[169] Jackson, J.R.: Jobshop-like queueing systems. Manage. Sci. 10(1), 131-142 (1963) · doi:10.1287/mnsc.10.1.131
[170] Jain, A.: Priority and dynamic scheduling in a make-to-stock queue with hyperexponential demand. Naval Res. Logistics 53(5), 363-382 (2006) · Zbl 1156.90332 · doi:10.1002/nav.20146
[171] Jia, Q.S.: On solving event-based optimization with average reward over infinite stages. IEEE Trans. Autom. Control 56(12), 2912-2917 (2011) · Zbl 1368.90168 · doi:10.1109/TAC.2011.2159653
[172] Jo, K.Y., Maimon, O.Z.: Optimal dynamic load distribution in a class of flow-type flexible manufacturing systems. Eur. J. Oper. Res. 55(1), 71-81 (1991) · Zbl 0739.90030 · doi:10.1016/0377-2217(91)90192-X
[173] Jo, K.Y., Stidham, S.: Optimal service-rate control of M/G/1 queueing systems using phase methods. Ann. Appl. Probab. 15(3), 616-637 (1983) · Zbl 0512.60087 · doi:10.2307/1426622
[174] Johansen, S.G., Larsen, C.: Computation of a near-optimal service policy for a single-server queue with homogeneous jobs. Eur. J. Oper. Res. 134(3), 648-663 (2001) · Zbl 0984.90008 · doi:10.1016/S0377-2217(00)00290-3
[175] Karlin, S.: A First Course in Stochastic Processes. Academic Press, New York (1968) · Zbl 0177.21102
[176] Karlin, S., Taylor, H.E.: A Second Course in Stochastic Processes. Elsevier, New York (1981) · Zbl 0469.60001
[177] Kella, O.: Optimal control of the vacation scheme in an M/G/1 queue. Oper. Res. 38(4), 724-728 (1990) · Zbl 0719.90033 · doi:10.1287/opre.38.4.724
[178] Kelly, F.P.: Networks of queues. Ann. Appl. Probab. 8(2), 416-432 (1976) · Zbl 0337.60076 · doi:10.2307/1425912
[179] Kelly, F.P.: Reversibility and Stochastic Networks. Cambridge University Press, Cambridge (1979) · Zbl 0422.60001
[180] Kelly, F.P.: Loss networks. Ann. Appl. Probab. 1(3), 319-378 (1991) · Zbl 0743.60099 · doi:10.1214/aoap/1177005872
[181] Kemeny, J.G., Snell, J.L., Knapp, A.W.: Denumerable Markov Chains: with a Chapter of Markov Random Fields by David Griffeath. Springer, New York (1976). https://doi.org/10.1007/978-1-4684-9455-6 · doi:10.1007/978-1-4684-9455-6
[182] Kitaev, M.Y., Rykov, V.V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995) · Zbl 0876.60077
[183] Kitaev, M.Y., Serfozo, R.F.: M/M/1 queues with switching costs and hysteretic optimal control. Oper. Res. 47(2), 310-312 (1999) · Zbl 0986.60092 · doi:10.1287/opre.47.2.310
[184] Kleinrock, L.: Queueing Systems, Volume 1: Theory. Wiley Interscience, New York (1975) · Zbl 0334.60045
[185] Kleinrock, L.: Queueing Systems, Volume II: Computer Applications. Wiley Interscience, New York (1976) · Zbl 0361.60082
[186] Koçağa, Y.L., Ward, A.R.: Admission control for a multi-server queue with abandonment. Queueing Syst. 65(3), 275-323 (2010) · Zbl 1247.90103 · doi:10.1007/s11134-010-9176-z
[187] Kofman, E., Lippman, S.A.: An M/M/1 dynamic priority queue with optional promotion. Oper. Res. 29(1), 174-188 (1981) · Zbl 0453.60093 · doi:10.1287/opre.29.1.174
[188] Kolobov, A.: Planning with Markov decision processes: an AI perspective. Synthesis Lectures on Artificial Intelligence and Machine Learning 6(1), 1-210 (2012) · Zbl 1270.68014 · doi:10.2200/S00426ED1V01Y201206AIM017
[189] Koole, G.: On the pathwise optimal Bernoulli routing policy for homogeneous parallel servers. Math. Oper. Res. 21(2), 469-476 (1996) · Zbl 0855.60095 · doi:10.1287/moor.21.2.469
[190] Koole, G.: The deviation matrix of the M/M/\(1/ \infty\) and M/M/1/N queue, with applications to controlled queueing models. In: Proceedings of the 37th IEEE Conference on Decision and Control, vol. 1, pp. 56-59 (1998)
[191] Koole, G., Mandelbaum, A.: Queueing models of call centers: an introduction. Ann. Oper. Res. 113(1-4), 41-59 (2002) · Zbl 1013.90090 · doi:10.1023/A:1020949626017
[192] Koole, G.: Convexity in tandem queues. Probab. Eng. Inf. Sci. 18(1), 13-31 (2004) · Zbl 1048.60070 · doi:10.1017/S0269964804181023
[193] Koole, G.: Monotonicity in Markov reward and decision chains: theory and applications. Found. Trends Stoch. Syst. 1(1), 1-76 (2007) · Zbl 1133.90414
[194] Koole, G.: Call Center Optimization (2013). Lulu.com
[195] Koole, G., Nain, P.: On the value function of a priority queue with an application to a controlled polling model. Queueing Syst. 34(1-4), 199-214 (2000) · Zbl 0942.90016 · doi:10.1023/A:1019109103725
[196] Krishnamurthy, V.: Partially Observed Markov Decision Processes. Cambridge University Press, Cambridge (2016) · Zbl 1334.90002 · doi:10.1017/CBO9781316471104
[197] Ku, C.Y., Jordan, S.: Access control of parallel multiserver loss queues. Perform. Eval. 50(4), 219-231 (2002) · doi:10.1016/S0166-5316(02)00083-4
[198] Kumar, A.: Task allocation in multiserver systems-a survey of results. Sadhana 15(4-5), 381-395 (1990) · Zbl 1075.90513 · doi:10.1007/BF02811333
[199] Kumar, R., Lewis, M.E., Topaloglu, H.: Dynamic service rate control for a single-server queue with Markov-modulated arrivals. Naval Res. Logistics 60(8), 661-677 (2013) · Zbl 1410.90053 · doi:10.1002/nav.21560
[200] Kuo, Y.: Optimal adaptive control policy for joint machine maintenance and product quality control. Eur. J. Oper. Res. 171(2), 586-597 (2006) · Zbl 1090.90053 · doi:10.1016/j.ejor.2004.09.022
[201] Kuri, J., Kumar, A.: Optimal control of arrivals to queues with delayed queue length information. IEEE Trans. Autom. Control 40(8), 1444-1450 (1995) · Zbl 0833.90045 · doi:10.1109/9.402238
[202] Kuri, J., Kumar, A.: On the optimal control of arrivals to a single queue with arbitrary feedback delay. Queueing Syst. 27(1-2), 1-16 (1997) · Zbl 0892.90078 · doi:10.1023/A:1019145611110
[203] Kushner, H.J.: Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Springer, New York (2001). https://doi.org/10.1007/978-1-4613-0005-2 · Zbl 0988.90004 · doi:10.1007/978-1-4613-0005-2
[204] Kushner, H.J., Martins, L.F.: Heavy traffic analysis of a controlled multiclass queueing network via weak convergence methods. SIAM J. Control Optim. 34(5), 1781-1797 (1996) · Zbl 0857.90041 · doi:10.1137/S0363012994275592
[205] Kushner, H.J., Ramachandran, K.M.: Optimal and approximately optimal control policies for queues in heavy traffic. SIAM J. Control Optim. 27(6), 1293-1318 (1989) · Zbl 0691.93068 · doi:10.1137/0327066
[206] Lakshmi, C., Iyer, S.A.: Application of queueing theory in health care: a literature review. Oper. Res. Health Care 2(1-2), 25-39 (2013)
[207] Lamond, B.F.: Optimal admission policies for a finite queue with bursty arrivals. Ann. Oper. Res. 28(1), 243-260 (1991) · Zbl 0717.90028 · doi:10.1007/BF02055584
[208] Larsen, R.L., Agrawala, A.K.: Control of a heterogeneous two-server exponential queueing system. IEEE Trans. Software Eng. 4, 522-526 (1983) · Zbl 0517.90031 · doi:10.1109/TSE.1983.234960
[209] Larrañaga, M., Ayesta, U., Verloop, I.M.: Index policies for a multi-class queue with convex holding cost and abandonments. ACM SIGMETRICS Perform. Eval. Rev. 42(1), 125-137 (2014) · doi:10.1145/2637364.2591983
[210] Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM (1999) · Zbl 0922.60001
[211] Lautenbacher, C.J., Stidham, S.: The underlying Markov decision process in the single-leg airline yield-management problem. Transp. Sci. 33(2), 136-146 (1999) · Zbl 1003.90022 · doi:10.1287/trsc.33.2.136
[212] Lee, N., Kulkarni, V.G.: Optimal arrival rate and service rate control of multi-server queues. Queueing Syst. 76(1), 37-50 (2014) · Zbl 1299.90102 · doi:10.1007/s11134-012-9341-7
[213] Leeuwen, D.V., Núnez-Queija, R.: Near-optimal switching strategies for a tandem queue. In: Boucherie, R., van Dijk, N. (eds.) Markov Decision Processes in Practice, vol. 248, pp. 439-459. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-47766-4_17 · Zbl 1375.90092 · doi:10.1007/978-3-319-47766-4_17
[214] Legros, B., Jouini, O., Koole, G.: Optimal scheduling in call centers with a callback option. Perform. Eval. 95, 1-40 (2016) · doi:10.1016/j.peva.2015.09.002
[215] Legros, B., Jouini, O., Koole, G.: A uniformization approach for the dynamic control of queueing systems with abandonments. Oper. Res. 66(1), 200-209 (2018) · Zbl 1442.90050 · doi:10.1287/opre.2017.1652
[216] Lewis, M.E., Ayhan, H., Foley, R.D.: Bias optimality in a queue with admission control. Probab. Eng. Inf. Sci. 13(3), 309-327 (1999) · Zbl 0969.90030 · doi:10.1017/S0269964899133047
[217] Li, J., Kameda, H.: Load balancing problems for multiclass jobs in distributed/parallel computer systems. IEEE Trans. Comput. 47(3), 322-332 (1998) · Zbl 1392.68129 · doi:10.1109/12.660168
[218] Li, Q.L.: Constructive Computation in Stochastic Models with Applications: The RG-Factorizations. Springer, Heidelberg and Tsinghua Press (2010). https://doi.org/10.1007/978-3-642-11492-2 · Zbl 1208.60073 · doi:10.1007/978-3-642-11492-2
[219] Li, Q.L.: Nonlinear Markov processes in big networks. Spec. Matrices 4(1), 202-217 (2016) · Zbl 1343.60115
[220] Li, Q.L., Dai, G., Lui, J.C.S., Wang, Y.: The mean-field computation in a supermarket model with server multiple vacations. Discrete Event Dyn. Syst. 24(4), 473-522 (2014) · Zbl 1302.93197 · doi:10.1007/s10626-013-0171-5
[221] Li, Q.L., Du, Y., Dai, G., Wang, M.: On a doubly dynamically controlled supermarket model with impatient customers. Comput. Oper. Res. 55, 76-87 (2015) · Zbl 1348.90200 · doi:10.1016/j.cor.2014.10.004
[222] Li, Q.L., Li, Y.M., Ma, J.Y., Liu, H.L.: A complete algebraic transformational solution for the optimal dynamic policy in inventory rationing across two demand classes. Online Publication, pp. 1-62, arXiv:1908.09295 (2019)
[223] Li, Q.L., Liu, L.M.: An algorithmic approach on sensitivity analysis of perturbed QBD processes. Queueing Syst. 48(3-4), 365-397 (2004) · Zbl 1102.60072 · doi:10.1023/B:QUES.0000046582.57514.35
[224] Li, Q.L., Lui, J.C.S.: Block-structured supermarket models. Discrete Event Dyn. Syst. 26(2), 147C-182 (2016) · Zbl 1354.93019 · doi:10.1007/s10626-014-0199-1
[225] Liang, H.M., Kulkarni, V.G.: Optimal routing control in retrial queues. In: Shanthikumar, J.G., Sumita, U. (eds.) Applied Probability and Stochastic Processes, vol. 19, pp. 203-218. Springer, Boston (1999). https://doi.org/10.1007/978-1-4615-5191-1_14 · Zbl 0966.60097 · doi:10.1007/978-1-4615-5191-1_14
[226] Lin, W., Kumar, P.: Optimal control of a queueing system with two heterogeneous servers. IEEE Trans. Autom. Control 29(8), 696-703 (1984) · Zbl 0546.90035 · doi:10.1109/TAC.1984.1103637
[227] Lippman, S.A.: Applying a new device in the optimization of exponential queuing systems. Oper. Res. 23(4), 687-710 (1975) · Zbl 0312.60048 · doi:10.1287/opre.23.4.687
[228] Liu, Z., Nain, P., Towsley, D.: Sample path methods in the control of queues. Queueing Syst. 21(3-4), 293-335 (1995) · Zbl 0855.60094 · doi:10.1007/BF01149166
[229] Low, D.W.: Optimal pricing for an unbounded queue. IBM J. Res. Dev. 18(4), 290-302 (1974) · Zbl 0289.60054 · doi:10.1147/rd.184.0290
[230] Low, D.W.: Optimal dynamic pricing policies for an M/M/s queue. Oper. Res. 22(3), 545-561 (1974) · Zbl 0293.60091 · doi:10.1287/opre.22.3.545
[231] Lu, F.V., Serfozo, R.F.: M/M/1 queueing decision processes with monotone hysteretic optimal policies. Oper. Res. 32(5), 1116-1132 (1984) · Zbl 0547.60096 · doi:10.1287/opre.32.5.1116
[232] Ma, D.J., Cao, X.R.: A direct approach to decentralized control of service rates in a closed Jackson network. IEEE Trans. Autom. Control 39(7), 1460-1463 (1994) · Zbl 0800.49043 · doi:10.1109/9.299634
[233] Ma, J.Y., Xia, L., Li, Q.L.: Optimal energy-efficient policies for data centers through sensitivity-based optimization. Discrete Event Dyn. Syst., 1-40 (2019) · Zbl 1435.68056
[234] Ma, J.Y., Li, Q.L., Xia, L.: Optimal asynchronous dynamic policies in energy-efficient data centers. Online Publication, 1-63, arXiv:190103371 (2019)
[235] Maglaras, C.: Dynamic scheduling in multiclass queueing networks: stability under discrete-review policies. Queueing Syst. 31(3-4), 171-206 (1999) · Zbl 0934.90021 · doi:10.1023/A:1019106213778
[236] Mandelbaum, A., Yechiali, U.: Optimal entering rules for a customer with wait option at an M/G/1 queue. Manage. Sci. 29(2), 174-187 (1983) · Zbl 0532.90035 · doi:10.1287/mnsc.29.2.174
[237] Marin, A., Rossi, S.: Power control in saturated fork-join queueing systems. Perform. Eval. 116, 101-118 (2017) · doi:10.1016/j.peva.2017.08.008
[238] Markov, A.A.: Rasprostranenie zakona bol’shih chisel na velichiny, zavisyaschie drug ot druga. In: Izvestiya Fiziko-matematicheskogo obschestva pri Kazanskom universitete, 2-ya seriya, tom 15, pp. 135-156 (1906)
[239] Mendelson, H., Yechiali, U.: Controlling the GI/M/1 queue by conditional acceptance of customers. Eur. J. Oper. Res. 7(1), 77-85 (1981) · Zbl 0456.60085 · doi:10.1016/0377-2217(81)90051-5
[240] Menich, R., Serfozo, R.F.: Optimality of routing and servicing in dependent parallel processing systems. Queueing Syst. 9(4), 403-418 (1991) · Zbl 0748.90077 · doi:10.1007/BF01159224
[241] Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, London (1996). https://doi.org/10.1007/978-1-4471-3267-7 · Zbl 0925.60001 · doi:10.1007/978-1-4471-3267-7
[242] Miller, B.L.: Finite state continuous time Markov decision processes with applications to a class of optimization problems in queueing theory. Ph.D. dissertation, Stanford University, California, USA (1967)
[243] Mitchell, B.: Optimal service-rate selection in an M/G/1 Queue. SIAM J. Appl. Math. 24(1), 19-35 (1973) · Zbl 0233.60081 · doi:10.1137/0124004
[244] Nadar, E., Akan, M., Scheller-Wolf, A.: Technical note-optimal structural results for assemble-to-order generalized M-systems. Oper. Res. 62(3), 571-579 (2014) · Zbl 1302.90243 · doi:10.1287/opre.2014.1271
[245] Neuts, M.F.: Matrix-Geometric Solutions in Stochastic Models: An Algorithmic Approach. The Johns Hopkins University Press, Baltimore (1981) · Zbl 0469.60002
[246] Neuts, M.F.: Structured Stochastic Matrices of M/G/1Type and Their Applications. Dekker (1989) · Zbl 0695.60088
[247] Nishimura, S.: Service mechanism control and arrival control of a two-station tandem queue. J. Oper. Res. Soc. Japan 29(3), 191-205 (1986) · Zbl 0617.90031
[248] Nobel, R.D., Tijms, H.C.: Optimal control for an \(M^{\text{ X }} \)/G/1 queue with two service modes. Eur. J. Oper. Res. 113(3), 610-619 (1999) · Zbl 0947.90028 · doi:10.1016/S0377-2217(98)00085-X
[249] Nobel, R.D., Tijms, H.C.: Optimal control of a queueing system with heterogeneous servers and setup costs. IEEE Trans. Autom. Control 45(4), 780-784 (2000) · Zbl 0988.90007 · doi:10.1109/9.847122
[250] Okamura, H., Miyata, S., Dohi, T.: A Markov decision process approach to dynamic power management in a cluster system. IEEE Access 3, 3039-3047 (2015) · doi:10.1109/ACCESS.2015.2508601
[251] Pajarinen, J., Hottinen, A., Peltonen, J.: Optimizing spatial and temporal reuse in wireless networks by decentralized partially observable Markov decision processes. IEEE Trans. Mob. Comput. 13(4), 866-879 (2014) · doi:10.1109/TMC.2013.39
[252] Papadimitriou, C.H., Tsitsiklis, J.N.: The complexity of optimal queuing network control. Math. Oper. Res. 24(2), 293-305 (1999) · Zbl 0977.90008 · doi:10.1287/moor.24.2.293
[253] Pascual, R., Martínez, A., Giesen, R.: Joint optimization of fleet size and maintenance capacity in a fork-join cyclical transportation system. J. Oper. Res. Soc. 64(7), 982-994 (2013) · doi:10.1057/jors.2012.103
[254] Patrick, J., Begen, M.A.: Markov decision processes and its applications in healthcare. In: Handbook of Healthcare Delivery Systems. CRC (2011)
[255] Perez, A.P., Zipkin, P.: Dynamic scheduling rules for a multiproduct make-to-stock queue. Oper. Res. 45(6), 919-930 (1997) · Zbl 0895.90078 · doi:10.1287/opre.45.6.919
[256] Piunovskiy, A.B.: Bicriteria optimization of a queue with a controlled input stream. Queueing Syst. 48(1-2), 159-184 (2004) · Zbl 1067.90023 · doi:10.1023/B:QUES.0000039892.03630.24
[257] Plambeck, E., Kumar, S., Harrison, J.M.: A multiclass queue in heavy traffic with throughput time constraints: asymptotically optimal dynamic controls. Queueing Syst. 39(1), 23-54 (2001) · Zbl 1002.90015 · doi:10.1023/A:1017983532376
[258] Plum, H.J.: Optimal monotone hysteretic Markov policies in an M/M/1 queueing model with switching costs and finite time horizon. Zeitschrift für Oper. Res. 35(5), 377-399 (1991) · Zbl 0731.60090
[259] Powell, W.B., Humblet, P.: The bulk service queue with a general control strategy: theoretical analysis and a new computational procedure. Oper. Res. 34(2), 267-275 (1986) · Zbl 0612.60090 · doi:10.1287/opre.34.2.267
[260] Printezis, A., Burnetas, A.: Priority option pricing in an M/M/m queue. Oper. Res. Lett. 36(6), 700-704 (2008) · Zbl 1151.90365 · doi:10.1016/j.orl.2008.07.001
[261] Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994) · Zbl 0829.90134 · doi:10.1002/9780470316887
[262] Qiu, Q., Pedram, M.: Dynamic power management based on continuous-time Markov decision processes. In: Proceedings of the 36th annual ACM/IEEE Design Automation Conference, pp. 555-561 (1999)
[263] Robinson, D.R.: Markov decision chains with unbounded costs and applications to the control of queues. Adv. Appl. Probab. 8(1), 159-176 (1976) · Zbl 0343.60044 · doi:10.2307/1426027
[264] Robinson, D.R.: Optimization of priority queues-a semi-Markov decision chain approach. Manage. Sci. 24(5), 545-553 (1978) · Zbl 0374.60122 · doi:10.1287/mnsc.24.5.545
[265] Rosberg, Z., Kermani, P.: Customer routing to different servers with complete information. Adv. Appl. Probab. 21(4), 861-882 (1989) · Zbl 0691.60083 · doi:10.2307/1427771
[266] Rosberg, Z., Varaiya, P., Walrand, J.: Optimal control of service in tandem queues. IEEE Trans. Autom. Control 27(3), 600-610 (1982) · Zbl 0497.90024 · doi:10.1109/TAC.1982.1102957
[267] Ross, K.W.: Optimal dynamic routing in Markov queueing networks. Automatica 22(3), 367-370 (1986) · Zbl 0585.90034 · doi:10.1016/0005-1098(86)90035-X
[268] Rue, R.C., Rosenshine, M.: Optimal control for entry of many classes of customers to an M/M/1 queue. Naval Res. Logistics 28(3), 489-495 (1981) · Zbl 0468.60096 · doi:10.1002/nav.3800280312
[269] Rue, R.C., Rosenshine, M.: Some properties of optimal control policies for entry to an M/M/1 queue. Naval Res. Logistics 28(4), 525-532 (1981) · Zbl 0534.60086 · doi:10.1002/nav.3800280402
[270] Rue, R.C., Rosenshine, M.: Optimal control of entry to an M/\(E_k/1\) queue serving several classes of customers. Naval Res. Logistics 30(2), 217-226 (1983) · Zbl 0529.90046 · doi:10.1002/nav.3800300204
[271] Rue, R.C., Rosenshine, M.: The application of semi-Markov decision processes to queueing of aircraft for landing at an airport. Transp. Sci. 19(2), 154-172 (1985) · doi:10.1287/trsc.19.2.154
[272] Rykov, V.V.: Controllable queueing systems. Itogi Nauki i Tekhniki. Seriya “Teoriya Veroyatnostei. Matematicheskaya Statistika. Teoreticheskaya Kibernetika” 12, 45-152 (1975) (There is English translation in Journal of Soviet Mathematics). (In Russian)
[273] Rykov, V.V.: Monotone control of queueing systems with heterogeneous servers. Queueing Syst. 37(4), 391-403 (2001) · Zbl 1017.90026 · doi:10.1023/A:1010893501581
[274] Rykov, V.V.: Controllable queueing systems: from the very beginning up to nowadays. Reliab. Theor. Appl. 12(2(45)), 39-61 (2017)
[275] Rykov, V.V., Efrosinin, D.: Optimal control of queueing systems with heterogeneous servers. Queueing Syst. 46(3-4), 389-407 (2004) · Zbl 1061.90035 · doi:10.1023/B:QUES.0000027992.91461.1e
[276] Rykov, V.V., Efrosinin, D.: On optimal control of systems on their life time. In: Lisnianski, A., Frenkel, I. (eds.) Recent Advances in System Reliability. Springer Series in Reliability Engineering, pp. 307-319. Springer, London (2012). https://doi.org/10.1007/978-1-4471-2207-4_22 · doi:10.1007/978-1-4471-2207-4_22
[277] Rykov, V.V., Lembert, E.: Optimal dynamic priorities in single-line queueing systems. Eng. Cybern. 5(1), 21-30 (1967)
[278] Sanajian, N., Abouee-Mehrizi, H., Balcıoglu, B.: Scheduling policies in the M/G/1 make-to-stock queue. J. Oper. Res. Soc. 61(1), 115-123 (2010) · Zbl 1193.90112 · doi:10.1057/jors.2008.139
[279] Savaşaneril, S., Griffin, P.M., Keskinocak, P.: Dynamic lead-time quotation for an M/M/1 base-stock inventory queue. Oper. Res. 58(2), 383-395 (2010) · Zbl 1226.90027 · doi:10.1287/opre.1090.0717
[280] Schassberger, R.: A note on optimal service selection in a single server queue. Manage. Sci. 21(11), 1326-1331 (1975) · Zbl 0325.60088 · doi:10.1287/mnsc.21.11.1326
[281] Sennott, L.I.: Average cost semi-Markov decision processes and the control of queueing systems. Probab. Eng. Inf. Sci. 3(2), 247-272 (1989) · Zbl 1134.60408 · doi:10.1017/S0269964800001121
[282] Sennott, L.I.: Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York (2009) · Zbl 0997.93503
[283] Serfozo, R.F.: Optimal control of random walks, birth and death processes, and queues. Adv. Appl. Probab. 13(1), 61-83 (1981) · Zbl 0446.60062 · doi:10.2307/1426467
[284] Serfozo, R.F.: Introduction to Stochastic Networks. Springer, New York (1999). https://doi.org/10.1007/978-1-4612-1482-3 · Zbl 0983.60006 · doi:10.1007/978-1-4612-1482-3
[285] Shanthikumar, J.G., Yao, D.D.: Stochastic monotonicity in general queueing networks. J. Appl. Probab. 26(2), 413-417 (1989) · Zbl 0686.60099 · doi:10.2307/3214048
[286] Shioyama, T.: Optimal control of a queuing network system with two types of customers. Eur. J. Oper. Res. 52(3), 367-372 (1991) · Zbl 0725.60102 · doi:10.1016/0377-2217(91)90171-Q
[287] Sigaud, O., Buffet, O. (eds.): Markov Decision Processes in Artificial Intelligence. Wiley (2013)
[288] e Silva, E.D.S., Gerla, M.: Queueing network models for load balancing in distributed systems. J. Parallel Distrib. Comput. 12(1), 24-38 (1991) · Zbl 0753.68016
[289] Sobel, M.J.: Optimal operation of queues. In: Clarke, A.B. (ed.) Mathematical Methods in Queueing Theory, vol. 98, pp. 231-261. Springer, Heidelberg (1974). https://doi.org/10.1007/978-3-642-80838-8_12 · Zbl 0368.60109 · doi:10.1007/978-3-642-80838-8_12
[290] Solodyannikov, Y.V.: Control and observation for dynamical queueing networks I. Autom. Remote Control 75(3), 422-446 (2014) · Zbl 1301.93175 · doi:10.1134/S0005117914030023
[291] Sparaggis, P.D., Towsley, D., Cassandras, C.G.: Optimal control of multiclass parallel service systems. Discrete Event Dyn. Syst. 6(2), 139-158 (1996) · Zbl 0849.90065 · doi:10.1007/BF01797236
[292] Stidham, S.: On the optimality of single-server queuing systems. Oper. Res. 18(4), 708-732 (1970) · Zbl 0247.60060 · doi:10.1287/opre.18.4.708
[293] Stidham, S.: Socially and individually optimal control of arrivals to a GI/M/1 queue. Manage. Sci. 24(15), 1598-1610 (1978) · Zbl 0493.90040 · doi:10.1287/mnsc.24.15.1598
[294] Stidham, S.: Optimal control of admission to a queueing system. IEEE Trans. Autom. Control 30(8), 705-713 (1985) · Zbl 0563.90044 · doi:10.1109/TAC.1985.1104054
[295] Stidham, S.: Scheduling, routing, and flow control in stochastic networks. In: Fleming, W., Lions, P.L. (eds.) Stochastic Differential Systems, Stochastic Control Theory and Applications, vol. 10, pp. 529-561. Springer, New York (1988). https://doi.org/10.1007/978-1-4613-8762-6_31 · Zbl 0638.00029 · doi:10.1007/978-1-4613-8762-6_31
[296] Stidham, S.: Analysis, design, and control of queueing systems. Oper. Res. 50(1), 197-216 (2002) · Zbl 1163.90308 · doi:10.1287/opre.50.1.197.17783
[297] Stidham, S.: On the optimality of a full-service policy for a queueing system with discounted costs. Math. Methods Oper. Res. 62(3), 485-497 (2005) · Zbl 1084.60059 · doi:10.1007/s00186-005-0040-6
[298] Stidham, S.: Optimal Design of Queueing Systems. Chapman and Hall/CRC, Boca Raton (2009) · Zbl 1202.60146 · doi:10.1201/9781420010008
[299] Stidham, S., Prabhu, N.U.: Optimal control of queueing systems. In: Clarke, A.B. (ed.) Mathematical Methods in Queueing Theory, vol. 98, pp. 263-294. Springer, Heidelberg (1974). https://doi.org/10.1007/978-3-642-80838-8_13 · Zbl 0285.60087 · doi:10.1007/978-3-642-80838-8_13
[300] Stidham, S., Weber, R.: A survey of Markov decision models for control of networks of queues. Queueing Syst. 13(1-3), 291-314 (1993) · Zbl 0772.90082 · doi:10.1007/BF01158935
[301] Sun, L., Dong, H., Hussain, F.K., Hussain, O.K., Chang, E.: Cloud service selection: state-of-the-art and future research directions. J. Netw. Comput. Appl. 45, 134-150 (2014) · doi:10.1016/j.jnca.2014.07.019
[302] Syski, R.: A personal view of queueing theory. In: Frontiers in Queueing: Models and Applications in Science and Engineering, pp. 3-18. CRC Press (1997) · Zbl 0871.60073
[303] Tassiulas, L., Ephremides, A.: Throughput properties of a queueing network with distributed dynamic routing and flow control. Adv. Appl. Probab. 28(1), 285-307 (1996) · Zbl 0846.90038 · doi:10.2307/1427922
[304] Tijms, H.C.: Stochastic Models: An Algorithmic Approach. Wiley (1994) · Zbl 0838.60075
[305] Tijms, H.C., van der Duyn Schouten, F.A.: A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information. Eur. J. Oper. Res. 21(2), 245-253 (1985) · Zbl 0569.90036 · doi:10.1016/0377-2217(85)90036-0
[306] Tirdad, A., Grassmann, W.K., Tavakoli, J.: Optimal policies of \(M(t)/M/c/c\) queues with two different levels of servers. Eur. J. Oper. Res. 249(3), 1124-1130 (2016) · Zbl 1346.90263 · doi:10.1016/j.ejor.2015.10.040
[307] Towsley, D., Sparaggis, P.D., Cassandras, C.G.: Optimal routing and buffer allocation for a class of finite capacity queueing systems. IEEE Trans. Autom. Control 37(9), 1446-1451 (1992) · Zbl 0757.60099 · doi:10.1109/9.159590
[308] Vanberkel, P.T., Boucherie, R.J., Hans, E.W., Hurink, J.L., Litvak, N.: A survey of health care models that encompass multiple departments. Int. J. Health Manag. Inf. 1(1), 37-69 (2010)
[309] Van der Weij, W., Bhulai, S., Van der Mei, R.: Optimal scheduling policies for the limited processor sharing queue. Technical report WS2008-5, Department of Mathematics, Vrije University (2008)
[310] Van Dijk, N.M.: Queueing Networks and Product Forms: A Systems Approach. Wiley (1993)
[311] Van Dijk, N.M., Puterman, M.L.: Perturbation theory for Markov reward processes with applications to queueing systems. Adv. Appl. Probab. 20(1), 79-98 (1988) · Zbl 0642.60100 · doi:10.2307/1427271
[312] Van Nunen, J.A.E.E., Puterman, M.L.: Computing optimal control limits for GI/M/s queuing systems with controlled arrivals. Manage. Sci. 29(6), 725-734 (1983) · Zbl 0521.90049 · doi:10.1287/mnsc.29.6.725
[313] Varma, S.: Optimal allocation of customers in a two server queue with resequencing. IEEE Trans. Autom. Control 36(11), 1288-1293 (1991) · doi:10.1109/9.100940
[314] Veatch, M.H.: Queueing control problems for production/inventory systems. Ph.D. dissertation, Massachusetts Institute of Technology (1992)
[315] Veatch, M.H., Wein, L.M.: Monotone control of queueing networks. Queueing Syst. 12(3-4), 391-408 (1992) · Zbl 0825.90414 · doi:10.1007/BF01158810
[316] Vercraene, S., Gayon, J.P., Karaesmen, F.: Effects of system parameters on the optimal cost and policy in a class of multidimensional queueing control problems. Oper. Res. 66(1), 150-162 (2018) · Zbl 1456.90059 · doi:10.1287/opre.2017.1600
[317] Wakuta, K.: Optimal control of an M/G/1 queue with imperfectly observed queue length when the input source is finite. J. Appl. Probab. 28(1), 210-220 (1991) · Zbl 0744.60116 · doi:10.2307/3214751
[318] Wang, D.X., Cao, X.R.: Event-based optimization for POMDPs and its application in portfolio management. In: Proceedings of the 18th IFAC World Congress, vol. 44, no. 1, pp. 3228-3233 (2011) · doi:10.3182/20110828-6-IT-1002.01514
[319] Weber, R.R.: On the optimal assignment of customers to parallel servers. J. Appl. Probab. 15(2), 406-413 (1978) · Zbl 0378.60095 · doi:10.2307/3213411
[320] Weber, R.R., Stidham, S.: Optimal control of service rates in networks of queues. Adv. Appl. Probab. 19(1), 202-218 (1987) · Zbl 0617.60090 · doi:10.2307/1427380
[321] White, L.B.: A new policy evaluation algorithm for Markov decision processes with quasi birth-death structure. Stoch. Models 21(2-3), 785-797 (2005) · Zbl 1090.90198 · doi:10.1081/STM-200056019
[322] Winkler, A.: Dynamic scheduling of a single-server two-class queue with constant retrial policy. Ann. Oper. Res. 202(1), 197-210 (2013) · Zbl 1263.90027 · doi:10.1007/s10479-011-0950-z
[323] Wu, C.H., Lin, J.T., Chien, W.C.: Dynamic production control in a serial line with process queue time constraint. Int. J. Prod. Res. 48(13), 3823-3843 (2010) · Zbl 1197.90309 · doi:10.1080/00207540902922836
[324] Xia, L.: Service rate control of closed Jackson networks from game theoretic perspective. Eur. J. Oper. Res. 237(2), 546-554 (2014) · Zbl 1304.90071 · doi:10.1016/j.ejor.2014.01.038
[325] Xia, L.: Event-based optimization of admission control in open queueing networks. Discrete Event Dyn. Syst. 24(2), 133-151 (2014) · Zbl 1296.93115 · doi:10.1007/s10626-013-0167-1
[326] Xia, L., Cao, X.R.: Performance optimization of queueing systems with perturbation realization. Eur. J. Oper. Res. 218(2), 293-304 (2012) · Zbl 1244.90070 · doi:10.1016/j.ejor.2011.07.039
[327] Xia, L., Chen, S.: Dynamic pricing control for open queueing networks. IEEE Trans. Autom. Control 63(10), 3290-3300 (2018) · Zbl 1423.90063 · doi:10.1109/TAC.2018.2799576
[328] Xia, L., He, Q.M., Alfa, A.S.: Optimal control of state-dependent service rates in a MAP/M/1 queue. IEEE Trans. Autom. Control 62(10), 4965-4979 (2017) · Zbl 1390.90266 · doi:10.1109/TAC.2017.2679139
[329] Xia, L., Jia, Q.S.: Parameterized Markov decision process and its application to service rate control. Automatica 54, 29-35 (2015) · Zbl 1318.93062 · doi:10.1016/j.automatica.2015.01.006
[330] Xia, L., Jia, Q.S., Cao, X.R.: A tutorial on event-based optimization-a new optimization framework. Discrete Event Dyn. Syst. 24(2), 103-132 (2014) · Zbl 1296.93116 · doi:10.1007/s10626-013-0170-6
[331] Xia, L., Shihada, B.: Max-Min optimality of service rate control in closed queueing networks. IEEE Trans. Autom. Control 58(4), 1051-1056 (2013) · Zbl 1369.90054 · doi:10.1109/TAC.2012.2218145
[332] Xu, S.H., Righter, R., Shanthikumar, J.G.: Optimal dynamic assignment of customers to heterogeneous servers in parallel. Oper. Res. 40(6), 1126-1138 (1992) · Zbl 0770.90028 · doi:10.1287/opre.40.6.1126
[333] Xu, S.H., Zhao, Y.Q.: Dynamic routing and jockeying controls in a two-station queueing system. Adv. Appl. Probab. 28(4), 1201-1226 (1996) · Zbl 0867.90057 · doi:10.2307/1428170
[334] Xu, J., Hajek, B.: The supermarket game. Stoch. Syst. 3(2), 405-441 (2013) · Zbl 1311.60109 · doi:10.1287/12-SSY093
[335] Yang, R., Bhulai, S., van der Mei, R.: Structural properties of the optimal resource allocation policy for single-queue systems. Ann. Oper. Res. 202(1), 211-233 (2013) · Zbl 1263.90028 · doi:10.1007/s10479-011-0933-0
[336] Yao, D.D., Schechner, Z.: Decentralized control of service rates in a closed Jackson network. IEEE Trans. Autom. Control 34(2), 236-240 (1989) · Zbl 0666.93125 · doi:10.1109/9.21110
[337] Yechiali, U.: Customers’ optimal joining rules for the GI/M/s queue. Manage. Sci. 18(7), 434-443 (1972) · Zbl 0239.60093 · doi:10.1287/mnsc.18.7.434
[338] Yeh, L., Thomas, L.C.: Adaptive control of M/M/1 queues-continuous-time Markov decision process approach. J. Appl. Probab. 20(2), 368-379 (1983) · Zbl 0521.90050 · doi:10.2307/3213809
[339] Yildirim, U., Hasenbein, J.J.: Admission control and pricing in a queue with batch arrivals. Oper. Res. Lett. 38(5), 427-431 (2010) · Zbl 1202.90091 · doi:10.1016/j.orl.2010.07.003
[340] Yoon, S., Lewis, M.E.: Optimal pricing and admission control in a queueing system with periodically varying parameters. Queueing Syst. 47(3), 177-199 (2004) · Zbl 1048.60077 · doi:10.1023/B:QUES.0000035313.20223.3f
[341] Zayas-Cabán, G., Xie, J., Green, L.V., Lewis, M.E.: Dynamic control of a tandem system with abandonments. Queueing Syst. 84(3-4), 279-293 (2016) · Zbl 1386.90031 · doi:10.1007/s11134-016-9489-7
[342] Zeng, Y., Chaintreau, A., Towsley, D., Xia, C.H.: A necessary and sufficient condition for throughput scalability of fork and join networks with blocking. ACM SIGMETRICS Perform. Eval. Rev. 44(1), 25-36 (2016) · doi:10.1145/2964791.2901470
[343] Zeng, Y., Chaintreau, A., Towsley, D., Xia, C.H.: Throughput scalability analysis of fork-join queueing networks. Oper. Res. 66(6), 1728-1743 (2018) · Zbl 1446.90057 · doi:10.1287/opre.2018.1748
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.