×

zbMATH — the first resource for mathematics

Input-output conformance testing for software product lines. (English) Zbl 1392.68144
Summary: We extend the theory of input-output conformance (IOCO) testing to accommodate behavioral models of software product lines (SPLs). We present the notions of residual and spinal testing. These notions allow for structuring the test process for SPLs by taking variability into account and extracting separate test suites for common and specific features of an SPL. The introduced notions of residual and spinal test suites allow for focusing on the newly introduced behavior and avoiding unnecessary re-test of the old one. Residual test suites are very conservative in that they require retesting the old behavior that can reach to new behavior. However, spinal test suites more aggressively prune the old tests and only focus on those test sequences that are necessary in reaching the new behavior. We show that residual testing is complete but does not usually lead to much reduction in the test-suite. In contrast, spinal testing is not necessarily complete but does reduce the test-suite. We give sufficient conditions on the implementation to guarantee completeness of spinal testing. Finally, we specify and analyze an example regarding the Ceiling Speed Monitoring Function from the European Train Control System.

MSC:
68N30 Mathematical aspects of software engineering (specification, verification, metrics, requirements, etc.)
Software:
mCRL2
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Beohar, H.; Mousavi, M. R., Input-output conformance testing based on featured transition systems, (Proc. of 29th ACM Symposium of Applied Computing: Software Verification and Testing Track, ACM SAC SVT ’14, (2014), ACM), 1272-1278
[2] Beohar, H.; Mousavi, M. R., Spinal test suites for software product lines, (Schlingloff, H.; Petrenko, A. K., Proc. of 9th Workshop on Model Based Testing, MBT’2014, Electron. Proc. Theor. Comput. Sci., vol. 141, (2014), Open Publishing Association), 44-55
[3] da Mota Silveira Neto, P. A.; do Carmo Machado, I.; McGregor, J. D.; de Almeida, E. S.; de Lemos Meira, S. R., A systematic mapping study of software product lines testing, Inf. Softw. Technol., 53, 5, 407-423, (2011)
[4] Engström, E.; Runeson, P., Software product line testing - a systematic mapping study, Inf. Softw. Technol., 53, 1, 2-13, (2011)
[5] Oster, S.; Wübbeke, A.; Engels, G.; Schürr, A., Model-based software product lines testing survey, (Zander, J.; Schieferdecker, I.; Mosterman, P., Model-Based Testing for Embedded Systems, (2011), CRC Press), 339-381
[6] Lamancha, B. P.; Usaola, M. P.; Velthius, M. P., Systematic review on software product line testing, (Cordeiro, J.; Virvou, M.; Shishkov, B., Software and Data Technologies, Commun. Comput. Inf. Sci., vol. 170, (2013), Springer), 58-71
[7] Thüm, T.; Apel, S.; Kästner, C.; Kuhlemann, M.; Schaefer, I.; Saake, G., Analysis strategies for software product lines, (2012), School of Computer Science, University of Magdeburg, Technical report FIN-004-2012
[8] Tretmans, J., Model based testing with labelled transition systems, (Hierons, R. M.; Bowen, J. P.; Harman, M., Formal Methods and Testing, LNCS, vol. 4949, (2008), Springer), 1-38
[9] Yannakakis, M.; Lee, D., Testing of finite state systems, (Gottlob, G.; Grandjean, E.; Seyr, K., Proc. of 12th International Workshop on Computer Science Logic, CSL ’98, Lect. Notes Comput. Sci., vol. 1584, (1999), Springer), 29-44
[10] (Broy, M.; Jonsson, B.; Katoen, J.-P.; Leucker, M.; Pretschner, A., Model-Based Testing of Reactive Systems, Lect. Notes Comput. Sci., vol. 3472, (2005), Springer) · Zbl 1070.68088
[11] Lochau, M.; Schaefer, I.; Kamischke, J.; Lity, S., Incremental model-based testing of delta-oriented software product lines, (Brucker, A. D.; Julliand, J., Proc. of 6th International Conference on Tests and Proofs, Lect. Notes Comput. Sci., vol. 7305, (2012), Springer), 67-82 · Zbl 1354.68059
[12] Varshosaz, M.; Beohar, H.; Mousavi, M. R., Delta-oriented FSM-based testing, (Butler, M.; Conchon, S.; Zaïdi, F., Proc. of 17th International Conference on Formal Engineering Methods, ICFEM 2015, Lect. Notes Comput. Sci., vol. 9407, (2015), Springer), 366-381
[13] Thüm, T.; Apel, S.; Kästner, C.; Schaefer, I.; Saake, G., A classification and survey of analysis strategies for software product lines, ACM Comput. Surv., 47, 1, (2014), Article 6
[14] Schaefer, I.; Rabiser, R.; Clarke, D.; Bettini, L.; Benavides, D.; Botterweck, G.; Pathak, A.; Trujillo, S.; Villela, K., Software diversity: state of the art and perspectives, Softw. Tools Technol. Transf., 14, 5, 477-495, (2012)
[15] Classen, A., Modelling with FTS: a collection of illustrative examples, (2010), University of Namur, Tech. Rep. P-CS-TR SPLMC-00000001
[16] Schmid, K.; Rabiser, R.; Grünbacher, P., A comparison of decision modeling approaches in product lines, (Proc. of 5th International Workshop on Variability Modeling of Software-Intensive Systems, VAMOS ’11, (2011), ACM), 119-126
[17] Czarnecki, K.; Grünbacher, P.; Rabiser, R.; Schmid, K.; Wasowski, A., Cool features and tough decisions: a comparison of variability modeling approaches, (Proc. of 6th International Workshop on Variability Modelling of Software-Intensive Systems, VAMOS ’12, (2012), ACM), 173-182
[18] Sinnema, M.; Deelstra, S., Classifying variability modeling techniques, Inf. Softw. Technol., 49, 7, 717-739, (2007)
[19] Asirelli, P.; ter Beek, M. H.; Gnesi, S.; Fantechi, A., Formal description of variability in product families, (Proc. of 15th International Software Product Line Conference, SPLC ’11, (2011), IEEE), 130-139
[20] Asirelli, P.; ter Beek, M. H.; Fantechi, A.; Gnesi, S., A model-checking tool for families of services, (Bruni, R.; Dingel, J., Proc. of the Joint 13th IFIP WG 6.1 and 30th IFIP WG 6.1 International Conference on Formal Techniques for Distributed Systems, FMOODS’11/FORTE’11, Lect. Notes Comput. Sci., vol. 6722, (2011), Springer), 44-58
[21] Classen, A.; Cordy, M.; Schobbens, P. Y.; Heymans, P.; Legay, A.; Raskin, J. F., Featured transition systems: foundations for verifying variability-intensive systems and their application to LTL model checking, IEEE Trans. Softw. Eng., 39, 8, 1069-1089, (2013)
[22] Classen, A.; Heymans, P.; Schobbens, P.-Y.; Legay, A.; Raskin, J.-F., Model checking lots of systems: efficient verification of temporal properties in software product lines, (Proc. of 32nd ACM/IEEE International Conference on Software Engineering, vol. 1, ICSE ’10, (2010), ACM), 335-344
[23] Fischbein, D.; Uchitel, S.; Braberman, V., A foundation for behavioural conformance in software product line architectures, (Proc. on Role of Software Architecture for Testing and Analysis, (2006), ACM), 39-48
[24] Gruler, A.; Leucker, M.; Scheidemann, K., Modeling and model checking software product lines, (Barthe, G.; de Boer, F. S., Proc. of 10th IFIP WG 6.1 International Conference on Formal Methods for Open Object-Based Distributed Systems, FMOODS ’08, Lect. Notes Comput. Sci., vol. 5051, (2008), Springer), 113-131
[25] Larsen, K. G.; Nyman, U.; Wąsowski, A., Modal I/O automata for interface and product line theories, (De Nicola, R., Proc. of 16th European Symposium on Programming, ESOP ’07, Lect. Notes Comput. Sci., vol. 4421, (2007), Springer), 64-79 · Zbl 1187.68296
[26] Beohar, H.; Varshosaz, M.; Mousavi, M. R., Basic behavioral models for software product lines: expressiveness and testing pre-orders, Sci. Comput. Program., 123, 42-60, (2016)
[27] Tribastone, M., Behavioral relations in a process algebra for variants, (Proc. of 18th International Software Product Line Conference, SPLC’14, (2014), ACM Press), 82-91
[28] Muschevici, R.; Proença, J.; Clarke, D., Modular modelling of software product lines with feature nets, (Barthe, G.; Pardo, A.; Schneider, G., Proc. of 9th Int. Conf. on Softw. Eng. and Formal Methods, SEFM ’11, Lect. Notes Comput. Sci., vol. 7041, (2011), Springer), 318-333 · Zbl 1350.68063
[29] Larsen, K.; Thomsen, B., A modal process logic, (Proc. of 3rd IEEE Annual Symposium on Logic in Computer Science, LICS ’88, (1988), IEEE), 203-210
[30] Weissleder, S.; Schlingloff, H., Automatic model-based test generation from uml state machines, (Zander, J.; Schieferdecker, I.; Mosterman, P. J., Model-Based Testing for Embedded Systems, (2011), CRC Press), 339-381
[31] Gomaa, H., Designing software product lines with UML, (2005), Addison-Wesley
[32] Atkinson, C.; Bayer, J.; Bunse, C.; Kamsties, E.; Laitenberger, O.; Laqua, R.; Muthig, D.; Peach, B.; Wüst, J.; Zettel, J., Component-based product line engineering with UML, (2001), Addison-Wesley
[33] Clarke, D.; Helvensteijn, M.; Schaefer, I., Abstract delta modeling, (Visser, E.; Järvi, J., Proc. of 9th International Conference on Generative Programming and Component Engineering, GPCE ’10, (2010), ACM), 13-22
[34] Lochau, M.; Lity, S.; Lachmann, R.; Schaefer, I.; Goltz, U., Delta-oriented model-based integration testing of large-scale systems, J. Syst. Softw., 91, 63-84, (2014)
[35] Lity, S.; Morbach, T.; Thüm, T.; Schaefer, I., Applying incremental model slicing to product-line regression testing, (Kapitsaki, G. M.; Santana de Almeida, E., Proc. of 15th International Conference on Software Reuse: Bridging with Social-Awareness, ICSR ’16, Lect. Notes Comput. Sci., vol. 9679, (2016), Springer), 3-19
[36] EI-Fakih, K.; Yevtushenko, N.; van Bochmann, G., FSM-based incremental conformance testing methods, IEEE Trans. Softw. Eng., 30, 7, 425-436, (2004)
[37] Pap, Z.; Subramaniam, M.; Kovács, G.; Németh, G.Á., A bounded incremental test generation algorithm for finite state machines, (Petrenko, A.; Veanes, M.; Tretmans, J.; Grieskamp, W., Proc. of 19th IFIP TC6/WG6.1 International Conference on Testing of Software and Communicating Systems and 7th International Workshop on Formal Approaches to Testing Software, TestCom/FATES ’07, Lect. Notes Comput. Sci., vol. 4581, (2007), Springer), 244-259
[38] da Silva Simão, A.; Petrenko, A., Fault coverage-driven incremental test generation, Comput. J., 53, 9, 1508-1522, (2010)
[39] Braunstein, C.; Peleska, J.; Schulze, U.; Hübner, F.; Huang, W.-L.; Haxthausen, A. E.; Vu Hong, L., A sysml test model and test suite for the ETCS ceiling speed monitor, (2014), University of Bremen, Work Package 4 OETCS/WP4/CSM-01/00
[40] Braunstein, C.; Haxthausen, A. E.; Huang, W.-L.; Hübner, F.; Peleska, J.; Schulze, U.; Vu Hong, L., Complete model-based equivalence class testing for the etcs ceiling speed monitor, (Merz, S.; Pang, J., Proc. of 12th Int. Conf. on Formal Methods and Software Engineering, SEFM’14, Lect. Notes Comput. Sci., vol. 8829, (2014), Springer), 380-395
[41] Kang, K.; Cohen, S.; Hess, J.; Novak, W.; Peterson, S., Feature-oriented domain analysis (FODA) feasibility study, (1990), Software Engineering Institute, Carnegie Mellon University, Tech. Rep. CMU/SEI-90-TR-21
[42] Schobbens, P.-Y.; Heymans, P.; Trigaux, J.-C., Feature diagrams: a survey and a formal semantics, (Proc. of the 14th IEEE International Conference on Requirements Engineering, RA ’06, (2006), IEEE), 136-145
[43] de Langen, M. A., Vehicle function correctness, (2013), Eindhoven University of Technology, Masters thesis
[44] Lochau, M.; Kamischke, J., Parameterized preorder relations for model-based testing of software product lines, (Margaria, T.; Steffen, B., Proc. of 5th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change, Part I, ISoLA ’12, Lect. Notes Comput. Sci., vol. 7609, (2012), Springer), 223-237
[45] Plotkin, G. D., A structural approach to operational semantics, (1981), University of Aarhus, Tech. Rep. DAIMI FN-19
[46] Groote, J. F.; Mousavi, M. R., Modeling and analysis of communicating systems, (2014), MIT Press
[47] Cranen, S.; Groote, J. F.; Keiren, J. J.A.; Stappers, F. P.M.; de Vink, E. P.; Wesselink, W.; Willemse, T. A.C., An overview of the mcrl2 toolset and its recent advances, (Piterman, N.; Smolka, S. A., Proc. of 19th Int. Conf. on Tools and Algorithms for the Construction and Analysis of Systems, TACAS ’13, Lect. Notes Comput. Sci., vol. 7795, (2013), Springer), 199-213 · Zbl 1381.68198
[48] Huang, W.-L.; Peleska, J., Exhaustive model-based equivalence class testing, (Yenigün, H.; Yilmaz, C.; Ulrich, A., Proc. of 25th IFIP WG 6.1 Int. Conf. on Testing Software and Systems, Lect. Notes Comput. Sci., vol. 8254, (2013), Springer), 49-64
[49] da Silva Simão, A.; Petrenko, A., Generating complete and finite test suite for ioco: is it possible?, (Schlingloff, H.; Petrenko, A. K., Proc. of 9th Workshop on Model-Based Testing, MBT ’14, Electron. Proc. Theor. Comput. Sci., vol. 141, (2014), Open Publishing Association), 56-70
[50] Devroey, X.; Perrouin, G.; Legay, A.; Cordy, M.; Schobbens, P.; Heymans, P., Coverage criteria for behavioural testing of software product lines, (Margaria, T.; Steffen, B., Proc. of 6th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change, ISoLA ’14, Lect. Notes Comput. Sci., vol. 8802, (2014), Springer), 336-350
[51] Volpato, M.; Tretmans, J., Towards quality of model-based testing in the ioco framework, (Proc. of 2013 International Workshop on Joining Academia and Industry Contributions to testing Automation, JAMAICA ’13, (2013), ACM), 41-46
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.