×

An empirical investigation into the learning effects of management flight simulators: a mental models approach. (English) Zbl 1394.90402

Summary: There are many claims about the learning effects of management flight simulators (MFS) as a public education and communication tool in water resource management. However, there is still lack of empirical evidence to support this claim. To address this issue, an exploratory experimental study was conducted to examine the learning effects a series of MFS had on the mental model of water users in the Australian capital territory (ACT). Participants’ mental models of the causal relationships that influence water availability were elicited before and after interacting with a series of increasingly complex MFS, and compared to the reference model structure underpinning the MFS. Results showed that the MFS experience improved participants’ causal knowledge of the model structure and generated critical learning incidents. The findings are interpreted in light of the study limitations along with future research directions.

MSC:

90B90 Case-oriented studies in operations research
91A90 Experimental studies
91A80 Applications of game theory
91B16 Utility theory
91B06 Decision theory
90B50 Management decision making, including multiple objectives

Software:

SEXTANT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] ABS (2011). Australian bureau of statistics: census dictionary Australia, Catalogue Number 2901.0.; ABS (2011). Australian bureau of statistics: census dictionary Australia, Catalogue Number 2901.0.
[2] Ahmad, S.; Simonovic, S. P., Spatial system dynamics: new approach for simulation of water resources systems, Journal of Computing in Civil Engineering, 18, 4, 331-340 (2004)
[3] Alessi, S.; Kopainsky, B., System dynamics and simulation/gaming overview, Simulation & Gaming, 46, 3-4, 223-229 (2015)
[4] Andersen, D. F.; Chung, I. J.; Richardson, G. P., Issues in designing interactive games based on system dynamics models, (Proceedings of the 1990 international system dynamics conference. Proceedings of the 1990 international system dynamics conference, Chestnut Hill (1990)), 31-45
[5] Bakken, B.; Gould, J.; Kim, D., Experimentation in learning organizations: A management flight simulator approach, European Journal of Operational Research, 59, 1, 167-182 (1992)
[6] Bußwolder, P., The effect of a structured method on mental model accuracy and performance in a complex task, Systems, 3, 4, 264-286 (2015)
[7] Cockerill, K., Communicating how water works: results from a community water education program, The Journal of Environmental Education, 41, 3, 151-164 (2010)
[8] Cronin, M. A.; Gonzalez, C., Understanding the building blocks of dynamic systems, System Dynamics Review, 23, 1, 1-17 (2007)
[9] CSIRO (2010) Climate variability and change in south-eastern Australia: a synthesis of findings from Phase 1 of the South Eastern Australian Climate Initiative (SEACI). http://www.seaci.org/publications/documents/SEACI-1
[10] Davidsen, P. I.; Spector, J. M., Critical reflections on system dynamics and simulation/gaming, Simulation & Gaming, 46, 3-4, 430-444 (2015)
[11] Deegan, M.; Stave, K.; MacDonald, R.; Andersen, D.; Ku, M.; Rich, E., Simulation-based learning environments to teach complexity: the missing link in teaching sustainable public management, Systems, 2, 2, 217-236 (2014)
[12] de Winter, J. C., Using the student’s \(t\)-test with extremely small sample sizes, Practical assessment, Research & Evaluation, 18, 10, 1-12 (2013)
[13] Diehl, E.; Sterman, J. D., Effects of feedback complexity on dynamic decision making, Organizational Behavior and Human Decision Processes, 62, 2, 198-215 (1995)
[14] Doyle, J. K.; Ford, D. N., Mental models concepts for system dynamics research, System Dynamics Review, 14, 1, 3-29 (1998)
[15] Doyle, J. K.; Radzicki, M. J.; Trees, W. S., Measuring change in mental models of complex dynamic systems, Complex decision making, 269-294 (2008), Springer: Springer BerlinHeidelberg
[16] Du Toit, D. R.; Biggs, H.; Pollard, S., The potential role of mental model methodologies in multi-stakeholder negotiations: integrated water resources management in South Africa, Ecology and Society, 16, 3, 21 (2011)
[17] Elsawah, S., “Simply, we need to build a new dam”, is it really “SIMPLE”, (Proceedings of the 27th international conference of system dynamics society (PhD colloquium). Proceedings of the 27th international conference of system dynamics society (PhD colloquium), Albuquerque, USA (2009))
[18] Elsawah, S.; McLucas, A.; Mazanov, J., Using a cognitive mapping approach to frame the perceptions of water users about managing water resources: a case study in the Australian capital territory, Water Resources Management, 27, 9, 3441-3456 (2013)
[19] Elsawah, S.; McLucas, A.; Mazanov, J., Communicating about water issues in Australia: a simulation/gaming approach, Simulation & Gaming, 46, 6, 713-741 (2015)
[20] Fernández, J. M.; Selma, M. A.E., The dynamics of water scarcity on irrigated landscapes: Mazarrón and Aguilas in south‐eastern Spain, System Dynamics Review, 20, 2, 117-137 (2004)
[21] Foote, J. L.; Gregor, J. E.; Hepi, M. C.; Baker, V. E.; Houston, D. J.; Midgley, G., Systemic problem structuring applied to community involvement in water conservation, Journal of the Operational Research Society, 58, 5, 645-654 (2007) · Zbl 1177.90266
[22] Ford, A., Testing the Snake River explorer, System Dynamics Review, 12, 4, 305-329 (1996)
[23] Gadgil, S.; Nokes-Malach, T. J.; Chi, M. T., Effectiveness of holistic mental model confrontation in driving conceptual change, Learning and Instruction, 22, 1, 47-61 (2012)
[24] Gary, M. S.; Wood, R. E., Mental models, decision rules, and performance heterogeneity, Strategic Management Journal, 32, 6, 569-594 (2011)
[25] Gogi, A.; Tako, A. A.; Robinson, S., An experimental investigation into the role of simulation models in generating insights, European Journal of Operational Research, 249, 3, 931-944 (2016) · Zbl 1346.91036
[26] Gonzalez, C.; Dutt, V., A generic dynamic control task for behavioral research and education, Computers in Human Behavior, 27, 5, 1904-1914 (2011)
[27] Größler, A., Don’t let history repeat itself-methodological issues concerning the use of simulators in teaching and experimentation, System Dynamics Review, 20, 3, 263 (2004)
[28] Hjorth, P.; Bagheri, A., Navigating towards sustainable development: a system dynamics approach, Futures, 38, 1, 74-92 (2006)
[29] Huff, A., Mapping strategic thought (1990), Wiley: Wiley New York
[30] Isendahl, N.; Dewulf, A.; Brugnach, M.; François, G.; Möllenkamp, S.; Pahl-Wostl, C., Assessing framing of uncertainties in water management practice, Water Resources Management, 23, 15, 3191-3205 (2009)
[31] Karakul, M.; Qudrat-Ullah, H., How to improve dynamic decision making? Practice and promise, Complex decision making, 3-24 (2008), Springer: Springer Berlin Heidelberg
[32] Kelly, R. A.; Jakeman, A. J.; Barreteau, O.; Borsuk, M. E.; ElSawah, S.; Hamilton, S. H., Selecting among five common modelling approaches for integrated environmental assessment and management, Environmental Modelling & Software, 47, 159-181 (2013)
[33] Kolkman, M.; Kok, M.; Van der Veen, A., Mental model mapping as a new tool to analyse the use of information in decision-making in integrated water management, Physics and Chemistry of the Earth, 30, 4-5, 317-332 (2005)
[34] Kollmuss, A.; Agyeman, J., Mind the gap: why do people act environmentally and what are the barriers to pro-environmental behavior?, Environmental Education Research, 8, 3, 239-260 (2002)
[35] Kuruppu, N.; Liverman, D., Mental preparation for climate adaptation: the role of cognition and culture in enhancing adaptive capacity of water management in Kiribati, Global Environmental Change, 21, 2, 657-669 (2011)
[36] Lee, C. B., The interactions between problem solving and conceptual change: system dynamic modelling as a platform for learning, Computers & Education, 55, 3, 1145-1158 (2010)
[37] McLucas, A. C.; Ryan, M. J., Combining generic structures and systems engineering to manage complexity in system dynamics modelling, Journal of Battlefield Technology, 8, 3, 33 (2005)
[38] Miller, G. A., The magical number seven, plus or minus two: some limits on our capacity for processing information, Psychological Review, 63, 2, 81-97 (1956)
[39] Mohammed, S.; Klimoski, R.; Rentsch, J. R., The measurement of team mental models: we have no shared schema, Organizational Research Methods, 3, 2, 123-165 (2000)
[40] Morgan, M.; Fischhoff, B.; Bostrom, A.; Atman, C., Risk communication: a mental models approach (2002), Cambridge University Press: Cambridge University Press Cambridge
[41] Moxnes, E., Overexploitation of renewable resources: the role of misperceptions, Journal of Economic Behavior and Organization, 37, 1, 107-127 (1998)
[42] Mulder, Y. G.; Lazonder, A. W.; de Jong, T., Comparing two types of model progression in an inquiry learning environment with modelling facilities, Learning and Instruction, 21, 5, 614-624 (2011)
[43] Mulder, Y. G.; Lazonder, A. W.; de Jong, T., Using heuristic worked examples to promote inquiry-based learning, Learning and Instruction, 29, 56-64 (2014)
[44] Pahl-Wostl, C., Transitions towards adaptive management of water facing climate and global change, Water Resources Management, 21, 1, 49-62 (2007)
[45] Paich, M.; Sterman, J. D., Boom, bust, and failures to learn in experimental markets, Management Science, 39, 1439-1458 (1993)
[46] Qudrat-Ullah, H., Perceptions of the effectiveness of system dynamics-based interactive learning environments: an empirical study, Computers & Education, 55, 3, 1277-1286 (2010)
[47] Qudrat-Ullah, H., Yes we can: improving performance in dynamic tasks, Decision Support Systems, 61, 23-33 (2014)
[48] Richardson, G. P.; Andersen, D. F.; Maxwell, T. A.; Stewart, T. R., Foundations of mental model research, (Proceedings of the 1994 international system dynamics conference (1994))
[49] Schaffernicht, M., Are you experienced?-a model of learning systems thinking skills, (Proceedings of the 23rd international conference on system dynamics. Proceedings of the 23rd international conference on system dynamics, Boston (2005))
[50] Schaffernicht, M.; Groesser, S. N., A comprehensive method for comparing mental models of dynamic systems, European Journal of Operational Research, 210, 1, 57-67 (2011)
[51] Schaffernicht, M.; Groesser, S. N., Learning to think in circles: improving mental models of a dynamic system, (30th international system dynamics conference. 30th international system dynamics conference, St. Gallen, Switzerland (2012))
[52] Schaffernicht, M. F.; Groesser, S. N., The SEXTANT software: a tool for automating the comparative analysis of mental models of dynamic systems, European Journal of Operational Research, 238, 2, 566-578 (2014) · Zbl 1338.91010
[53] Senge, P. M.; Sterman, J. D., Systems thinking and organizational learning: acting locally and thinking globally in the organization of the future, European Journal of Operational Research, 59, 1, 137-150 (1992)
[54] Stave, K. A., Using system dynamics to improve public participation in environmental decisions, System Dynamics Review, 18, 2, 139-167 (2002)
[55] Stave, K., A system dynamics model to facilitate public understanding of water management options in Las Vegas, Nevada, Journal of Environmental Management, 67, 4, 303-313 (2003)
[56] Stave, K. A., Participatory system dynamics modeling for sustainable environmental management: observations from four cases, Sustainability, 2, 2762-2784 (2010)
[57] Stave, K. A.; Beck, A.; Galvan, C., Improving learners’ understanding of environmental accumulations through simulation, Simulation & Gaming, 46, 3-4, 270-292 (2015)
[58] Sterman, J. D., Learning in and about complex systems, System Dynamics Review, 10, 291-330 (1994)
[59] Sterman, J. D., Business dynamics: systems thinking and modeling for a complex world (2000), Irwin/McGraw-Hill: Irwin/McGraw-Hill Boston
[60] Sterman, J. D.; Sweeney, L. B., Understanding public complacency about climate change: adults’ mental models of climate change violate conservation of matter, Climatic Change, 80, 3-4, 213-238 (2007)
[61] Sterman, J. D., Does formal system dynamics training improve people’s understanding of accumulation?, System Dynamics Review, 26, 4, 316-334 (2010)
[62] Sterman, J.; Franck, T.; Fiddaman, T.; Jones, A.; McCauley, S.; Rice, P., World climate: a role-play simulation of climate negotiations, Simulation & Gaming, 46, 3-4, 348-382 (2014)
[63] Sweeney, L. B.; Sterman, J. D., Bathtub dynamics: initial results of a systems thinking inventory, System Dynamics Review, 16, 4, 249-286 (2000)
[64] Tidwell, V. C.; Passell, H. D.; Conrad, S. H.; Thomas, R. P., System dynamics modeling for community-based water planning: an application to the middle Rio Grande, Aquatic Sciences, 66, 4, 357-372 (2004)
[65] Thompson, J. P.; Howick, S.; Belton, V., Critical learning incidents in system dynamics modelling engagements, European Journal of Operational Research, 249, 3, 945-958 (2016)
[66] van Borkulo, SylviaP.; van Joolingen, WouterR.; Savelsbergh, ElwinR.; Jong, Ton de, What can be learned from computer modeling? Comparing expository and modeling approaches to teaching dynamic systems behavior, Journal of Science Education and Technology, 21, 2, 267-275 (2012), 2012
[67] White, B. Y.; Frederiksen, J. R., Causal model progressions as a foundation for intelligent learning environments, Artificial intelligence, 42, 1, 99-157 (1990)
[68] Williams, A.; Lansey, K.; Washburne, J., A dynamic simulation based water resources education tool, Journal of Environmental Management, 90, 1, 471-482 (2009)
[69] Winz, I.; Brierley, G.; Trowsdale, S., The use of system dynamics simulation in water resources management, Water Resources Management, 23, 7, 1301-1323 (2009)
[70] Winz, I., An interdisciplinary exploration of urban stormwater management (2009), The University of Auckland
[71] Yasarcan, H., Improving understanding, learning, and performances of novices in dynamic managerial simulation games, Complexity, 15, 4, 31-42 (2010)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.