zbMATH — the first resource for mathematics

A hybrid MPI-OpenMP parallel implementation for pseudospectral simulations with application to Taylor-Couette flow. (English) Zbl 1390.76623
Summary: A hybrid-parallel direct-numerical-simulation method with application to turbulent Taylor-Couette flow is presented. The Navier-Stokes equations are discretized in cylindrical coordinates with the spectral Fourier-Galerkin method in the axial and azimuthal directions, and high-order finite differences in the radial direction. Time is advanced by a second-order, semi-implicit projection scheme, which requires the solution of five Helmholtz/Poisson equations, avoids staggered grids and renders very small slip velocities. Nonlinear terms are evaluated with the pseudospectral method. The code is parallelized using a hybrid MPI-OpenMP strategy, which, compared with a flat MPI parallelization, is simpler to implement, allows to reduce inter-node communications and MPI overhead that become relevant at high processor-core counts, and helps to contain the memory footprint. A strong scaling study shows that the hybrid code maintains scalability up to more than 20,000 processor cores and thus allows to perform simulations at higher resolutions than previously feasible. In particular, it opens up the possibility to simulate turbulent Taylor-Couette flows at Reynolds numbers up to $$\mathcal{O}(10^5)$$. This enables to probe hydrodynamic turbulence in Keplerian flows in experimentally relevant regimes.

MSC:
 76M20 Finite difference methods applied to problems in fluid mechanics 76M22 Spectral methods applied to problems in fluid mechanics 65M70 Spectral, collocation and related methods for initial value and initial-boundary value problems involving PDEs 65M06 Finite difference methods for initial value and initial-boundary value problems involving PDEs 65Y05 Parallel numerical computation 76D05 Navier-Stokes equations for incompressible viscous fluids
BRENT; FFTW; MPI
Full Text:
References:
 [1] Rayleigh, L., On the dynamics of revolving fluids, Proc R Soc Lond A, 93, 148-154, (1917) · JFM 46.1397.04 [2] Balbus, S., A turbulent matter, Nature, 470, 475-476, (2011) [3] Ji, H.; Burin, M.; Schartman, E.; Goodman, J., Hydrodynamic turbulence cannot transport angular momentum effectively in astrophysical disks, Nature, 444, 343-346, (2006) [4] Schartman, E.; Ji, H.; Burin, M. J.; Goodman, J., Stability of quasi-Keplerian shear flow in a laboratory experiment, Astron Astrophys, 543, A94:, (2012), [Astrophysical processes] [5] Edlund, E. M.; Ji, H., Nonlinear stability of laboratory quasi-Keplerian flows, Phys Rev E, 89, 2, 021004, (2014) [6] Paoletti, M. S.; Lathrop, D. P., Angular momentum transport in turbulent flow between independently rotating cylinders, Phys Rev Lett, 106, 024501, (2011) [7] Paoletti, M. S.; van Gils, D. P.M.; Dubrulle, B.; Sun, C.; Lohse, D.; Lathrop, D. P., Angular momentum transport and turbulence in laboratory models of Keplerian flows, Astron Astrophys, 547, A64:, (2012), [Astrophysical processes] [8] Avila, M., Stability and angular-momentum transport of fluid flows between corotating cylinders, Phys Rev Lett, 108, 124501, (2012) [9] Ostilla-Mónico, R.; Verzicco, R.; Grossmann, S.; Lohse, D., Turbulence decay towards the linearly stable regime of Taylor-Couette flow, J Fluid Mech, 748, R3, (2014) [10] Pope, S. B., Turbulent flows, (2000), Cambridge University Press · Zbl 0966.76002 [11] Orszag, S. A.; Patterson, G. S., Numerical simulation of three-dimensional homogeneous isotropic turbulence, Phys Rev Lett, 28, 76-79, (1972) [12] Moin, P.; Mahesh, K., Direct numerical simulation: a tool in turbulence research, Ann Rev Fluid Mech, 30, 539-578, (1998) [13] Jiménez, J., Computing high-Reynolds-number turbulence: will simulations ever replace experiments?, J Turbul, 4, 1-13, (2003) · Zbl 1083.76539 [14] Spalart, P. R., Direct simulation of a turbulent boundary layer up to R_{θ}=1410, J Fluid Mech, 187, 61-98, (1988) · Zbl 0641.76050 [15] Schlatter, P.; Örlü, R., Assessment of direct numerical simulation data of turbulent boundary layers, J Fluid Mech, 659, 116-126, (2010) · Zbl 1205.76139 [16] Kim, J.; Moin, P.; Moser, R., Turbulence statistics in fully developed channel flow at low Reynolds number, J Fluid Mech, 177, 133-166, (1987) · Zbl 0616.76071 [17] Hoyas, S.; Jiménez, J., Scaling of the velocity fluctuations in turbulent channels up to re_{τ}=2003, Phys Fluids, 18, 011702, (2006) [18] Eggels, J. G.M.; Unger, F.; Weiss, M. H.; Westerweel, J.; Adrian, R. J.; Friedrich, R., Fully developed turbulent pipe flow: a comparison between direct numerical simulation and experiment, J Fluid Mech, 268, 175-209, (1994) [19] Wu, X.; Baltzer, J. R.; Andrian, R. J., Direct numerical simulation of a 30R long turbulent pipe flow at R^{+}=685: large- and very large-scale motions, J Fluid Mech, 698, 235-281, (2012) · Zbl 1250.76116 [20] Coughlin, K.; Marcus, P. S., Turbulent bursts in Couette-Taylor flow, Phys Rev Lett, 77, 11, 2214-2217, (1996) [21] Dong, S., Direct numerical simulation of turbulent Taylor-Couette flow, J Fluid Mech, 587, 373-393, (2007) · Zbl 1141.76411 [22] Brauckmann, H. J.; Eckhardt, B., Direct numerical simulation of local and global torque in Taylor-Couette flow up to re=30000, J Fluid Mech, 718, 398-427, (2013) · Zbl 1284.76219 [23] Moser, R. D.; Moin, P.; Leonard, A., A spectral numerical method for the Navier-Stokes equations with applications to Taylor-Couette flow, J Comput Phys, 52, 524-544, (1983) · Zbl 0529.76034 [24] Marcus, P. S., Simulation of Taylor-Couette flow. part 1. numerical methods and comparison with experiment, J Fluid Mech, 146, 1, 45-64, (1984) · Zbl 0561.76037 [25] Canuto, C.; Hussaini, M. Y.; Quarteroni, A.; Zang, T. A., Spectral methods: evolution to complex geometries and applications to fluid dynamics, (2007), Springer · Zbl 1121.76001 [26] Meseguer, A.; Avila, M.; Mellibovsky, F.; Marques, F., Solenoidal spectral formulations for the computation of secondary flows in cylindrical and annular geometries, Eur Phys J Special Topic, 146-1, 249-259, (2007) [27] Kasloff, D.; Tal-Ezer, H., A modified Chebyshev pseudospectral method with an $$\mathcal{O}(N^{- 1})$$ time step restriction, J Comput Phys, 104, 457-469, (1993) · Zbl 0781.65082 [28] Hugues, S.; Randriamampianina, A., An improved projection scheme applied to pseudospectral methods for the incompressible Navier-Stokes equations, Int J Numer Meth Fluids, 28, 501-521, (1998) · Zbl 0932.76065 [29] Raspo, I.; Hugues, S.; Serre, E.; Randriamampianina, A.; Bontoux, P., A spectral projection method for the simulation of complex three-dimensional rotating flows, Comput Fluids, 31, 745-767, (2002) · Zbl 1028.76036 [30] Avila, M.; Grimes, M.; Lopez, J. M.; Marques, F., Global endwall effects on centrifugally stable flows, Phys Fluids, 20, 104104, (2008) · Zbl 1182.76036 [31] Czarny, O.; Serre, E.; Bontoux, P.; Lueptow, R. M., Interaction between Ekman pumping and the centrifugal instability in Taylor-Couette flow, Phys Fluids, 15, 2, 467-477, (2003) · Zbl 1185.76099 [32] Czarny, O.; Serre, E.; Bontoux, P.; Lueptow, R. M., Interaction of wavy cylindrical Couette flow with endwalls, Phys Fluids, 16, 4, 1140-1148, (2004) · Zbl 1186.76120 [33] Mercader, I.; Batiste, O.; Alonso, A., An efficient spectral code for incompressible flows in cylindrical geometries, Comput Fluids, 39, 2, 215-224, (2010) · Zbl 1242.76221 [34] Gresho, P. M.; Sani, R. L., On pressure boundary conditions for the incompressible Navier-Stokes equations, Int J Numer Methods Fluids, 7, 1111-1145, (1987) · Zbl 0644.76025 [35] Orszag, S. A.; Patera, A. T., Secondary instability of wall-bounded shear flows, J Fluid Mech, 128, 347-385, (1983) · Zbl 0556.76039 [36] Fornberg, B., A practical guide to pseudospectral methods, (1998), Cambridge University Press · Zbl 0912.65091 [37] Karniadakis, G. E.; Israeli, M.; Orszag, S. A., High-order splitting methods for the incompressible Navier-Stokes equations, J Comput Phys, 97, 414-443, (1991) · Zbl 0738.76050 [38] Dongarra, J.; Beckman, P., The international exascale software project roadmap, Int J High Perform Comput Appl, 25, 1, 3-60, (2011), ISSN 1094-3420 [39] Frigo, M.; Johnson, S. G., The design and implementation of FFTW3, Proc IEEE, 93, 2, 216-231, (2005), [special issue on “Program Generation, Optimization, and Platform Adaptation”] [40] Jones, C. A., The transition to wavy Taylor vortices, J Fluid Mech, 157, 135-162, (1985) [41] King, G. P.; Li, Y.; Lee, W.; Swinney, H. L.; Marcus, P. S., Wave speeds in wavy Taylor-vortex flow, J Fluid Mech, 141, 365-390, (1984) [42] Brent, R. P., Algorithms for minimization without derivatives, (2002), Dover Publications · Zbl 1009.90133 [43] Coles, D., Transition in circular Couette flow, J Fluid Mech, 21, 03, 385-425, (1965) · Zbl 0134.21705 [44] Barkley, D.; Tuckerman, L. S., Computational study of turbulent laminar patterns in Couette flow, Phys Rev Lett, 94, 1, 014502, (2005) [45] Shi, L.; Avila, M.; Hof, B., Scale invariance at the onset of turbulence in Couette flow, Phys Rev Lett, 110, 204502, (2013) [46] Maretzke, S.; Hof, B.; Avila, M., Transient growth in linearly stable Taylor-Couette flows, J Fluid Mech, 742, 254-290, (2014)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.