Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


ParFlow+p4est

An open-source parallel watershed flow model

Scientific area

High resolution parallel simulation of variably saturated flow.

Short description

ParFlow is an integrated hydrology modeling software that simulates saturated and variably saturated flow in heterogeneous porous media. The code employs a discretization of the three dimensional Richards equation based on cell centered finite differences. The nonlinearity of the resulting equations is addressed with a globalized Newton's method. The sparse system of algebraic equations arising in each Newton step is solved numerically with a multigrid preconditioned Krylov iteration. ParFlow has been demonstrated previously to have excellent solver scalability up to 16,384 processes.

As part of our work in the DFG SFB/TR32 project D8, we have identified ParFlow's way of parallelization of the computational mesh as the main limiting factor in further extending its scalability. Hence, we have reorganized the mesh subsystem to use fast partitioning algorithms provided by the parallel adaptive mesh refinement software library p4est, which is known for its modularity and proven scalability.

We have created a modified version of ParFlow that is coupled to p4est to use it as its parallel mesh manager, currently focusing on uniform meshes. Our approach is minimally invasive and preserves most of ParFlow's data structures, the configuration system and the setup and solver pipeline. We have executed the modified version on up to 458,752 MPI processes of Juqueen, demonstrating good parallel scalability and significantly reduced setup time.

ParFlow+p4estSaturation field from an infiltritation experiment in the presence of macroporosity and layered soil horizons. The heterogeneous permeability parameter is simulated to model vertical and lateral preferential flow paths, respectively.

Scalability

  • 458,752 cores on BlueGene/Q (JUQUEEN)

Performance of ParFlow+p4est on JUQUEENStrong scaling timing results of ParFlow+p4est on JUQUEEN for different problem sizes. The test case solves a 3D regular topography problem in which lateral flow is driven by slopes based on sine and cosine functions.

Programming language and model

  • Fortran/C
  • MPI

Tested on platforms

  • BlueGene/Q

Application developers, contributors, and contact

ParFlow:
Stefan Kollet
Forschungszentrum Jülich IBG-3
Wilhelm-Johnen-Straße
52428 Jülich
s.kollet@fz-juelich.de

Reed M. Maxwell
Colorado School of Mines
1500 Illinois St
Golden, CO 80401
rmaxwell@mines.edu

Carol S. Woodward
Center for Applied Scientific Computing
Lawrence Livermore National Laboratory
Livermore, CA 94550-9234
cswoodward@llnl.gov

p4est:
Carsten Burstedde
Institut für Numerische Simulation
Wegelerstraße 6
53115 Bonn
burstedde@ins.uni-bonn.de

ParFlow+p4est:
Jose A. Fonseca
Institut für Numerische Simulation
Wegelerstraße 6
53115 Bonn
fonseca@ins.uni-bonn.de

(Text and images provided by the developers)


Servicemeu

Homepage