Navigation and service

Usage of PETSc on JUWELS

PETSc version 3.14 has been installed on JUWELS in several versions. All versions for Intel compilers use mkl and include several other libraries.
PETSc/3.14 is configured with standard double precision and integer values
PETSc/3.11.1_debug is configured with --with-debugging=1
PETSc/3.14_int8 is configured with --with-64-bit-indices=1
PETSc/3.11.1_int8_debug is configured with --with-64-bit-indices=1 and --with-debugging=1
PETSc/3.14_complex is configured with --with-scalar-type=complex
PETSc/3.11.1_complex_debug is configured with --with-scalar-type=complex and --with-debugging=1

The debug versions are only available in Stages/Devel-2019a
The latest versions and debug versions will be available in Stages/Devel-2020. To access those versions say

ml load Stages/Devel-2020

Preparations necessary to use PETSc on JUWELS

First you have to load the PETSc version you want, for example:

module load intel-para [Intel InelMPI] [GCC ParaStationMPI]
module load PETSc/3.14
With that command the variables PETSC_DIR and PETSC_ARCH are set to the corresonding PETSc version.

If you don't want to use the PETSc makefiles make sure that your makefile contains the statements

include ${PETSC_DIR}/ ${PETSC_ARCH}/lib/petsc/conf/petscvariables
include ${PETSC_DIR}/ ${PETSC_ARCH}/lib/petsc/conf/petscrules


Examples are available in $PETSC_DIR/src.

To run for instance ex1 from $PETSC_DIR/src/vec/vec/examples/tutorials you have to do the following:

# load the PETSc you want

module load intel-para
module load PETSc/3.14

# copy ex1.c and the makefile to the current directory

cp $PETSC_DIR/share/petsc/examples/src/vec/vec/examples/tutorials/ex1.c .
cp $PETSC_DIR/share/petsc/examples/src/vec/vec/examples/tutorials/makefile .

# compile and link the example code

make ex1

To execute the example on 2 processors you have to write an batchfile runex1 with the following commands:

#!/bin/bash -x
#SBATCH --nodes=1
#SBATCH --ntasks=2
#SBATCH --partition=batch srun -np 2 ex1

To submit the batchfile to the slurm batchsystem say

sbatch batchfile

In the sources of the examples you find a section //TEST in the end where you can see how the examples is to be called.