Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


Example

In order to explain the usage and features of jumel in more detail, an example application Mapt ("Memory allocation per task") is provided in

$JUMEL_ROOT/examples/Mapt

It is a small MPI application written in C where each MPI task allocates a certain amount of memory, depending on its rank:

//
// Memory allocation per task
//

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <mpi.h>
#include <time.h>

int main (int argc,char *argv[]){

int nranks,my_rank;
float my_memory = 0;
int *my_allocation;

MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD, &nranks);
MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
if (my_rank == 0) printf ("I am master
of %d\n",nranks);

my_memory = (my_rank % 3) + 0.5;

sleep(my_rank+2.0);
printf("Rank: %d. Allocating %f Gigabytes.",
my_rank,my_memory);
my_allocation = (int *)
malloc(my_memory*1024.0*1024.0*1024.0);
MPI_Barrier(MPI_COMM_WORLD);
sleep((nranks-my_rank)*+1.0);
free(my_allocation);
sleep(2.0);
MPI_Barrier(MPI_COMM_WORLD);

MPI_Finalize();
return(0);
}

To compile the application and run it with jumel on two nodes with 8 tasks per node, use the job script mapt.js provided in the Mapt folder

#!/bin/bash -x
#MSUB -l nodes=2:ppn=8
#MSUB -l walltime=00:05:00
#MSUB -v tpt=1

module load jumel

mpicc -o mapt.x mapt.c
mpiexec -np 16 -e PBS_JOBID\
$JUMEL_BIN/jumel -n -p -t -a "mapt.x" > mapt.out

This will log the memory consumption of the application "mapt.x" (value for the option -a, do not forget the quotationmarks) per node (option -n) and per task (option -t). Additionally, a table with resources available to the application on each node will be printed in the node log files (option -p). The default time step for gathering the information is 10 seconds (can be changed with the jumel option -d, WARNING: using a time step smaller than 5 seconds might be useless, since the extraction of the values might be slower).

After the run is finished, you can find the results in the subfolder .memlog. There are two node logfiles, node<node-name>_task000000.log and node<node-name>_task000008.log, and 16 task logfiles task000000.log to task000015.log. All logfiles are in ASCII format and can be directly used as input for gnuplot.

Each node logfile consists of a header which includes general information about the run, for example the call to the application, date and time of the run, the PBS-ID of the job and the node name. Additionally, the resources available on the node are listed (obtained with the ulimit command). This section is followed by a table with the node statistics for every time step.

Each task logfile has the same header section as a node logfile, followed by a table with task statistics.

In order to ease the analysis of the data, the juman postprocessing tool can be used to visualize the memory consumption (gnuplot and gv must be installed, which are not part of the jumel software). With

juman -s all -i -n -t

the complete analysis (option -s all) for the node statistics (option -n) and the task statistics (option -t) is performed. The option -i uses gv to display the analysis. For the analysis of the node statistics the key MFree is used, for the analysis of the task statistics the key VmSize is used. Other keys can be specified with the options --nodekey and --taskkey, respectively.

The following graphs were obtained from a jumel run of the Mapt application analysed with juman on JUROPA.

1) Memory consumption given by VmSize (in kBytes) per task at each time step:

Memory consumption given by VmSize (in kBytes) per task.

2) Minimum and maximum value of VmSize (in kBytes) across tasks ploted against time step:rank(max VmSize value),rank(min VmSize value):

Minimum and Maximum value of VmSize (in kBytes) across tasks.

3) Total value of VmSize (in kBytes), summed over all tasks per time step:

Total value of VmSize (in kBytes), summed over all tasks.

4) MFree values (in kBytes) for each node:

MFree values (in kBytes) for each node.

5) Minimum and Maximum values of MFree (in kBytes) across nodes ploted against time step:node(max MFree value),node(min MFree value)::

Minimum and Maximum values of MFree (in kBytes) across nodes.

6) Total value of MFree (in kBytes) summed over all nodes per time step:

Total value of MFree (in kBytes) summed over all nodes.


Servicemeu

Homepage