Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


Jumel (JUROPA Memory Logger)

A tool for analysing the memory consumption of applications

General information

The JUROPA memory logger is intended for monitoring the memory usage of applications on NUMA architectures (especially JUROPA). It consists currently of two Python scripts:

  • jumel (JUROPA Memory Logger), the actual logger
  • juman (JUROPA Memory Analyzer), a postprocessing tool

Concept

The logger is started with mpiexec and subsequently starts the application to monitor. The monitoring is done per task and/or per node. Information are gathered in certain time steps from files provided by the operating system or from commands that are issued by the logger itself. Currently, the following resources are used/monitored:

  • Monitoring by task
    file /proc/<PID>/status with the keys

    • VmExe (Memory of task marked as executable in kB)
    • VmSt (Stack memory of task in kB)
    • VmData (Heap memory of task in kB)
    • VmSize (Total memory consumption of task in kB)
    • VmLck (Memory locked by the kernel in kB)
    • VmLib (Memory used as shared memory in kB)
    • VmRSS (Resident Set Size of the task in kB)
  • Monitoring per node
    command vmstat with the keys

    • MFree (Free memory of the node in kB)
    • TWait (Number of waiting tasks on the node)
    • Idle (CPU idling in %)
    • TDead (Number of dead tasks on the node)
    • UsedUs (CPU used by user processes in %)
    • UsedKe (CPU used by Kernel in %)

When monitoring per task each task writes the value for each key once per time step to the file .memlog/task<MPI-rank>.log and waits for the next time step. When monitoring by node the process running on core 0 of each node writes the value for each key once per time step to the file .memlog/node<node-name>_task<MPI-task>.log. Both (monitoring per task and monitoring per node) can be active at the same time.

The data produced by jumel can be visualized by the juman script. It performs a basic statistical analysis of the data (minimum, maximum and total sum of values) and generates corresponding graphs.

Contact: Dr Florian Janetzko


Servicemeu

Homepage