Search

link to homepage

Institute for Advanced Simulation (IAS)

Navigation and service


LLVIEW: graphical monitoring of LoadLeveler controlled cluster

Download the Software

  1. -> LLview, Version 1.3p13
  2. -> LML_da (data access), Version 1.1p3

Prerequisites

The old data access component llview_da needs a C-compiler and access to the LoadLeveler data access API and library (llapi). The new data access component LML_da is a set of PERL scripts, which require a PERL interpreter to be installed on the target system. Moreover, the following external modules are required for LML_da depending on which remote target it is executed:
  • FindBin, Getopt::Long, Data::Dumper, XML::Simple, Time::Local,Time::HiRes
  • Lite, Storable, Carp, Config::IniFiles, Cwd, String::Scanf
  • File::Basename, XML::Parser, Exporter, File::Temp, Text::ParseWords, Switch
  • Sys::Hostname, IO::Socket::UNIX
In general, these modules are expected to be deployed with the default PERL distribution. If a module is missing, refer to section Install Perl modules locally for installation instructions.

The LLview client is written in Perl and uses following additional modules of Perl:

(1)Tk, Tk::NoteBook, Tk::FileSelect, Compress::Zlib
(2)Archive::Tar, IO::String, Config::IniFiles, XML::Parser::Lite
(3)LWP::UserAgent, Time::HiRes

Some of these modules are normally included in the standard Perl distribution (3). Another part of these modules (2) are only ASCII-based perl-Modules (.pm). These are included in the distribution of LLview (in the ./lib directory) and should be found by perl. The remaining modules (1) of the list above have to be compiled. A pre-compiled version of these modules is available on the download page for AIX 5.2 (perl.rte and freeware perl.rpm) and for the SuSE 9.2 distribution of perl. If you need these modules, download the corresponding file and untar it in the ./lib directory of llview. After this, Perl should find these modules.

If there are still external modules missing, you can find installation instructions in the section Install Perl modules locally.

We have tested the LLview server on AIX based system and the client on different Linux platforms and also under Windows XP using the ActivePerl distribution of Perl (tested with version 5.8.4.810 of ActivePerl).

Installation

LLview is split into LML_da, running on the login-node of your target system, and the LLview-Client, visualizing the data gathered by LML_da usually on your local system. This installation guide explains how to install both components. In this use case LML_da is run on the remote system and the LLview client is executed on a local machine with ssh access to the remote system.

Installation of LML_da

LML_da gathers status data from your remote system.
  1. Copy the /da folder of your LLview distribution to your remote system into the install directory (referred to as $instdir). The /da folder can be found in the LML_da-<version>.tar.gz file.
  2. Create a directory (in the following referred to as $tmp), where tmp files can be generated by LML_da.
  3. Choose the LML_da configuration file for your target system. The configuration files are available in the /da folder. The name schema for these files is LML_da_<targetsystemname>_sample.conf (e.g. LML_da_SLURM_sample.conf). Choose the most appropriate configuration file and adjust it to your system. An example workflow is documented in detail in LML_da and its configuration. Place your workflow configuration file ($conf) into $tmp.
  4. You should at least check the attributes in the <vardef> section. Here you have to specify the relative path to $instdir, the name of a "tmp" directory, which is placed inside of $tmp. Moreover, a "perm" directory will be created inside of $tmp.

  5. Test your LML_da installation by executing the following command from within the $tmp directory:
    $install/LML_da.pl -c=$conf > log.txt 2>err.txt
  6. Check the generated log files log.txt and err.txt for possible error messages. Read them carefully and try to fix them by for instance creating missing directories or checking for typos. The XML-files containing the current status of your target system are placed in the $tmp/tmp directory. Especially, check, if the XML file of the last step is generated. This file is referred to as "$final.xml". This is the input file for the LLview-client.

  7. The above command needs to be invoked in a given update interval (e.g. every minute). Configure a crontab for this task. To add a crontab, do the following:
    call "crontab -e"

    Place the line "* * * * * /path/to/script" into your crontab config file. This will call the script every minute. Create a script, which calls the command in step 4 and place this script at /path/to/script. Check, if the script is really called every minute, which should update the files in $tmp/tmp.

  8. Create a script, which simply prints the content of $final.xml to stdout. This script will be called via SSH from the LLview-client. E.g. create the file "getdata" within the $tmp directory. getdata could contain:
    cat /absolute/path/to/tmp/$final.xml

Note, that the path of the printed file has to be given as absolute path. Change the access rights of getdata so that it can be executed.

Installation of LLview-client

The LLview-client visualizes the input data provided by LML_da.
  1. Start the LLview-client with the corresponding configuration file. At first the client is extracted from the llview-<version>.tar.gz file into the "client" folder. Then the client is started by calling
    ./client/llview -rc <config-file>

    The config-file (e.g. sample.rc) defines how the input data is accessed and how it is visualized. It specifies the layout of the data visualization.

  2. Via the dialog at "options->change" you can specify how data is accessed by the llview-client. In the "General" tag choose "llqxml" as data source, which will allow to call the LML_da script created in step 6 of the LML_da installation.
  3. Switch to the "llqxml" tab to configure access to the target system.
    • Specify the "path to llqxml", which is the path to the remote script (getdata).
    • Enable "use ssh to execute llqxml"
    • specify the hostname of remote system and the userid, with which you connect to the remote system.

    Save your changes by clicking on the "Save" button at the bottom of the options dialog.

  4. Enable the "active" checkbox and click on the update button. LLview should now display the current status of the remote system. You can now use the options dialog for adjusting the LLview layout to your own needs (see Managing Configuration Options).

Combine LLview with webserver

You can setup LLview for transferring status data to a webserver. This data can be accessed directly by the LLview client.

On the server side, the status data needs to be copied to a save directory on the webserver. Therefore, you can add an additional step in the $config file, which triggers the data transmission. This step could for instance use scp or trigger a corresponding php/cgi script of the webserver, where the XML file is sent by POST. This step configuration with scp could for instance look like this:

<step active="1" exec_after="usage" id="copyTarget" type="execute">
<cmd exec="cp $stepinfile $stepoutfile"/>
<cmd exec="scp $stepoutfile *user*@*webserver*:*folder*/llview_status.xml"/>
</step>

This step expects the "usage" step to be the last step in your previous LML_da configuration. This step only works, if the crontab is executed with corresponding credentials. Otherwise, a webserver script could handle the data transfer.

The client has to be configured to update its data from the webserver. Go to Options->Change. Activate "WWW: from Web-Server" as data source in the General tab. Switch to the "WWW" tab, configure the address to the web-server and the path on the web-server to the XML file. It is recommended to protect these files via htaccess. If the file is protected, you can provide the login credentials in the "WWW" tab, too. This configuration has several advantages:

  • the update process is usually much faster, since an update simply downloads the latest XML file placed at the webserver.
  • the XML file can be accessed from any client. The ssh connection is not required anymore.

Install Perl modules locally

When running the llview client or LML_da, there might be certain PERL modules missing such as LWP, XML::Simple or Tk. Perl modules can be installed locally with the following method:

  1. Get the cpanm script placed in tar file App-cpanminus-<version>.tar.gz available at cpan
  2. Unpack the downloaded tar and run the script for each missing module. E.g. for LWP run
    perl App-cpanminus-<version>/bin/cpanm -f -L . LWP

    This will install the LWP library in the current directory ("."). All library files will be placed into ./lib/perl5. This path has to be included in the library search path of the llview scripts.

  3. Make the library path known to the llview script. There are multiple ways to do that as described here. The simplest way for LML_da is to add the new library path to the PERL5LIB environment variable. E.g. export PERL5LIB=/path/to/library/lib/perl5:$PERL5LIB

    For the LLview client you can add new library paths like this:
    Edit the script ./llview/llview
    Right at the beginning there are several additional library paths added.
    E.g. use lib "$FindBin::RealBin/lib/plib_aix5.3p6_32bit_perl5.8.2/lib/";
    Add your new library path accordingly. For LWP you need to add the following line:
    use lib "$FindBin::RealBin/relative/to/lib/perl5";


Servicemeu

Homepage