PSNC DRMAA for IBM LoadLeveler

Introduction

PSNC DRMAA for LoadLeveler is an implementation of Open Grid Forum DRMAA 1.0 (Distributed Resource Management Application API) specification for submission and control of jobs to IBM Tivoli LoadLeveler. Using DRMAA, grid applications builders, portal developers and ISVs can use the same high-level API to link their software with different cluster/resource management systems.

This software also enables the integration of SMOA Computing with the underlying LoadLeveler system for remote multi-user job submission and control over Web Services.

Installation

To compile and install the library just go to main source directory and type::

  $ ./configure [options] && make
  $ sudo make install

The library was tested with LoadLeveler version 3.5. (for AIX operating systems). If you encountered any problems using the library on the different systems, please use the contact e-mails for reporting the problem.

Notable ./configure script options:

--with-ll-inc LL_INCLUDE_PATH

Path to LL header files (i.e. directory containing llapi.h ). By default the library tries to guess the LL_INCLUDE_PATH and LL_LIBRARY_PATH based on location of the llsubmit executable.

--with-ll-lib LL_LIBRARY_PATH

Path to LL libraries (i.e. directory containing libllapi.a ).

--prefix INSTALLATION_DIRECTORY

Root directory where PSNC DRMAA for LoadLeveler shall be installed. When not given library is installed alongside with LL.

--enable-debug

Compiles library with debugging enabled (with debugging symbols not stripped, without optimizations, and with many log messages enabled). Useful when you are to debug DRMAA enabled application or investigate problems with DRMAA library itself.

There are no unusual requirements for basic usage of library: ANSI C compiler and standard make program should suffice. If you have taken sources directly from SVN repository or wish to run test-suite you would need additional developer tools. For further information regarding GNU build system see the INSTALL file.

Configuration

During DRMAA session initialization (drmaa_init) library tries to read its configuration parameters from locations: /etc/ll_drmaa.conf, ~/.ll_drmaa.conf and from file given in LL_DRMAA_CONF environment variable (if set to non-empty string). If multiple configuration sources are present then all configurations are merged with values from user-defined files taking precedence (in following order: $LL_DRMAA_CONF, ~/.ll_drmaa.conf, /etc/ll_drmaa.conf).

Currently recognized configuration parameters are:

cache_job_state

According to DRMAA specification every drmaa_job_ps() call should query DRM system for job state. With this option one may optimize communication with DRM. If set to positive integer drmaa_job_ps() returns remembered job state without communicating with DRM for cache_job_state seconds since last update. By default library conforms to specification (no caching will be performed).

Type: integer, default: 0

job_categories

Dictionary of job categories. It's keys are job categories names mapped to native specification strings. Attributes set by job category can be overridden by corresponding DRMAA attributes or native specification. Special category name default is used when drmaa_job_category job attribute was not set.

Type: dictionary with string values, default: empty dictionary

terminate_job_on_vacated

Attribute which determines if job should be terminated just after entering the vacated state.

Type: integer, default: 1

Configuration file syntax

Configuration file is in form a dictionary. Dictionary is set of zero or more key-value pairs. Key is a string while value could be a string, an integer or another dictionary.

  configuration: dictionary | dictionary_body
  dictionary: '{' dictionary_body '}'
  dictionary_body: (string ':' value ',')*
  value: integer | string | dictionary
  string: unquoted-string | single-quoted-string | double-quoted-string
  unquoted-string: [^ \t\n\r:,0-9][^ \t\n\r:,]*
  single-quoted-string: '[^']*'
  double-quoted-string: "[^"]*"
  integer: [0-9]+

Native specification

DRMAA interface allows to pass DRM dependent job submission options. Those options may be specified directly by setting drmaa_native_specification job template attribute or indirectly by the drmaa_job_category job template attribute. The legal format of the native options looks like::

  @ll_cmd_keyword1 = value @ll_cmd_keyword2 = value1 value2 value3 

where the ll_cmd_keyword can be any of the keyword accepted by the LoadLeveler. It is user responsibility to provide legal combination of the drmaa_native_specification/drmaa_job_category and the other DRMAA attributes.

Release notes

• 1.0.1 - first public release

Known bugs and limitations

Library covers all DRMAA 1.0 specification with exceptions listed below. It was successfully tested with IBM Tivoli LoadLeveler 3.5.0.5 on AIX OS and passes 43/44 tests of the official DRMAA test-suite. All mandatory and nearly all optional job attributes (except job run duration soft limit, job run duration hard limit, drmaa_transfer_files and drmaa_deadline_time) are implemented.

Known limitations:

• drmaa_control() - DRMAA_CONTROL_RESUME and DRMAA_CONTROL_SUSPEND are not implemented as suspending jobs in LoadLeveler requires administrator privileges.

Authors

The library was developed by:

• Michal Matloka - core implementation
• Mariusz Mamonski - AIX portability issues

This library relies heavily on the Fedstage DRMAA utils code developed by:

• Lukasz Ciesnik.

Acknowledgments

We would like to thank Louisiana Optical Network Initiative for providing us access to the clusters managed by LoadLeveler.

Developer tools

Although not needed for library user the following tools may be required if you intend to develop PSNC DRMAA for LoadLeveler:

• GNU autotools (autoconf, automake, libtool),
• Bison parser generator,
• Docutils for processing this README,
• LaTeX for creating documentation in PDF format,
• Doxygen for generating source code documentation.

Links

Bison:  http://www.gnu.org/software/bison/
Docutils:  http://docutils.sourceforge.net/
LaTeX:  http://www.latex-project.org/
Doxygen:  http://www.stack.nl/~dimitri/doxygen/
DRMAA:  http://www.drmaa.org/
DRMAA 1.0 specification:  http://www.ogf.org/documents/GFD.133.pdf
Official DRMAA test-suite:  http://drmaa.org/testsuite.php
SMOA Computing:  http://larix.man.poznan.pl/wiki/SMOA_Computing
IBM Tivoli LoadLeveler:  http://www-03.ibm.com/systems/software/loadleveler/index.html
Louisiana Optical Network Initiative:  http://loni.org/
LoadLeveler Keywords:  http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/topic/com.ibm.cluster.loadl.doc/loadl33/am2ug30223.html#jobkey