= PSNC DRMAA for IBM !LoadLeveler =

== Introduction ==

PSNC DRMAA for !LoadLeveler is an implementation of Open Grid Forum [#DRMAA DRMAA] 
1.0 (Distributed Resource Management Application API) [#drmaa_specification specification] for submission and control of jobs to [#LoadLeveler IBM Tivoli LoadLeveler].  Using DRMAA,
grid applications builders, portal developers and ISVs can use the same
high-level API to link their software with different cluster/resource
management systems.

This software also enables the integration of [#smoa_computing SMOA Computing] with the
underlying !LoadLeveler system for remote multi-user job submission and control
over Web Services.

== Installation ==

To compile and install the library just go to main source directory
and type::
{{{
#!sh
  $ ./configure [options] && make
  $ sudo make install
}}}
The library was tested with !LoadLeveler version 3.5. (for AIX operating systems).
If you encountered any problems using the library on the different systems, please use 
the contact e-mails for reporting the problem.

Notable `./configure` script options:

  `--with-ll-inc` LL_INCLUDE_PATH
    Path to LL header files (i.e. directory containing `llapi.h` ). By default the library tries 
    to guess the `LL_INCLUDE_PATH` and `LL_LIBRARY_PATH` based on location
    of the `llsubmit` executable.

  `--with-ll-lib` LL_LIBRARY_PATH
    Path to LL libraries (i.e. directory containing `libllapi.a` ).

  `--prefix` INSTALLATION_DIRECTORY
    Root directory where PSNC DRMAA for !LoadLeveler shall be installed.
    When not given library is installed alongside with LL.

  `--enable-debug`
    Compiles library with debugging enabled (with debugging symbols not
    stripped, without optimizations, and with many log messages enabled).
    Useful when you are to debug DRMAA enabled application
    or investigate problems with DRMAA library itself.

There are no unusual requirements for basic usage of library: ANSI C
compiler and standard make program should suffice.  If you have taken
sources directly from SVN repository or wish to run test-suite you would
need additional [=#dev_tools developer tools].  For further information regarding
GNU build system see the INSTALL file.


== Configuration ==


During DRMAA session initialization (`drmaa_init`) library tries to
read its configuration parameters from locations: `/etc/ll_drmaa.conf`,
`~/.ll_drmaa.conf` and from file given in `LL_DRMAA_CONF` environment
variable (if set to non-empty string).  If multiple configuration
sources are present then all configurations are merged with values
from user-defined files taking precedence (in following order:
`$LL_DRMAA_CONF`, `~/.ll_drmaa.conf`, `/etc/ll_drmaa.conf`).

Currently recognized configuration parameters are:

  cache_job_state
    According to DRMAA specification every `drmaa_job_ps()` call should
    query DRM system for job state.  With this option one may optimize
    communication with DRM.  If set to positive integer `drmaa_job_ps()`
    returns remembered job state without communicating with DRM for
    `cache_job_state` seconds since last update.  By default library
    conforms to specification (no caching will be performed).

    Type: integer, default: 0

  job_categories
    Dictionary of job categories.  It's keys are job categories names
    mapped to [=#native_specification native specification] strings.  Attributes set by job
    category can be overridden by corresponding DRMAA attributes or
    native specification.  Special category name `default` is used when
    `drmaa_job_category` job attribute was not set.

    Type: dictionary with string values, default: empty dictionary
	
  terminate_job_on_vacated
	Attribute which determines if job should be terminated just after entering the `vacated` state.
	
	Type: integer, default: 1

=== Configuration file syntax ===

Configuration file is in form a dictionary.
Dictionary is set of zero or more key-value pairs.
Key is a string while value could be a string, an integer
or another dictionary.


{{{
  configuration: dictionary | dictionary_body
  dictionary: '{' dictionary_body '}'
  dictionary_body: (string ':' value ',')*
  value: integer | string | dictionary
  string: unquoted-string | single-quoted-string | double-quoted-string
  unquoted-string: [^ \t\n\r:,0-9][^ \t\n\r:,]*
  single-quoted-string: '[^']*'
  double-quoted-string: "[^"]*"
  integer: [0-9]+
}}}
[=#native_specification]
== Native specification ==

DRMAA interface allows to pass DRM dependent job submission options.
Those options may be specified directly by setting `drmaa_native_specification` job
template attribute or indirectly by the `drmaa_job_category` job template attribute. 
The legal format of the native options looks like::
{{{
  @ll_cmd_keyword1 = value @ll_cmd_keyword2 = value1 value2 value3 
}}}
where the `ll_cmd_keyword` can be any of the [=#keyword keyword] accepted by the !LoadLeveler. It is user
responsibility to provide legal combination of the `drmaa_native_specification`/`drmaa_job_category`
and the other DRMAA attributes.

== Release notes ==

 * 1.0.1 - first public release

=== Known bugs and limitations === 

Library covers all [#drmaa_specification DRMAA 1.0 specification] with exceptions listed
below.  It was successfully tested with [#LoadLeveler IBM Tivoli LoadLeveler 3.5.0.5] on AIX
OS and passes 43/44 tests of the [#test_suite official DRMAA test-suite]. All mandatory and
nearly all optional job attributes (except job run duration soft limit, job run duration hard limit, 
drmaa_transfer_files and drmaa_deadline_time) are implemented.

Known limitations:

 * `drmaa_control()` - `DRMAA_CONTROL_RESUME` and `DRMAA_CONTROL_SUSPEND` are not implemented as suspending jobs in !LoadLeveler requires administrator privileges. 

=== Authors ===

The library was developed by:

 * Michal Matloka - core implementation
 * Mariusz Mamonski - AIX portability issues

This library relies heavily on the ''Fedstage DRMAA utils'' code developed by:

 * Lukasz Ciesnik.

=== Acknowledgments ===

We would like to thank [#loni Louisiana Optical Network Initiativ] for providing us access to the clusters managed by !LoadLeveler.

[=#dev_tools]
=== Developer tools ===

Although not needed for library user the following tools may be required
if you intend to develop PSNC DRMAA for !LoadLeveler:

 * GNU autotools (autoconf, automake, libtool),
 * [#Bison Bison] parser generator,
 * [#docutils Docutils] for processing this `README`,
 * [#latex LaTeX] for creating documentation in PDF format,
 * [#doxygen Doxygen] for generating source code documentation.

=== Links ===

[=#Bison] Bison: http://www.gnu.org/software/bison/ \\
[=#docutils] Docutils:  http://docutils.sourceforge.net/ \\
[=#latex] LaTeX:     http://www.latex-project.org/ \\
[=#doxygen] Doxygen:   http://www.stack.nl/~dimitri/doxygen/ \\
[=#DRMAA] DRMAA: http://www.drmaa.org/ \\
[=#drmaa_specification] DRMAA 1.0 specification: http://www.ogf.org/documents/GFD.133.pdf \\ 
[=#test_suite] Official DRMAA test-suite: http://drmaa.org/testsuite.php \\
[=#smoa_computing]SMOA Computing: http://larix.man.poznan.pl/wiki/SMOA_Computing \\
[=#LoadLeveler] IBM Tivoli !LoadLeveler: http://www-03.ibm.com/systems/software/loadleveler/index.html \\
[=#loni] Louisiana Optical Network Initiative: http://loni.org/ \\
[=#keyword] !LoadLeveler Keywords: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/topic/com.ibm.cluster.loadl.doc/loadl33/am2ug30223.html#jobkey \\