Application mapping

The QCG-Computing service allows to use abstract application names that are automatically mapped to the absolute paths of the wrapper scripts. The mapping is stored in the file:

/etc/qcg/qcg-comp/application_mapfile

The file has the following syntax:

APPLICATION-NAME VERSION SCRIPT-PATH

where VERSION can be an asterix (*) which means "any version". The mapping file is periodically reloaded (5 minutes by default), so there is no need to restart the qcg-compd service after updating the file. Example application_mapfile file:

MATLAB * /opt/qcg-app-scripts/apps/matlab.app
NAMD * /opt/qcg-app-scripts/apps/namd.app
bash * /opt/qcg-app-scripts/apps/bash.app
R * /opt/qcg-app-scripts/apps/R.app
CFX * /opt/qcg-app-scripts/apps/cfx.app
fluent * /opt/qcg-app-scripts/apps/fluent.app
nwchem * /opt/qcg-app-scripts/apps/nwchem.app
g09	* /opt/qcg-app-scripts/apps/g09.app

The scripts for the most common applications are provided by the qcg-appscripts RPM package.

IMPORTANT: For the correct work QCG-Computing requires the mapping for the bash application, which is mandatory i.e.:

bash * /opt/qcg-app-scripts/apps/bash.app

Frequently Asked Question: Can I use simply mapping to /bin/bash

bash * /bin/bash

? The answer is "No". You must use the bash.qcg which does some additional work (e.g. setup environment variables like QCG_NODEFILE).

qcg-appscripts package

The qcg-appscripts package contains QCG application scripts for the most common applications. Role of the application scripts, apart from launching the proper application, is as follows:

• loading the appropriate modules,
• converting input files to UNIX character encoding,
• setup environment,
• launching user's helper scripts - preprocess, postprocess, assistent,
• monitoring the execution of application through defined schemes or user scripts,
• handling interactive jobs,
• etc.

All files are installed in the directory:

    /usr/share/qcg-appscripts

The directory contains the following sub-directories:

• apps - application configuration files,
• app-scripts - application scripts,
• config - QCG application scripts configuration,
• core - QCG scripts library,
• tools - QCG tools used by applications.

The administrator must create/edit application configuration files (apps) that contain specific to each cluster settings, such as:

• name of the application module,
• environment variables needed by application.

Files from apps directory should be referenced by QCG-Computing application mapping file (/etc/qcg/qcg-comp/application_mapfile).

The application scripts must be accessible by jobs running on cluster worker nodes. Thus directories apps, app-scripts, config, core and tools must be copied to directory shared by all worker nodes. This is done manually by the administrator by running:

/usr/sbin/qcg-appscripts-deploy

script included in the package. The script reads:

/etc/qcg/qcg-comp/app-scripts/config

configuration file for destination directory, where scripts should be deployed.

Installation

To install QCG application scripts:

1. install qcg-appscripts RPM package
2. edit /etc/qcg/qcg-comp/app-scripts/config configuration file and set cluster_shared_path variable to point to a directory shared by all worker nodes
3. launch qcg-appscripts-deploy script
4. edit/create application configuration files in $cluster_shared_path/apps directory and refer to them in the QCG-Computing application mapping file (/etc/qcg/qcg-comp/application_mapfile)

Upgrade

After instalation of new QCG application scripts package:

1. launch qcg-appscripts-deploy script to update files in shared directory.

qcg-appscripts-deploy updates directories app-scripts, core, tools. apps directories (besides bash.app, common.app and template.app is not overwritten.

New application

To handle application, following elements must be provided:

• application configuration file (script in apps directory),
• application script (script in app-scripts directory),
• mapping in /etc/qcg/qcg-comp/application_mapfile file.

Application configuration file

The application configuration file is simple script that:

• loads the required modules or sets PATH/LD_LIBRARY_PATH variables,
• calls proper application script (from the app-scripts directory).

To create new application configuration apps/template.app file can be used as a base.

Example application configuration file for NAMD application:

#!/bin/bash
. $( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/common.app

module load namd/current
MPIEXEC=charmrun
MPIEXEC_ARGS="++verbose ++local +p`cat $QCG_NODEFILE |  wc -l`"

. $QCG_APP_ROOT_DIR/app-scripts/namd.qcg

Application script

The application script is responsible for:

• validation of input arguments,
• preparation of input data,
• application launch with proper parameters,
• optional post processing of output data/files.

Example application script file for NAMD application:

#!/bin/bash 

QCG_APP_NAME="namd"
. $QCG_APP_ROOT_DIR/core/core.qcg


if [ $# -ne 1 ]
then
    echo "Illegal number of arguments: $#"
    echo "Usage:"
    echo "namd.qcg NAMD-FILE"
    exit 1
fi

#QCG -> NAMD
MAIN_FILE="$1"
NAMD_PATH=`which namd2`

qcg_log_debug "main file     = $MAIN_FILE "
qcg_log_debug "using NAMD    = $NAMD_PATH "

qcg_app_wrapper ${MPIEXEC:-mpiexec} $MPIEXEC_ARGS $NAMD_PATH  $MAIN_FILE

Every application script should include core.qcg script which provides rich function library that can be used by application scripts. In the above simple example, application script only validates number of arguments and uses qcg_app_wrapper function to launch MPI version of the NAMD application. The qcg_app_wrapper function should always be used to launch application, as it also activates other QCG mechanism such as: user's pre-processing & post-processing scripts, application monitoring, output file notifications, mounting remote file systems etc.

Any modification/conversion of input file should be done in function qcg_pre_callback, for example:

function qcg_pre_callback {
    if [ -e "$BASH_SCRIPT" ]; then
            dos2unix $BASH_SCRIPT 2>> qcg.debug
    fi
}

if [ "$BASH_SCRIPT_ARGS" ]; then
    qcg_app_wrapper /bin/bash '$BASH_ARGS' -c \"'$BASH_SCRIPT $BASH_SCRIPT_ARGS'\"
else
    qcg_app_wrapper /bin/bash '$BASH_ARGS' -c '$BASH_SCRIPT'
fi

New application version

To create new version of already supported application, it is sufficient to:

1. copy existing application configuration file, eg:

   cp /opt/qcg-app-scripts/apps/namd.app /opt/qcg-app-scripts/apps/namd-6.2.app
   

2. change loaded module and/or environment variables in $cluster_shared_path/apps/namd-6.2.app
3. add new application version to the QCG application mapping file (/etc/qcg/qcg-comp/application_mapfile), eg:

   ...
   NAMD 6.2 /opt/qcg-app-scripts/apps/namd-6.2.app
   ...
   

Configuration

config/config.conf

• QCG_SCRATCH_DIR - the location of job temporary directory (e.g $TMPDIR) - OPTIONAL - default to $TMPDIR (if set) or "/tmp",
• QCG_NO_SCRATCH - if not set, application will be launched in QCG_SCRATCH_DIR
• QCG_DEBUG - if not set do not create qcg.debug file - OPTIONAL,
• MACHINE_FILE - the name of the environment variable pointing to the machine file (e.g. PBS_NODEFILE) - OPTIONAL.
• QCG_GROUP_DIR_ROOT - The root of groups home directory (PL-Grid only). Default: PLG_GROUPS_SHARED - OPTIONAL,
• QCG_NTF_URL - address of QCG-Notification broker service
• PLG_GROUPS_SHARED - same as QCG_GROUP_DIR_ROOT