Version 15 (modified by mmamonski, 12 years ago) (diff) |
---|
The QCG-Computing service allows to map abstract application name to the absoluth path of the wrapper script. The mapping file is stored in file:
/opt/plgrid/qcg/etc/qcg-comp/application_mapfile
The file has the following syntax:
APPLICATION-NAME VERSION SCRIPT-PATH
where VERSION can be an asterix (*) which means "any version". There is no need to restart the qcg-compd service after updating the file. Example application_mapfile file:
MATLAB * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/matlab.qcg NAMD * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/namd.qcg bash * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/bash.qcg R * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/R.qcg CFX * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/cfx.qcg fluent * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/fluent.qcg nwchem * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/nwchem.qcg g09 * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/g09.qcg qcg-ompi * /opt/exp_soft/plgrid/qcg-app-scripts/app-scripts/qcg-openmpi/qcg-openmpi.qcg
The example scripts can be checkouted from our svn:
svn co https://www.qoscosgrid.org/svn/qcg-computing/trunk/app-scripts
Please put the dir under location that is available from all worker nodes (e.g. /opt/exp_soft/plgrid/qcg-app-scripts/). Every script sources in the beginning its own, site specific, configuration file (e.g. matlab.qcg sources matlab.conf) which is responsible for initializing application environment (e.g. loading environment module, setting PATH). In the next sections we will shortly describe what should be provide in such file for every supported application. Make sure that all the files have execution and read rights for all.
common.conf
- QCG_SCRATCH_DIR - the location of job temporary directory (e.g $TMPDIR) - MANDATORY
- QCG_NO_SCRATCH - if set then the job do not change dir to local job scratch directory
- QCG_DEBUG - if set produces debug messages into qcg.debug file
- MACHINE_FILE - the name of the environment variable pointing to the machine file.
bash.qcg
This application needs no configuration as it is assumed that bash binary cane be always found in /bin directory.
matlab.qcg
In matlab.conf simply load matlab module:
module load matlab/current
Or set the PATH variable:
export PATH=$PATH:/opt/exp_soft/matlab/bin/
namd.qcg
In namd.conf load the NAMD module
module load namd/current
Additionally if namd2 must be started using different command than mpiexec you can provide additional configuration variables:
MPIEXEC=charmrun MPIEXEC_ARGS="++verbose ++local +p`cat $QCG_NODEFILE | wc -l`"
g09.qcg
In g09.conf load the Gaussian module
module load gaussian/current
and set the following variables
#amount of memory in bytes allocated for the job MEM_BYTES=`qstat -f $PBS_JOBID | grep Resource_List.mem | cut -f 2 -d = | tr "b" " " ` #primary, unlimited, scratch GAUSSIAN_SCRATCH="/mnt/lustre/$USER" #additional fast scratches, the value after colon denotes max used size in MB GAUSSIAN_LIMITED_SCRATCHES="$TMPDIR:100" #used as shred memory key (must be an integer!) QCG_GAUSSIAN_SESSION_KEY=${PBS_JOBID%%.*}
moreover you have to compile the QCG gaussian helper library that is LD_PRELOADed before starting gaussian:
cd app-scripts/tools/qcg-gaussian make gcc -ggdb -Wall -fPIC -p -o libqcg-gaussian-dbg.so -shared qcg-gaussian.c -ldl gcc -O3 -Wall -fPIC -p -o libqcg-gaussian.so -shared qcg-gaussian.c -ldl
abinit.qcg
in abinit.conf load the abinit module
module load abinit/current
alternatively you may set and extra mpiexec arguments, e.g.:
MPIEXEC_ARGS=" --mca orte_tmpdir_base $PWD "
or set additional variables
export MV2_ENABLE_AFFINITY=0
Application Scripts - Developers view
This section is target at QCG developers rather than administrators, its list environment variables which influence job life cycle:
Common Input Environment Variables
- QCG_MODULES_LIST - list of environment modules to be loaded (separated by spaces)
- QCG_PREPROCESS - file name of the pre process script
- QCG_MONITOR - file name of the monitor script
- QCG_MONITOR_INTERVAL - determines how frequently the QCG_MONITOR script should be called (in seconds). - TBD
- QCG_NTF_CONSUMER_URL - the address of the Notification consumer interested in receiving notification about job output status change - TBD
- QCG_NTF_WATCH_PATTERN - the regular expression that should trigger notification - TBD
- QCG_NTF_WATCH_FILE - the file to be watched (default: stdout/err file) - TBD
- QCG_NTF_WATCH_INTERVAL - determines how often the watched file should be checked (in seconds) - TBD
- QCG_POSTPROCESS - file name of the post process script
- QCG_OUTERR_FILE - target filename of the joined output/error stream file (default: output.log)
- QCG_ZIPPED_INPUTS - set if the input files must be unziped first (e.g. QCG_ZIPPED_INPUTS=inputs.zip)
- QCG_ZIP_OUTPUTS - whether to zip results into $QCG_ZIP_OUTPUTS.zip file
- QCG_ZIP_OUTPUTS_FILTER - The wildcard pattern of files to be stored in the zip file (TBD)
- QCG_COMP_PROCESSES_MAP - topology of hybrid application (e.g. QCG_COMP_PROCESSES_MAP="4:1:1:1")
Application Scripts - Users view
Here we list environment variables available in any job submitted as QCG-Application (e.g. bash):
- QCG_NODEFILE - the machinefile like file with list of nodes allocated for the job