The page describes the usage of the QCG-Broker Client, which is the command line text client for the QCG-Broker service.

System pre-requirements

• Installed qcg-broker-client package. The RPM installation procedure for SL5/6 is described  here

User Requirements

• User's credential To secure the communication and to authenticate the user to the service the X509 proxy certificate is needed.

Configuration

For its work the client needs user credentials to be present and properly configured. Additionally every user can overwrite some global settings to adapt configuration to its own preferences.

For details please visit the QCG-Client Configuration? page.

QCG Broker Client

The QCG-Client is a java based text client offering access to the QCG infrastructure from the command-line level. The QCG-Client is offered in two versions and the choice of version depends on user's experience and needs:

• QCG-SimpleClient (recommended) is a set of tools (similar to ones offered by queuing systems) for managing of jobs. For user's convenience the simple format of description of jobs (based on #QCG directives) was proposed allowing in a simple way to describe the most popular use cases.
• QCG-AdvancedClient - the client intended for more experienced users requiring more advanced features like workflows, parameter sweep, tasks with many groups of processes with different requirements. The job has to be described by XML document.

QCG Simple Client

The QCG Simple client is recommended for all users who do not need advanced capabilities of the QCG middleware like workflows, parallel jobs with topologies or parameter sweep jobs. In return, it offers access to the most frequently used functionality in very simple and intuitive way. The QCG Simple client consists of a set of command line tools modelled after the set of tools known from popular queueing systems. All tasks submitted to be executed by QCG-Broker must be described in a formal way using the concept of directives known also from queueing systems.

qcg-* Tools

Set of tools allowing to submit/control tasks and to reserve/control resources managed by the QCG infrastructure.

The detailed description of the qcg-* tools?

Submission and controlling of tasks:

•  qcg-cancel - cancel task(s)
•  qcg-clean - clean the working directories of given tasks
•  qcg-connect - connect interactive session to the task
•  qcg-info - display detailed information about the given tasks
•  qcg-list - lists tasks in the system
•  qcg-peek - display ending of (stdout, stderr) streams
•  qcg-proxy - create user proxy certificate
•  qcg-refetch - retry/repeat the transfer of output files/directories
•  qcg-refresh_proxy - refresh user proxy certificate for given tasks
•  qcg-sub - submit the task to be processes by QCG services

Resources reservation and controll:

•  qcg-rcancel - cancel reservation(s)
•  qcg-reserve - reserve resources
•  qcg-rinfo - display information about the given reservation(s)
•  qcg-rlist - list reservation in the system

QCG-Simple description format

Every task submitted to the system has to be described in a formal way. The current section describes the QCG-Simple format which is recommended and sufficient for most tasks. The format does not allow to describe more sophisticated scenarios like workflows, parameter sweep tasks, parallel tasks with topologies. Such scenarios are supported by the XML format called QCG-JobProfile. The QCG-Simple description format is similar to the concept used in queueing systems. The document submitted to be processed by the system is an script with QCG directives and invocation of user code. The QCG directives inform the system how to process the task (eg. about resource requirements and input/output files and directories). The QCG directives are lines starting with the #QCG string.

The detailed description of the #QCG directives

List of #QCG directives:

•  #application application - name of the application
•  #argument argument - argument of the application
•  #deadline deadline- deadline for the task execution
•  #environment environment - environment variable
•  #error error - path for the standard error
•  #executable executable - path to the executable
•  #grant grant- name of the grant
•  #host host - name of the candidate host
•  #input input - path of the standard input file
•  #memory memory - memory requirements
•  #module module- name of the required module
•  #monitor monitor - task monitoring script
•  #name name- human readable name of the task
•  #native native - parameters/requirements passed directly to the queueing system
•  #nodes nodes - requested topology of parallel application
•  #not-after not-after - execute not later then
•  #not-before not-before - do not execute before
•  #note note - human readable description of the task
•  #notify notify - notify about changes of task status
•  #output output - location for the standard output file
•  #persistent persistent - do not remove the working directory
•  #postprocess postprocess - task epilog
•  #preprocess preprocess - task prolog
•  #procs procs - number of requested slots/processes
•  #properties properties - node properties
•  #queue queue - selected queueu
•  #reservation reservation - reservation identifier
•  #stage-in-dir stage-in-dir - input directory
•  #stage-in-file stage-in-file - input file
•  #stage-out-dir stage-out-dir - output directory
•  #stage-out-file stage-out-file - output file
•  #use-reservation use-reservation - use the advance reservation mechanism
•  #use-scratch use-scratch - use the scratch space
•  #walltime walltime - execution duration
•  #watch-output watch-output - monitor the output and send notifications about the progress
•  variable variable - description variable (replaced only in values of QCG directives)

Tasks examples

Please visit the QCG-Simple examples page for description examples.

Contact

Please report all problems and send questions to the qcg@… address.