| Version 17 (modified by piontek, 12 years ago) (diff) |
|---|
General information
Invoked with -h/--help option all qcg-* commands display the coherent information about the usage.
qcg-<command> -h qcg-<command> --help
Many qcg- commands take as an argument list of task identifiers identifiers. The list consists of identifiers in the following form JOBIDTASKID separated by the space character. The JOBID is an identifier of the job and TASKID is an optional identifier of the task. If the TASKID is not given, the default one - "task" is assumed. For tasks described in the QCG-Simple format the JOBID identifier is enough.
qcg-cancel
The command allows to cancel list of tasks.
Cancel task(s)
usage: qcg-cancel [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
usage: qcg-client cancel_task [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
Options:
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-cancel J1363865127610__1345 qcg-cancel J1363865127610__1345/task123 qcg-cancel J1363865127610__1345 J1363865128119__5737
qcg-clean
The command removes working directories of the set of tasks.
Clean task(s). Remove working directory
usage: qcg-clean [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
usage: qcg-client clean_task [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
Options:
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-clean J1363865127610__1345 qcg-clean J1363865127610__1345/task qcg-clean J1363865127610__1345 J1363865128119__5737
qcg-connect
The command opens an interactive shell session in the working directory of the task.
IMPORTANT: It is possible to connect only to the task in the RUNNING state.
Connect to the task. Open interactive terminal in the working directory of the given task
usage: qcg-connect [-h] [-v] JOBID[/TASKID]
usage: qcg-client connect_to_task [-h] [-v] JOBID[/TASKID]
Options:
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-connect J1363865127610__1345 qcg-connect J1363865127610__1345/task
qcg-info
The command displays comprehensive information about the given set of tasks.
Display information about task(s)
usage: qcg-info [-d] [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
usage: qcg-client task_info [-d] [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
Options:
-d,--description display user's job description
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-info J1363865127610__1345 qcg-info J1363865127610__1345/task
Below there is an example of the qcg-info command output
[plgpiontek@qcg host]$ qcg-info -d J1376043927718__9398
https://qcg-broker.man.poznan.pl:8443/qcg/services/
/C=PL/O=GRID/O=PSNC/CN=qcg-broker/qcg-broker.man.poznan.pl
UserDN = /C=PL/O=PL-Grid/O=Uzytkownik/O=PCSS/CN=Tomasz Piontek/CN=plgpiontek
ProxyLifetime = 6 Days 21 Hours 50 Minutes 51 Seconds
J1376043927718__9398 :
Note:
UserDN: /C=PL/O=PL-Grid/O=Uzytkownik/O=PCSS/CN=Tomasz Piontek/CN=plgpiontek
TaskType: SINGLE
SubmissionTime: Fri Aug 09 12:25:28 CEST 2013
FinishTime: Fri Aug 09 12:27:14 CEST 2013
ProxyLifetime: P6DT21H50M48S
Status: FINISHED
StatusDesc:
StartTime: Fri Aug 09 12:25:28 CEST 2013
Purged: true
DescriptionType: QCG_SIMPLE
Description:
#QCG procs=5
#QCG walltime=PT5M
#QCG output=p2.${JOB_ID}
cat $QCG_NODEFILE
# Provided as QCG-Broker Client argument
# Added by QCG-Broker Client
#QCG client-url=gsiftp://qcg.man.poznan.pl:2811/
#QCG working-dir=/home/plgrid/plgpiontek/reef/EGI/
#QCG script-file=./p2.qcg
#QCG environment=QCG_TASK_INDEX->0
Allocation:
HostName: zeus.cyfronet.pl
ProcessesCount: 5
ProcessesGroupId: qcg
Status: FINISHED
StatusDescription:
SubmissionTime: Fri Aug 09 12:25:28 CEST 2013
FinishTime: Fri Aug 09 12:27:19 CEST 2013
LocalSubmissionTime: Fri Aug 09 12:25:31 CEST 2013
LocalStartTime: Fri Aug 09 12:26:54 CEST 2013
LocalFinishTime: Fri Aug 09 12:27:14 CEST 2013
Purged: true
qcg-list
The command displays table with information about tasks that meet the given criteria.
Display information about tasks
usage: qcg-list [-C <period>] [-h] [-n <NOTE>] [-N] [-p <PURGED>] [-R] [-S <period>] [-s <status(es)>] [-v]
usage: qcg-client get_tasks [-C <period>] [-h] [-n <NOTE>] [-N] [-p <PURGED>] [-R] [-S <period>] [-s <status(es)>] [-v]
Options:
-C,--changed <period> changed in the period of time (not supported yet)
-h,--help display help message
-n,--note <NOTE> list elements with the given note
-N,--no-list do not display list
-p,--purged <PURGED> display whether the working directory is purged or
not (PURGED=true|false).
-R,--report display summary report.
-S,--submitted <period> submited in the period of time. If the status is
defined the default value of the period is 1 day.
-s,--status <status(es)> status(es) of the job. Lack of this option is
equivalent to requesting unterminated activities.
-v,--version display version
qcg-list qcg-list -S 1d qcg-list -s failed qcg-list -S 1d -s all -R
- The options -C/--changed and -S/--submitted accept definition of time period of days ("d"), hours ("h") or minutes ("m").
- The Option -s/--status allows to limit the list to task with concrete statuses. It is possible to specify list of statuses separate by commas.
- The option -n/--note supports search patterns with "%" as a wildcard. (e.g. "experiment_%").
List of possible task statuses:
- UNSUBMITTED – processing of the task postponed due to the precedence constraints,
- UNCOMMITED - tasks waits to be committed,
- QUEUED – tasks waits in the queue to be processed,
- PREPROCESSING – the execution environment in prepared,
- PENDING – application waits in the queuing system to be executed,
- RUNNING – task (application) is running,
- STOPPED – application has finished, system in ready to stage out results,
- POSTPROCESSING – system stages out results or cleans the working directory
- FINISHED – the whole task (application and all transfers) has finished
- FAILED – processing of the task failed
- CANCELED – the task has been canceled by the user or system
For the user convenience set of some constants was defined:
- all - tasks in any status,
- terminated - tasks in any terminal status (FINISHED,FAILED,CANCELED)
- unterminated - unfinished tasks (UNSUBMITED,UNCOMMITED,QUEUED,PREPROCESSING,PENDING,RUNNING,STOPPED,POSTPROCESSING).
List of displayed columns:
| Column | Meaning |
|---|---|
| IDENTIFIER | task identifier |
| NOTE | value of the #QCG note directive |
| SUBMISSION TIME | time when the task was submitted to the system |
| START TIME | estimated of real start time |
| FINISH TIME | time when the task has been finished |
| STATUS | task status |
| HOSTNAME | | name of the cluster which the task is or was executed on |
| FLAGS | Some additional information about the task |
| STATUS DESCRIPTION | human readable description (reason) of the status. Should be present for FAILED status |
FLAGS:
- S - QCG-Simple dialect
- X - QCG-JobProfile (XML) dialect
- J - JSDL dialect
- P - the working directory has been purged
- UP - the working directory has not been purged.
[plgpiontek@qcg host]$ qcg-list -S 10h -s all -R https://qcg-broker.man.poznan.pl:8443/qcg/services/ /C=PL/O=GRID/O=PSNC/CN=qcg-broker/qcg-broker.man.poznan.pl UserDN = /C=PL/O=PL-Grid/O=Uzytkownik/O=PCSS/CN=Tomasz Piontek/CN=plgpiontek ProxyLifetime = 6 Days 20 Hours 49 Minutes 13 Seconds JOB IDENTIFIER NOTE SUBMISSION TIME START TIME FINISH TIME STATUS HOSTNAME FLAGS STATUS DESCRIPTION J1376032871771__3345 09.08.13 09:21 09.08.13 09:21 FAILED galera S UP errorMessage = jobS* J1376033086383_TASK_9* 09.08.13 09:24 09.08.13 09:32R 09.08.13 12:26 CANCELED zeus no* X UP J1376033996852__3397 09.08.13 09:39 09.08.13 09:41 09.08.13 09:41 FINISHED zeus S P J1376034209604_TASK_8* 09.08.13 09:43 09.08.13 09:52 09.08.13 10:22 FAILED nova ze* X UP Application failed J1376034216649_TASK_4* 09.08.13 09:43 09.08.13 09:49 09.08.13 09:58 FAILED nova ze* X UP Application failed J1376034222276_TASK_6* 09.08.13 09:43 09.08.13 09:53 09.08.13 10:05 FAILED nova ze* X UP Application failed J1376035574438_TASK_8* 09.08.13 10:06 09.08.13 10:21 09.08.13 12:26 CANCELED nova ze* X UP J1376043152403__8834 09.08.13 12:12 09.08.13 12:17 CANCELED inula S P J1376043474972__0601 09.08.13 12:17 09.08.13 12:20 09.08.13 12:23 FINISHED nova S P J1376043567862__3102 09.08.13 12:19 09.08.13 12:20 09.08.13 12:23 FINISHED hydra S P J1376043655456__7730 09.08.13 12:20 09.08.13 12:21 09.08.13 12:23 FINISHED hydra S P J1376043927254__5479 09.08.13 12:25 09.08.13 12:27 09.08.13 12:27 FINISHED hydra S P J1376043927718__9398 09.08.13 12:25 09.08.13 12:26 09.08.13 12:27 FINISHED zeus S P J1376047699549__5485 09.08.13 13:28 09.08.13 13:28 09.08.13 13:28 FINISHED inula S P IMPORTANT: * - the string was truncated to the width of the column List of truncated job identifiers: J1376033086383_TASK_9115 J1376034209604_TASK_8944 J1376034216649_TASK_4688 J1376034222276_TASK_6093 J1376035574438_TASK_8535 Number of tasks: FINISHED : 7 CANCELED : 3 FAILED : 4 TOTAL: 14
qcg-peak
The command allows to watch the ending of the requested file. If the file is not specified the concatenation of stdout and stderr is displayed.
Peek the output file(s)
usage: qcg-peek [-c <NUMBER>] [-f <FILE>] [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
usage: qcg-client peek_output [-c <NUMBER>] [-f <FILE>] [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
Options:
-c,--characters <NUMBER> number of characters
-f,--file <FILE> file to peek
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-peek J1363865127610__1345 qcg-peek -f output.txt J1363865127610__1345 qcg-peek -f output.txt -c 1000 J1363865127610__1345
qcg-proxy
Create user proxy usage: proxy_init [-h] [-v] Options: -h,--help display help message -v,--version display version
qcg-refetch
The commands allows to repeat once again the transfer of output files and directories. It can be useful when the first automatic attempt failed due to lack of valid user proxy certificate or due to temporal network problems. The user proxy certificate can be refreshed using the qcg-refresh_proxy command.
Refetch output files
usage: qcg-refetch [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
usage: qcg-client stage_out [-h] [-v] JOBID[/TASKID] [JOBID[/TASKID]] ...
Options:
-h,--help display help message
-v,--version display version
Arguments:
JOBID identifier of the job
TASKID optional identifier of the task
The default identifier of task is 'task'
qcg-refetch J1363865127610__1345 qcg-refetch J1363865127610__1345/task
qcg-refresh_proxy
The command allows to refresh or prolongs the user proxy certificate for the set of experiments (jobs). The valid proxy certificate is needed for example to transfer output files and directories. The remaining life time of proxy certificate can be checked using the "qcg-info" command.
Refresh proxy for the given job(s) usage: qcg-client refresh_proxy [-h] [-v] JOBID [JOBID] ... Options: -h,--help display help message -v,--version display version Arguments: JOBID identifier of the job
qcg-sub
Submit job(s) to the QCG infrastructure usage: qcg-sub [-D <DIRECTIVE>] [-h] [-I] [-J | -S | -X] [-R <NUMBER>] [-v] FILE [FILE] ... usage: qcg-client submit_job [-D <DIRECTIVE>] [-h] [-I] [-J | -S | -X] [-R <NUMBER>] [-v] FILE [FILE] ... Options: -D,--directive <DIRECTIVE> adds #QCG directives to the description -h,--help display help message -I,--interactive Interactive mode -J,--jsdl JSDL dialect -R,--repeat <NUMBER> repeat submission (only for testing) -S,--simple QCG Simple dialect -v,--version display version -X,--xml QCG XML dialect
qcg-rcancel
Cancel reservation(s) usage: qcg-rcancel [-h] [-v] RESID [RESID] ... usage: qcg-client cancel_reservation [-h] [-v] RESID [RESID] ... Options: -h,--help display help message -v,--version display version Arguments: RESID identifier of the reservation
qcg-reserve
Create reservation(s) usage: qcg-reserve [-h] [-J | -S | -X] [-v] FILE [FILE] ... usage: qcg-client create_reservation [-h] [-J | -S | -X] [-v] FILE [FILE] ... Options: -h,--help display help message -J,--jsdl JSDL dialect -S,--simple QCG Simple dialect -v,--version display version -X,--xml QCG XML dialect Arguments: FILE Path to file with job description
qcg-rinfo
Display information about the given reservation(s) usage: qcg-rinfo [-d] [-D] [-h] [-v] RESID [RESID] ... usage: qcg-client reservation_info [-d] [-D] [-h] [-v] RESID [RESID] ... Options: -d,--description display user's job description -D,--qcg_description display QCG job description -h,--help display help message -v,--version display version Arguments: RESID identifier of the reservation
qcg-rlist
Display information about reservations
usage: qcg-rlist [-C <period>] [-h] [-n <NOTE>] [-N] [-R] [-S <period>] [-s <status(es)>] [-v]
usage: qcg-client get_reservations [-C <period>] [-h] [-n <NOTE>] [-N] [-R] [-S <period>] [-s <status(es)>] [-v]
Options:
-C,--changed <period> changed in the period of time (not supported yet)
-h,--help display help message
-n,--note list elements with the given note
-N,--no-list do not display list
-R,--report display summary report.
-S,--submitted <period> submited in the period of time. If the status is
defined the default value of the period is 1 day.
-s,--status <status(es)> status(es) of the job. Lack of this option is
equivalent to requesting unterminated activities.
-v,--version display version
