Changes between Version 109 and Version 110 of DCWoRMS
- Timestamp:
- 05/05/20 09:17:16 (5 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
DCWoRMS
v109 v110 88 88 Workload contains information about jobs, their structure, resource requirements, relationships, time intervals etc. We assumed a model in which each job consists of one or more tasks. A job may contain preceding constraints between tasks (workflow). The next sections provide information on how workloads are described and generated in DCworms. 89 89 90 The workload used to perform experiment consists of two parts: required swf/gwf format file and optional xml job description. Swf is the standard workload format, described by Dror Feitelson - see [http://www.cs.huji.ac.il/labs/parallel/workload/swf.html http://www.cs.huji.ac.il/labs/parallel/workload] for details. Xml job description format is described by following xsd schema [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/grms3/GrmsJobDescriptionSchema.xsd GrmsJobDescriptionSchema.xsd]90 The workload used to perform experiment consists of two parts: required swf/gwf format file and optional xml job description. Swf is the standard workload format, described by Dror Feitelson - see [http://www.cs.huji.ac.il/labs/parallel/workload/swf.html http://www.cs.huji.ac.il/labs/parallel/workload] for details. Xml job description format is described by following xsd schema [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/grms3/QCGJobDescriptionSchemaNew.xsd GrmsJobDescriptionSchema.xsd] 91 91 92 92 Better flexibility and expected functionality is achieved in DCworms by introducing support of new swf header comments: … … 133 133 The main goal of workload design was to ensure, that all job descriptions which were used in real resource management system like GRMS or obtained from SWF/GWF log can be used to perform experiment in DCworms simulator. However, it my be difficult for all users to reach such workloads, therefore workload generator was created. 134 134 135 Workload generator allows you to create any number of jobs and tasks, with sophisticated resource and time requirements. The result of generation process are: desired number of job descriptions in xml format and swf file with job descriptions and all necessary header parameters (see [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd] for details).135 Workload generator allows you to create any number of jobs and tasks, with sophisticated resource and time requirements. The result of generation process are: desired number of job descriptions in xml format and swf file with job descriptions and all necessary header parameters (see [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd] for details). 136 136 137 137 Configuration options are provided by two files: 138 138 * *.properties file, which should provide values of all parameters from createscenario group and resdesc parameter (see [[#ConfigurationFile | configuration file]] for details) 139 * xml task configuration file, which is described by xsd schema [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd] and contains configuration of random numbers generators, used to create job/task/workload parameters.139 * xml task configuration file, which is described by xsd schema [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd] and contains configuration of random numbers generators, used to create job/task/workload parameters. 140 140 141 141 Details about properties file are described above. Following part of this section describes all elements of xml configuration file. … … 208 208 209 209 ==== '''!ComputingResourceHostParameter''' ==== 210 Defines generator which creates <hostParameter/> element in task resource requirements section. See [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/grms3/GrmsJobDescriptionSchema.xsd GrmsJobDescriptionSchema.xsd] for detail description of task resource requirements.210 Defines generator which creates <hostParameter/> element in task resource requirements section. See [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/grms3/QCGJobDescriptionSchemaNew.xsd GrmsJobDescriptionSchema.xsd] for detail description of task resource requirements. 211 211 212 212 This element requires attribute named: ''metric''. The value of this attribute will be passed to task <hostParameter/> element as a value of its ''name'' attribute. The possible values of <hostParameter/> name attribute and thereby <!ComputingResourceHostParameter/> metric attribute are: osname, ostype, puarch, osversion, osrelease, memory, freememory, cpucount, freecpus, cpuspeed, application, diskspace, freediskspace, remoteSubmissionInterface, localResourceManager, hostname. In simulation values of cpucount and memory host parameters are used by default. Others are currently ignored. … … 216 216 217 217 ==== '''Preferences''' ==== 218 This is the complex element, which was designed to describe section <preferences/> in task requirements. See [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/grms3/GrmsJobDescriptionSchema.xsd GrmsJobDescriptionSchema.xsd] for detail description of task resource requirements.218 This is the complex element, which was designed to describe section <preferences/> in task requirements. See [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/grms3/QCGJobDescriptionSchemaNew.xsd GrmsJobDescriptionSchema.xsd] for detail description of task resource requirements. 219 219 220 220 Element <preferences/> consist of list of <parameter/> elements. Each parameter must provide <name/>, <importance/>, <optimizationType/> and <value/> elements. <endpoint/> is optional. The values of these elements are passed to the attributes in parameter element in task resource requirements section. Name of the parameter elements and task parameter attributes are the same. … … 226 226 227 227 ==== '''!ExecutionTime''' ==== 228 This complex element was designed to describe <!ExecutionTime/> section in task description. See [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/grms3/GrmsJobDescriptionSchema.xsd GrmsJobDescriptionSchema.xsd] for details.228 This complex element was designed to describe <!ExecutionTime/> section in task description. See [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/grms3/QCGJobDescriptionSchemaNew.xsd GrmsJobDescriptionSchema.xsd] for details. 229 229 230 230 <!ExecutionTime/> consists of four child elements which have following interpretation: … … 593 593 [http://ant.apache.org/ Apache Ant] is required to compile DCworms and run an experiment. 594 594 595 * Prepare working directory (''working_dir'') in your local system and download DCworms sources from [https:// apps.man.poznan.pl/svn/gssim/DCworms/].595 * Prepare working directory (''working_dir'') in your local system and download DCworms sources from [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse]. 596 596 The following command can be used to checkout DCworms sources 597 597 {{{ 598 svn checkout https://apps.man.poznan.pl/svn/gssim/DCworms/ 598 git clone https://git.man.poznan.pl/stash/scm/worms/dcworms.git 599 599 }}} 600 600 … … 611 611 }}} 612 612 613 DCworms project can be also directly checkout (using SVN clients for Eclipse) or imported (using Import Project wizard) to theEclipse IDE.613 DCworms project can be also directly pulled(using GIT clients) and imported (using Import Project wizard) to the IntelliJ IDEA or Eclipse IDE. 614 614 If so, the simulation can be run using the "runner" classes in the example/experimentX ditectory -> simply right click on the !RunExperiment.java class and choose Run As -> Java Application 615 615 It is recommended to run the Java VM with the following arguments: -Xms512m -Xmx2048m … … 1510 1510 Proper execution of workload generator requires: 1511 1511 1512 * '''workload characteristic file''' - pointed by the createscenario.tasksdesc parameter [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd]1512 * '''workload characteristic file''' - pointed by the createscenario.tasksdesc parameter [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/WorkloadSchema3g.xsd WorkloadSchema.xsd] 1513 1513 * '''description of resources''' - resdesc - the same which will be used for experiment execution 1514 1514 … … 1962 1962 }}} 1963 1963 As we can see, it is necessary to specify resource description file. This file should be valid with 1964 [https:// apps.man.poznan.pl/svn/gssim/DCworms/trunk/simulator/schemas/resources/DCwormsResSchema.xsdDCwormsResSchema.xsd]. It should contain description of resources (e.g. number of processors, their speed, memory etc.).1964 [https://git.man.poznan.pl/stash/projects/WORMS/repos/dcworms/browse/src/main/resources/simulator/schemas/resources/DCwormsResSchema.xsd]. It should contain description of resources (e.g. number of processors, their speed, memory etc.). 1965 1965 Furthermore, workload file is necessary to obtain information concerned with execution of simulation (e.g. execution of tasks). This workload file should be in the standard workload format (http://www.cs.huji.ac.il/labs/parallel/workload/swf.html) 1966 1966