[[PageOutline]]

= QCG Broker Installation =

The !QosCosGrid Resource Management System (QCG-Broker) is an open source meta-scheduling system, which allows developers to build and deploy resource management systems for large scale distributed computing infrastructures. The QCG-Broker, based on dynamic resource selection, mapping and advanced scheduling methodology, combined with feedback control architecture, deals with dynamic Grid environment and resource management challenges, e.g. load-balancing among clusters, remote job control or file staging support. Therefore, the main goal of the QCG-Broker is to manage the whole process of remote job submission to various batch queuing systems, clusters or resources. It has been designed as an independent core component for resource management processes which can take advantage of various low-level Core Services and existing technologies. Finally, the QCG-Broker can be considered as a robust system which provides abstraction of the complex grid infrastructure as well as a toolbox which helps to form and adapts to distributing computing environments.

= Requirements =

* '''Package dependencies''' - All packages needed to install and run QCG-Broker service are installed automatically by the YUM Package Manager.
* '''certificates''' - set of CA certificates has to be installed to allow proper mutual authentication between client and service.
* '''range of opened ports''' - QCG-Broker needs set of opened ports to be able to receive notifications about changes of jobs and tasks statuses and to communicate with gridFTP server.

= Preparation of the environment =

== Installation of CA certificates ==
To allow mutual authentication between service and clients please install the set of certificates of trusted Certification Authorities.
If the system does not have CA certificates already installed you can:
* install them globally following the [[https://www.qoscosgrid.org/trac/qcg/wiki/CA%20certificates | CA certificates installation]] guide
* configure QCG-Broker to use its own list of trusted CA certificates copping selected ones into the `~/.globus/certificates` directory.

== Software repositories ==
To install QCG-Broker service please configure the system with [[http://www.qoscosgrid.org/trac/qcg/wiki/Software%20repositories | QCG Software Repositories]].

= Installation =

'''IMPORTANT:''' Perform the installation as the administrator (`root` user).

'''IMPORTANT:''' A new account (`qcg-broker`) will be created during the installation.

* install QCG-Broker service and all dependencies using YUM Package Manager:
{{{
#!div style="font-size: 90%"
{{{#!sh
yum install qcg-broker
}}}
}}}

= Configuration =

== Database setup ==
* If PostgreSQL service has not been initialized (e.g. has been installed as dependency of qcg-broker package), perform initialization before service starting:
{{{
#!div style="font-size: 90%"
{{{#!sh
su -
service postgresql initdb
}}}
}}}

* Start the PostgreSQL service:
{{{
#!div style="font-size: 90%"
For SL5.x
{{{#!sh
su -
service postgresql start
}}}
}}}

* Create new database user (e.g. `qcg-broker`) authenticated via password (the same password must be later used to setup QCG-Broker deployment):
{{{
#!div style="font-size: 90%"
{{{#!sh
su - postgres
createuser -P qcg-broker

Enter password for new role: 
Enter it again: 
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n
CREATE ROLE

exit
}}}
}}}

* Create new database (e.g. qcg-broker) owned by the `qcg-broker` user:
{{{
#!div style="font-size: 90%"
{{{#!sh
su - qcg-broker
createdb -U qcg-broker qcg-broker
exit
}}}
}}}
 '''Note:''' The `qcg-broker` user must be allowed to create new databases.

* Depending on the local PostgreSQL configuration you may need to edit the `pg_hba.conf` file  (''host based authentication'' configuration file) to enable password authentication to the `qcg-broker` database for the `qcg-broker` user.

 For SL5.x: edit the `/var/lib/pgsql/data/pg_hba.conf` file.

 If the PostgreSQL installation is fully dedicated for QCG-Broker replace the default configuration with rules (lines) given below, otherwise insert them to the configuration file in a proper place:
{{{
#!div style="font-size: 90%"
{{{#!default
local   qcg-broker   qcg-broker                         md5
host    qcg-broker   qcg-broker   127.0.0.1/32          md5
}}}
}}}

 '''Note:''' Please reload the PostgreSQL server in order to make the changes visible e.g:
{{{
#!div style="font-size: 90%"
{{{#!sh
service postgresql reload
}}}
}}}


== QCG Broker setup ==

'''Note:''' All changes should be made as 'qcg-broker' user:
{{{
#!div style="font-size: 90%"
{{{#!sh
su - qcg-broker
}}}
}}}

* configure the QCG-Broker instance - all configuration variables are placed in the `/opt/plgrid/qcg/share/qcg-broker/deploy.prop` file
 * '''deploy.dir''' - directory where QCG-Broker was installed (`/opt/plgrid/qcg/share/qcg-broker`). Please do not modify this property.
 * '''deploy.certs.dir''' - directory with QCG-Broker credentials (must contain `qcg-brokercert.pem` and `qcg-brokerkey.pem` files)
 * '''deploy.cert.dn''' distinguish name of QCG-Broker credential
 * '''deploy.db.user''' PostgreSQL user name
 * '''deploy.db.pwd''' Password for PostgreSQL user
 * '''deploy.db.host''' PostgreSQL database host name (`127.0.0.1` by default. If PostgreSQL service is on the same host as QCG-Broker, this property should not be changed. Otherwise some changes in PostgreSQL configuration (`pg_hba.conf` file) may be needed).
 * '''deploy.db.name''' PostgreSQL database name
 * '''deploy.https_port''' the port which QCG-Broker will be listening for https requests on
 * '''deploy.rmi_port''' the port needed for internal communication between QCG-Broker components.
 * '''deploy.shutdown_port''' the port which the tomcat will be listening for shutdown requests on
 * '''deploy.port_range''' the range of opened ports

* deploy the configuration and setup the QCG-Broker database
{{{
#!div style="font-size: 90%"
{{{#!sh
/opt/plgrid/qcg/share/qcg-broker/config.sh
}}}
}}}


== QCG Broker resource providers setup ==
'''Note:''' All changes should be made as 'qcg-broker' user:
{{{
#!div style="font-size: 90%"
{{{#!sh
su - qcg-broker
}}}
}}}

* Edit the `/opt/plgrid/qcg/share/qcg-broker/broker/etc/providers.list` file specifying list of resource providers (instances of QCG BES/AR services controlling resources and creating together virtual pool of resources)

 '''providers.count'''=<n> - number of providers

 For x=0 to n-1 define locations of domain providers:

 '''provider.x.domain'''=<DOMAIN> - name of the administrative domain, which resources controlled by the given provider belong to,\\
 '''provider.x.type'''=qcg - type of the provider, DO NOT CHANGE THIS VALUE,\\
 '''provider.x.id'''=<PROVIDER ID> - identifier of the provider (must be globally unique),\\
 '''provider.x.location'''=<LOCATION>  - address of provider service.\\
 '''provider.x.storage'''=<GRIDFTP>  - address of the gridFTP server\\
 '''provider.x.hostname'''=<LIST OF NAMES>  - list of names of the site separated by comma\\
 '''provider.x.props'''=<PROPERTIES>  - list of properties separated by comma. For the PLGrid sites please set the "plgrid" property\\
 '''provider.x.active'''=<STATE>  - optional parameter (true|false) determining the availability of the provider

 For example:
{{{
#!div style="font-size: 90%"
{{{#!sh
# how many providers should be used for testing
providers.count=1

provider.0.domain=PCSS
provider.0.id=reef
provider.0.type=qcg
provider.0.location=httpg://qcg.reef.man.poznan.pl:19000
provider.0.storage=qcg.reef.man.poznan.pl
provider.0.hostname=reef.man.poznan.pl,reef,pcss
provider.0.props=plgrid
provider.0.active=true

}}}
}}}

* For cross-clusters job submission, edit the `/opt/plgrid/qcg/share/qcg-broker/broker/etc/config.prop` file, and set following variables:
 * '''broker.qoscos.proactive.pncURL''' - !ProActvie node coordinator address
 * '''broker.qoscos.ompi.pncURL''' - OpenMPI node coordinator address

 See [http://www.qoscosgrid.org/trac/qcg/wiki/installation_QCG_ProActive_Coordinator QCG-ProActive Coordinator], [http://www.qoscosgrid.org/trac/qcg/wiki/installation_QCG_OpenMPI_Coordinator QCG-OpenMPI Coordinator] for detailed information about installation of !ProActive and OpenMPI node coordinators.


== QCG Broker advance reservations setup (optional) ==

'''Note:''' All changes should be made as 'qcg-broker' user:
{{{
#!div style="font-size: 90%"
{{{#!sh
su - qcg-broker
}}}
}}}

To allow QCG-Broker  to create and manage advance reservations on resources accessible via [http://apps.man.poznan.pl/trac/qcg-computing QCG BES/AR] service some additional configuration steps are needed on Administrative Domain level: 
* the QCG-Broker credential must be mapped (in the `grid-mapfile` file) to the local user that is authorized to create and manage reservations. Usually the LRMS administrator user (like `sgeadmin` in SGE or `lsfadmin` in LSF) has sufficient rights, e.g.:
{{{
#!div style="font-size: 90%"
{{{#!default
"/C=PL/O=GRID/O=PSNC/CN=qcg-broker/qcg.man.poznan.pl" sgeadmin
}}}
}}}


== QCG Broker notifications setup (optional) ==

'''Note:''' All changes should be made as 'qcg-broker' user:
{{{
#!div style="font-size: 90%"
{{{#!sh
su - qcg-broker
}}}
}}}

To allow QCG-Broker to send notifications about user's task status change, edit `/opt/plgrid/qcg/share/qcg-broker/broker/etc/config.prop`, and set variable:
{{{
#!div style="font-size: 90%"
{{{#!sh
broker.broker.notificationbroker.url
}}}
}}}
to point to QCG-Notification service.

 See [http://www.qoscosgrid.org/trac/qcg-notification#Installation] for detailed information about installation of QCG-Notification service.


= Starting the service =
As the `qcg-broker` user type:
{{{
#!div style="font-size: 90%"
{{{#!sh
cd /opt/plgrid/qcg/share/qcg-broker
export JAVA_HOME=/usr/lib/jvm/java
./broker/bin/start-broker.sh
./service/bin/startup.sh
}}}
}}}

'''IMPORTANT:''' Do not forget to set `JAVA_HOME` or `JRE_HOME` environment variables.
= Stopping the service =
As the `qcg-broker` user type:
{{{
#!div style="font-size: 90%"
{{{#!sh
cd /opt/plgrid/qcg/share/qcg-broker
export JAVA_HOME=/usr/lib/jvm/java
./broker/bin/stop-broker.sh
./service/bin/shutdown.sh
}}}
}}}
= Veryfing the installation =
To verify the installation please use [[client_user_guide|QCG-Broker Client User Guide]].