Changes between Version 2 and Version 3 of installingGridFTP

Timestamp:

09/23/13 14:45:36 (11 years ago)

Author:

mmamonski

Comment:

--

installingGridFTP

@@ -1 @@
-[[PageOutline]] 
-
-= GridFTP Installation =
-
-This page provides step-by-step guide for site admins. It describes basic steps to install and configure GridFTP service.
-For any issue concerning the guide please contact [mailto:bogdanl@man.poznan.pl Bogdan Ludwiczak].
-
-= Globus Toolkit/GridFTP - installation =
-
-This sections provides overview of Globus Toolkit ver. 4.x installation procedure.
-
-Globus Toolkit is provided both in binary and source versions. It is also available as Java WS-core only or full version. For the purpose of the !QosCosGrid Infrastructure the full source installation is required, as it provides the required GridFTP features and in most cases is more reliable (though more troublesome in rare cases) and robust then binary version.
-
-Below are the steps required to compile and install Globus Toolkit with only GridFTP daemon and utilities.
-
-'''Note:''' this is only an outline of the installation process which should be sufficient to install GridFTP service in most cases, full and detailed information regarding toolkit installation can be found in official Globus [http://www.globus.org/toolkit/docs/4.0/ documentation]. 
-
-* Check if target system has all required software components:
- *  gcc (latest 3.4 or 4.1)
- *  GNU tar, sed, make
- *  Perl >5.6
- * Create a dedicated, non-privileged user named `globus`. This user owns the installation/deployment files compile and install Globus Toolkit performs administrative tasks such as deploying/configuring services, etc.
- * Create deployment directory
- * make sure that user the `globus` has read and write permissions in the installation directory
-
-'''Note:''' Following steps should be carried out by the `globus` user.
-
-* Download and untar Globus Toolkit ver.4.2.x all-source-installer (available at the  [[http://www.globus.org/toolkit/downloads/4.2.1/#source Globus Toolkit]] web site)
-* change current working directory to the source directory (i.e. `~/gt4.2.x-all-source-installer`)
-* set environmental variable `GLOBUS_LOCATION` to point to installation directory (e.g. `/opt/globus/gt421`)
-* finally use `./configure` to set up compilation and installation option. Use at least `--prefix` option to set the deployment directory.
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-./configure --prefix=${GLOBUS_LOCATION}
-}}}
-}}}
- '''Note:''' If you encounter an error '''configure: error: Unable to compile with SSL''', replace the above line with:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-./configure --disable-system-openssl --prefix=${GLOBUS_LOCATION}
-}}}
-}}}
-* Compile and install Globus Toolkit by running:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-make gridftp 
-make install
-}}}
-}}}
- If you wish to have a log file of the build, use "tee" command:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-make gridftp 2>&1 | tee build.log
-make install
-}}}
-}}}
-* GT4 unlike other software packages is made directly to target location (`$GLOBUS_LOCATION`)
-* If you omit `gridftp` for `make` the entire Globus Toolkit will be build what can take several hours to complete.
-* during `make install` phase Globus configuration will be initialized
-* You can also choose to build GridFTP server or client tools be specifying `globus_gridftp_server` or `globus-data-management-client` respectively.
-
-The belowe steps completes Globus Toolkit/GridFTP installation. The following figure summarizes shell commands used during typical installation process:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-bogdanl@cress ~ $ su -
-cress ~ # useradd -m globus 
-cress ~ # mkdir /opt/globus 
-cress ~ # chown globus:users /opt/globus
-cress ~ # su - globus
-globus@cress ~ $ wget http://www-unix.globus.org/ftppub/gt4/4.2.1/installers/src/gt4.2.1-all-source-installer.tar.bz2
-globus@cress ~ $ tar xjf gt4.2.1-all-source-installer.tar.bz2
-globus@cress ~ $ export GLOBUS_LOCATION=/opt/globus/gt421
-globus@cress ~ $ cd gt4.2.1-all-source-installer
-globus@cress ~/gt4.2.1-all-source-installer $ ./configure --prefix=${GLOBUS_LOCATION} \
-globus@cress ~/gt4.2.1-all-source-installer $ make gridftp
-cd gpt && OBJECT_MODE=32 ./build_gpt
-...
-Your build completed successfully.  Please run make install.
-globus@cress ~/gt4.2.1-all-source-installer $ make install
-}}}
-}}}
-
-= Globus Toolkit/GridFTP - basic configuration =
-
-== Security ==
-After Globus Toolkit installation, but before starting Globus services several aspects of GSI security need to be configured:
-* configure Globus (and thus GridFTP daemon) to trust a particular set of CAs (Certificate Authorities), i.e. place certificates of trusted CAs into designated directory - CA is trusted only if its CA certificate exists with the appropriate name in an appropriate directory. Moreover, for pre-ws services (including GridFTP), signing policy file must exist in the same location as CA certificate. In other words, one needs two files to trust given CA:
- * `cert_hash.0` - the trusted CA certificate and
- * `cert_hash.signing_policy` - the signing policy.
- Globus services and tools looks for that directory in following locations:
- * the value of `$X509_CERT_DIR` environment variable if it is set and the directory exists,
- * otherwise, in `$HOME/.globus/certificates` if it exists,
- * otherwise, in `/etc/grid-security/certificates` if it exists,
- * otherwise, in `$GLOBUS_LOCATION/share/certificates` if it exists.
-
-'''Note:''' We suggest to use `/etc/grid-security/certificates` as system wide trusted CAs directory, remember that `$X509_CERT_DIR` and `$HOME/.globus/certificates` have higher priority. 
-
-The `cert_hash.0`, i.e. certificate of the CA, is provided by CA, usually with appropriate hash name. Hash name consists of 8 hex-digits and suffix ".0" (e.g. `8a661490.0`). Valid hash can be obtained with following command (available in `$GLOBUS_LOCATION/bin/`):
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-openssl x509 -hash -noout -in ca_certificate
-}}}
-}}}
-* cert_hash.signing_policy usually is also provided by CA, but it can be constructed manually. The signing policy file has the following format:
-{{{
-#!div style="font-size: 90%"
-{{{#!default
-access_id_CA X509 'CA Distinguished Name'
-pos_rights globus CA:sign
-cond_subjects globus '"Name Pattern1" "Name Pattern2" ...'
-}}}
-}}}
- to get 'CA Distinguished Name' execute:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-openssl x509 -subject -noout -in cert_hash.0
-}}}
-}}}
-* "Name patternX" is a string used to match the distinguished names of certificates granted by the given CA. Usually, it is a CA name with common name replaced by wild card '*', e.g.:
-{{{
-#!div style="font-size: 90%"
-{{{#!default
-"/C=PL/O=GRID/CN=Polish Grid CA" -> '"/C=PL/O=GRID/*"'
-}}}
-}}}
-  it accepts "C=PL/O=GRID/OU=PSNC/CN=Bogdan Ludwiczak", but it rejects "O=GRID/OU=PSNC/CN=Bogdan Ludwiczak". "*" pattern accepts all certificates.
-
-* Configure appropriate default values for use by the grid-cert-request command which is used to generate certificates requests. The following files have to be properly configured to enable Globus tools to generate valid certificate requests:
- * `/etc/grid-security/globus-user-ssl.conf` - defines the distinguished name to use for a user's certificate request.
- * `/etc/grid-security/globus-host-ssl.conf` - defines the distinguished name for a host and service certificate request.
- * `/etc/grid-security/grid-security.conf` - is a main configuration file that contains the name and email address for the given CA.
-
- These files are usually provided by the CA, i.e. [[http://www.man.poznan.pl/plgrid-ca/|PL-Grid]]. Typically, CA configuration files are placed in `/etc/grid-security/certificates/` directory with additional extension `.CA_hash_name` and only appropriate symbolic links are created in `/etc/grid-security/`. Globus Toolkit provides `grid-default-ca` command which can be used to automatically create appropriate links.
-
-== Requesting host and user X.509 certificates ==
-All Globus service (including GridFTP) require a host (or service) certificate to operate. Also every user needs a user certificate do use Globus services. You can use Globus command `grid-cert-request` to generate `host/users` certificate request which should be send to your CA to be signed. Make sure that CA configuration files and certificate are in place before generating requests. This command will create 3 files:
-* an empty (file length is 0) `/etc/grid-security/hostcert.pem` or `~/.globus/usercert.pem`,
-* `/etc/grid-security/hostkey.pem` or `~/.globus/userkey.pem` file containing `host/use`r private key which must be kept secret - make sure that the unix access mode is set to 0400 or 0600 at most,
-* `/etc/grid-security/hostcert_request.pem` or `~/.globus/usercert_request.pem` file containing acctual request to be send to and signed by CA.
-
-'''Note:''' Before you can use grid-cert-request command you have to source Globus configuration script appropriate for your shell, i.e. `$GLOBUS_LOCATION/etc/globus-user-env.csh` or `$GLOBUS_LOCATION/etc/globus-user-env.sh`
-
-Send newly generated certificate request (i.e. `/etc/grid-security/hostcert_request.pem` or `~/.globus/usercert_request.pem` file) to the appropriate CA and wait for the certificate which should be send in return by your CA. Save the new certificate in `hostcert.pem` or `usercert.pem`. Now, request file can be deleted. Host certificate and private key should be owned by root user.
-Specify identity mapping information. Globus services map distinguished names (retrieved from certificates) to local identities (unix account) by means of `grid-mapfile`. Mappings have the following form:
-`"Distinguished Name" local_name`
-(every line of the file defines one mapping). To let user in, create an account and add mapping to the site's grid-mapfile. Globus looks for the file in the following locations:
-* the value of `GRIDMAP` environment variable if it is set,
-* otherwise, if service is run as root then grid map file is `/etc/grid-security/grid-mapfile`,
-* otherwise, the grid map file is `$HOME/.gridmap`
-* otherwise, in `/etc/grid-security/grid-mapfile`.
-
-'''Note:''' In the !QosCosGrid there is a possibility to generate a new user X.509 certificate in more [[http://node2.qoscosgrid.man.poznan.pl:80/gridsphere/gridsphere/guest/security/r/|user-friendly way]]
-
-== Firewall configuration ==
-
-Detailed information on GT firewall issues can be found at [[http://www.globus.org/toolkit/security/firewalls/]]. This paragraph only lists the minimum required firewall configuration and gives a short overview of the basic issues. To enable remote access to Globus Toolkit services the following TCP port should be opened for the incoming connections:
-* static port: 2811 for main GridFTP daemon port
-* ephemeral TCP ports: various Globus services require an arbitrary chosen TCP port range. It is controllable by environmental variable `GLOBUS_TCP_PORT_RANGE` for pre-ws components. Make sure it is defined in daemons environment (i.e. put it in (x)inetd configuration entries).
-
-Figure below gives an overview of typical Globus/GridFTP communication flow It shows the meaning of the aforementioned Globus TCP port range.
-
-[[Image(Firewall_gridftp.jpg, center)]]
-
-== GridFTP Daemon ==
-After successful installation of !GlobusToolkit, the GridFTP daemon binary is located in: `$GLOBUS_LOCATION/sbin/globus-gridftp-server` file. The daemon could be run in standalone mode, but we suggest using (x)inetd mode. To configure it this way, first add the following entry to `/etc/services` file:
-{{{
-#!div style="font-size: 90%"
-{{{#!default
-gsiftp          2811/tcp               # GridFTP (Port 2811 is the IANA registered GridFTP port) 
-}}}
-}}}
-Next add appropriate entry to (x)inetd configuration file(s). Remember to set `LD_LIBRARY_PATH`, `GLOBUS_LOCATION` and `GLOBUS_TCP_PORT_RANGE` in the (x)inetd entry environment, like in the xinetd example below:
-
-{{{
-#!div style="font-size: 90%"
-{{{#!default
-service gsiftp
-{
-      instances            = 100
-      cps                  = 50 10
-      per_source           = 50
-      socket_type          = stream
-      wait                 = no
-      user                 = root
-      env                  = LD_LIBRARY_PATH=/opt/globus/lib
-      env                  += GLOBUS_LOCATION=/opt/globus
-      env                  += GLOBUS_TCP_PORT_RANGE=9000,9999
-      server               = /opt/globus/sbin/globus-gridftp-server
-      server_args          =  -i -c /opt/globus/etc/gridftp.conf
-      log_on_success       += DURATION
-      log_on_failure       +=
-      nice                 = 10
-      disable              = no
-}
-}}}
-}}}
-
-Note that the above example specifies grid-ftp configuration file (`-c .../gridftp.conf` in `server_args` line). It is not required - GridFTP daemon should work without this file. A useful example of configuration file is:
-
-{{{
-#!div style="font-size: 90%"
-{{{#!sh 
-log_level ALL (or 'EERROR', 'WARN', 'INFO', 'DUMP')
-log_module stdio
-log_single /var/log/gridftp
-}}}
-}}}
-
-it enables logging which can be helpful in resolving potential problems. Finally, restart (x)inetd to activate the GridFTP daemon:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-/etc/init.d/xinetd restart
-}}}
-}}}
-
-== Users configuration ==
-All Globus Toolkit commands require appropriately configured environment. Globus Toolkit provides scripts for that purpose. To configure users environment for GT command do the following:
-for bash shell:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-export GLOBUS_LOCATION=/opt/globus/gt421
-. $GLOBUS_LOCATION/etc/globus-user-env.sh 
-}}}
-}}}
-
-or for csh shell:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-  setenv GLOBUS_LOCATION /opt/globus/gt421
-  source $GLOBUS_LOCATION/etc/globus-user-env.csh 
-}}}
-}}}
-
-Next you must obtain user certificates (as described in [[#RequestinghostanduserX.509certificates|Requesting host and user X.509 certificates]]) and then create user proxy by running command `grid-proxy-init`:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-grid-proxy-init 
-Your identity: /C=PL/O=GRID/O=PSNC/CN=Bogdan Ludwiczak
-Enter GRID pass phrase for this identity:
-Creating proxy ....................................................... Done
-Your proxy is valid until: Tue Apr 29 21:02:33 2008
-}}}
-}}}
-
-Next, become root and create a mapping in `/etc/grid-security/grid-mapfile`. You can use your favorite text editor for that purpose and append a line similar to this one:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-"/C=PL/O=GRID/O=PSNC/CN=Bogdan Ludwiczak" bogdanl
-}}}
-}}}
-or use GT `grid-mapfile-add-entry` command:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-grid-mapfile-add-entry -dn "/C=PL/O=GRID/O=PSNC/CN=Bogdan Ludwiczak" -ln bogdanl
-}}}
-}}}
-
-'''Note:''' "bogdanl" in examples above is a name of a local unix account.
-
-Now you can transfer some files with globus-url-copy tool:
-
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-globus-url-copy gsiftp://node1.qoscosgrid.man.poznan.pl/~/file file:///tmp/file.test
-globus-url-copy gsiftp://node1.qoscosgrid.man.poznan.pl/~/file gsiftp://node1.qoscosgrid.man.poznan.pl/~/file2
-globus-url-copy gsiftp://node1.qoscosgrid.man.poznan.pl/~/file gsiftp://node2.qoscosgrid.man.poznan.pl/~/file2
-}}}
-}}}
-
-= Quick Start =
-* create `globus` user and installation directory, make `globus` user an owner of this directory:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh  
-bogdanl@cress ~ $ su -
-cress ~ # useradd -m globus
-cress ~ # mkdir /opt/globus
-cress ~ # chown globus:users /opt/globus 
-}}}
-}}}
-
-* login as a `globus` user:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-cress ~ # su - globus
-}}}
-}}}
-
-* download Globus Toolkit source installer and untar it:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-globus@cress ~ $ wget http://www-unix.globus.org/ftppub/gt4/4.2.1/installers/src/gt4.2.1-all-source-installer.tar.bz2
-globus@cress ~ $ tar xjf gt4.2.1-all-source-installer.tar.bz2 
-}}}
-}}}
-* Configure and make sources:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-globus@cress ~ $ cd gt4.2.1-all-source-installer
-globus@cress ~ $ export GLOBUS_LOCATION=/opt/globus/gt421
-globus@cress ~/gt4.2.1-all-source-installer $ ./configure --prefix=${GLOBUS_LOCATION} \
-globus@cress ~/gt4.2.1-all-source-installer $ make gridftp
-cd gpt && OBJECT_MODE=32 ./build_gpt
-...
-Your build completed successfully.  Please run make install.
-globus@cress ~/gt4.2.1-all-source-installer $ make install
-}}}
-}}}
-* Request host certificates.
-* Start gridftp-server either standalone or by (x)inetd daemon
- * If you are going to use xinetd daemon add gridftp to `/etc/services`:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-bogdanl@cress ~ $ su -
-cress ~ # echo "gsiftp          2811/tcp               # GridFTP" >> /etc/services
-}}}
-}}}
- * and create new xinetd configuration file for gridftp service
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-cress ~ # cat /etc/xinetd.d/gridftp
-service gsiftp
-{
-      instances            = 100
-      cps                  = 50 10
-      per_source           = 50
-      socket_type          = stream
-      wait                 = no
-      user                 = root
-      env                  = LD_LIBRARY_PATH=/opt/globus/lib
-      env                  += GLOBUS_LOCATION=/opt/globus
-      env                  += GLOBUS_TCP_PORT_RANGE=9000,9999
-      server               = /opt/globus/sbin/globus-gridftp-server
-      server_args          =  -i -c /opt/globus/etc/gridftp.conf
-      log_on_success       += DURATION
-      log_on_failure       +=
-      nice                 = 10
-      disable              = no
-}
-}}}
-}}} 
- * restart xinetd:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-cress ~ # /etc/init.d/xinetd restart
-}}}
-}}}
-* test it:
-{{{
-#!div style="font-size: 90%"
-{{{#!sh
-bogdanl@cress ~ $ telnet cress 2811
-Trying 150.254.149.134...
-Connected to cress.man.poznan.pl.
-Escape character is '^]'.
-220 cress.man.poznan.pl GridFTP Server 2.3 (gcc32dbg, 1144436882-63) ready.
-bogdanl@cress ~ $ export GLOBUS_LOCATION=/opt/globus/gt421
-bogdanl@cress ~ $ . $GLOBUS_LOCATION/etc/globus-user-env.sh 
-bogdanl@cress ~ $ grid-proxy-init
-bogdanl@cress ~ $ globus-url-copy gsiftp://cress/~/file \
-      file:///tmp/file.test
-bogdanl@cress ~ $ globus-url-copy gsiftp://cress/~/file \
-      gsiftp://cress/~/file2
-}}}
-}}}
-
-= Tuning =
-Please read this page [[GridFTPTuning|GridFTP tuning]].
+Installation using packages: [http://apps.man.poznan.pl/trac/qcg/wiki/gridftp%20server |link]