HSI Manual: Chapter 7

Chapter 7: Working With Multiple HPSS Systems

NOTE: Information in this section applies to
HSI Version 2.6 and later. Use the version
command to see which level of HSI you are using.

This section contains information on the following topics:

Using HSI with multiple HPSS systems that are linked together via Junctions and Cross Cell authentication
Using HSI to access multiple HPSS systems that have not established Cross Cell trust
Opening connections to more than one HPSS system during a single HSI session
Description of the Logical Drive pathname syntax that HSI uses to associate files with a connection during a session
List of commands that are used to open, close, and switch between active sessions
Examples of storing, retrieving, and copying files while connected to multiple HPSS systems.

Using HSI with multiple HPSS systems that are linked together using Junctions and Cross Cell authentication

Beginning with release 4.1.1 of HPSS, it has been possible for cooperating sites to use the Distributed Computing Environment cross-cell trust facility to allow users from one HPSS site to be authenticated at another HPSS site, using credentials they obtained at the first HPSS site. Once this has been established, the HPSS name spaces can be linked together using a feature of HPSS called junctions, which act like symbolic links to tie the name spaces together. This is accomplished by the HPSS and/or DCE administrators, and is normally transparent to HSI. When working with systems that are linked together using junctions, the entire namespace appears as a single integrated HPSS system to HSI.

In the diagram below, a junction called /pub/sysb is used to link HPSS System 1 to HPSS System 2, such that a reference on HPSS System 1 to the file /pub/sysb/work/myproject will result in a reference on HPSS System 2 to the file /share/sys1/work/myproject.

Using HSI to Access Multiple HPSS Systems That Have Not Established Cross-Cell Trust

HSI version 2.6 has added the capability of establishing concurrent connections to more than one HPSS system at more than one site within a single session, without requiring any cross-cell trust relationship to be established between the sites; the only requirements are that you have an account on each of the HPSS systems that you wish to access, and that there is network connectivity from the client system on which HSI is executing, and the HPSS site(s) that you wish to work with.

When working with multiple HPSS systems, HSI treats each connection as a "logical drive", borrowing from the familiar legacy of the early personal computers. In this environment, there are three key concepts:

The notion of a current connection.
This is initially established when the first connection is opened as HSI begins executing. It can change during execution as connections to other HPSS systems are opened, and as you switch between logical drives. (See below) Each connection has its own separate context, containing items such as:
       - The address of the HPSS server host
       - The logical drive letter that you (or HSI) assigned for the connection
       - Your home current directory
       - The current working directory
       - The classes of service that are available on the HPSS system
The use of logical drive notation
as a command prefix, and to reference files and directories on a particular HPSS system. Logical drive notation simply consists of a non-case-sensitive drive letter followed by the colon (:) character (with no intervening whitespace), for example:
       D:
followed by the command or pathname, with no intervening spaces.
For example,
       d:ls would switch to connection D: and list the files in the current working directory.

       Ls C:*.c would list all of the files ending with the characters ".c" on the HPSS system associated with logical drive C:.

       get fromHPSS1 : D:file1 fromHPSS2 : F:file2 would fetch the file file1 from the HPSS system associated with drive D:, renaming it to the local file "fromHPSS1", and similarly would fetch the file file2 from the HPSS associated with drive F:, renaming it locally to ":fromHPSS2".

Note that as of HSI version 2.4,the colon character (:) used to separate the local and HPSS pathnames, must be surrounded by whitespace, unlike earlier versions of HSI. This requirement was added in order to avoid ambiguity in the inclusion of colon characters in pathnames (see the Release Notes for more information) . You can tell which HSI version you are using by typing
       version

Drive letters are not case sensitive, so, the following are equivalent:
- f:get myfile
- F:get myfile
It is not necessary to use this notation; commands and pathnames without a drive prefix always reference the current connection. If the multi-HPSS feature of HSI is not used, commands and pathnames can be used just as in earlier versions of HSI.
Depending upon the context, the use of a standalone drive letter
e.g. b: or Ls C:
may be interpreted as either of the following:
- Set the current connection to the HPSS system associated with the logical drive (first example, above) or
- Reference the current working directory for the HPSS system associated with the logical drive (second example, above).
In commands where a filename is required, the use of a standalone drive letter will cause a usage error, for example:

get P:
*** `get': missing path for P:
Third-party copies.
When copying files between HPSS systems, the default is to use a 3rd-party copy mechanism so that the data flows directly between HPSS systems, without having HSI in the middle of the transfer. See the CP command in Chapter 8 for more details. For some situations, such as copying files through a firewall, it may be necessary to use a store-and-forward mechanism to read data from either a local file or from an HPSS system inside the firewall, and then transfer it to an HPSS system outside the firewall. See the firewall command for more information on setting up store-and-forward transfers.

Commands used for Multi-HPSS HSI Sessions

Commands used to access multiple HPSS systems are shown in the table below, and are also described in more detail in Chapter 8. The minimum abbreviation for each command is shown in the text preceding the "[" character.

Command	Description
open or connect	Establishes a new connection to an HPSS system.
close	Closes an existing connection to an HPSS system.
lscon[nections] or showc[on]	Shows a list of currently established connections, including the home directory and the current working directory for the connection
lssi[tes]>command	Displays a list of site names that have been defined in the global .hsirc startup file, or in your private .hsirc file.
drive:	As a standalone command, this changes the current active connection.
Drive:command	Changes the current active connection, then issues the specified command in the context of the new connection.
setdrive	Changes the logical drive letter assigned to a connection

Example Scenario for A Multi-HPSS Session

A typical interactive scenario for using HSI to communicate with multiple HPSS systems is described below.

Set the prompt string so that we can tell which logical drive is our current active connection, and what our current working directory is within that connection.

We want our prompt string to look like this:

drive_letter:[hostname]working_directory->

To accomplish this, the PS1 variable can be set in the hsirc file, or interactively as follows:

>PS1 = "%d[%H]%w3->"

which will yield a prompt sting similar to this:

O:[hpss]/home/gleicher->

The %w3 will cause long working directory pathnames to be displayed by truncating to at most 3 subdirectories on the right-hand part of the prompt string, and replacing characters in the middle of the pathname with the string "...", e.g.

      
      O:[hpss]/home.../tools/scripts/backman->pwd
      pwd0: /home/gleicher/HPSS/tools/scripts/backman

Establish initial connection.

If the site administrator has set up the global hsirc file, or you have customized your private .hsirc file in your home directory, then all that is required is to startup HSI with no options (to connect to the site-defined default HPSS system), or
hsi -s sitename
to connect to the site called sitename. Note that this is a case-sensitive option, which must exactly match the sitename contained in the .hsirc stanza.

If the -site option is not used on the command line, the following optional parameters may be used to connect to any HPSS system that you have an account on:
hsi -h hostname -p port -l login_name -A authentication method -k keytab filename

The format of the login name varies, depending on the authentication method that you have chosen. For the DCE-combo method (-A dce)and the keytab method (-A keytab), the login name is your DCE principal name on the HPSS system to which you are connecting. At most sites, this will be the same as your login name on any other machines at that site, but check with the HPSS administrator at the site to make sure.

For kerberos authentication (-A kerberos),the login name is your principal name and the Kerberos Realm name, which is generally in uppercase, e.g.:
-A kerberos -l betty@SDSC.EDU

(See chapter 2, "Working with HPSS", subsection 2.3 "HSI Startup File" for more information about the contents of the HSIRC file)
Normally, the site administrator will set up an HSI wrapper script containing all the information regarding hostnames, ports, and authentication mechanism(s) to be used, so that the optional parameters shown below are usually not needed.

Establish connection to another site

As above, if the site administrator has set up a global hsirc file, or if you have customized your private .hsirc file to add the appropriate information, the command:
       open -s new_sitename
       or
       open -d drive_letter: can be used to establish a connection to the HPSS system designated by new_sitename. or drive_letter:.

The lssites command can be useful when running interactively, if you need help remembering the exact sitename(s) and/or drive letter(s) that have been set up by the HPSS administrators. For example, at ORNL the following sites are defined:

         O:[hpss]/home/gleicher->lssites
         Site, Logical Drive....Probe, "P:"
         Site, Logical Drive....ORNL, "O:"
         Site, Logical Drive....NERSC, "N:"

        O:[hpss]/home/gleicher->open -d N: -A dce
        Password: 
        N:[NERSC]/home/gleicher->

Display the list of active sites after opening the second connection:

       N:[NERSC]/home/gleicher/HPSS/tools/getpath->lscon
       List of Currently Active Connections
       Current default connection handle: 2
              Han-    Remote          Remote            HSM  Fire- I/O    Ctl   Data Port  Cmd
       Drive  dle    IP Address       Hostname         Level wall  Mode   Port   Min  Max Count
       ----------------------------------------------------------------------------------------
       O:   1 160.91.199.58  arm16.ccs.ornl.gov   4.1.4  off normal  1217     0 65535 1
                HomeDir: /home/gleicher
                pwd0: /home/gleicher/HPSS/tools/scripts/backman

     ->N:   2 128.55.128.138  heart-g0.nersc.gov  4.3.0  off normal  7597     0   0 1
                HomeDir: /nersc/ccc/gleicher
                pwd0: /nersc/ccc/gleicher

At this point, standard HSI commands can be used to list directories, store, retrieve and copy files using the logical drive pathname syntax, etc.

For this example, we want to store a copy of the local file "good_stuff" on one of the HPSS systems, and then replicate it on the other, in order to provide disaster recovery. We will do this by using the cp command, so that the data flows directly between the two HPSS systems, without having HSI in the middle of the transfer.

First, store the file on logical drive O: in the directory temp, which HPSS systems:

     O:[hpss]/home/gleicher->put good_stuff : o:~/temp/good_stuff
     put  good_stuff : /home/gleicher/temp/good_stuff ( 1083916 bytes, 397.5 KBS (cos=6))

Next, copy the file from the O: drive to the N: drive:

     N:[NERSC]/home/gleicher->cp O:~/temp/good_stuff N:~/temp/good_stuff
     cp  O:/home/gleicher/temp/good_stuff to N:/home/gleicher/temp/good_stuff (2001/09/16 23:36:18 1083916 bytes, 481.1 KBS (cos=12)

At this point, we are ready to close out the connection(s) and terminate the session(s). This can be done by simply exiting out of HSI. If we want to continue to work in one of the connections, but have no need to keep the connection open for the the other, then the close command can be used, e.g.:
N:[NERSC]/home/gleicher->close N:
O:[hpss]/home/gleicher/temp->

Notice that when the current active connection is closed, HSI switches to one of the other active connections, in this case, the only other active connection was drive O:. If an attempt is made to close this connection, the following error message will be displayed:

     O:[hpss]/home/gleicher/temp->close o:
     *** Cannot use close for the only open connection.
         Use exit (or one of its aliases) to terminate HSI

When HSI execution has terminated, a logfile containing a time-stamped record of all commands will be present in the hsilog file, which is normally written in the home directory. This can be controlled by setting the HSI_LOGFILE environment variable as follows:

If using CSH:
       setenv HSI_LOGFILE somepath
If using KSH
       export HSI_LOGFILE=somepath
If using SH (this will also work for KSH and BASH)
       export HSI_LOGFILE HSI_LOGFILE=somepath

somepath can be the reserved string none to disable automatic creation of a logifle. It can also contain metacharacters of the form %X as follows:

%H : is expanded to your local home directory pathname
%N : is expanded to the non-canonical local hostname, i.e., the hostname without any domain name. " snuffles.sdsc.edu" would become "snuffles".
%P : is expanded to the current HSI process ID
%U : is expanded to the local login name.

The default setting is usually determined by the HPSS administrator, and set in the HSI wrapper script. If the path already exists when HSI attempts to open the local logfile, HSI will try to append to it.

[ PAGE TOP ]