Chapter 1: HPSS and HSI Introduction 1.1 What is HPSS and Who Can Use It?

Storage in the HPSS system is defined as one or more dynamic hierarchies, consisting of various combinations of low, medium and high speed devices.

Files are first stored on the highest level of the hierarchy; after they have existed on disk for some period of time, or upon user command, files will be migrated to the next layer of storage in the hierarchy.

After files have migrated to a lower layer, they are eligible to be purged (removed) from higher layers as needed in order to make space available for creation and manipulation of files; when this occurs, the files no longer exist on the hierarchy from which they were purged. When files are accessed, if they are not already on the disk cache, they are cached (copied) from the highest level at which a copy resides directly back to the disk cache layer, where they can then be transferred to the requesting host

It is important to understand that migration and purging are two completely different operations - migration essentially makes a copy of a file (so there may be multiple copies of a file within a storage hierarchy at any point in time), and purging removes a file from a level. Once files have been purged from a higher level, they exist only in lower levels to which they have previously migrated.

HPSS transparently manages the storage hierarchies, migrating, purging and caching files as required based upon the dynamically changing environment. In general, no knowledge of the hierarchy by the user is necessary; however, a set of commands is available to control certain aspects of the operation, such as selection of the hierarchy to be used, and the capability to force migration, purging, or staging of selected files.

All storage requests for HPSS storage are honored because there are (currently) no file space restrictions. With HPSS, files can be

• stored, retrieved, copied, moved, replaced or deleted;
• organized in simple or complex tree structures; and
• shared with selected individuals or with the entire user community.

The HSI utility stores and retrieves complete files; access to partial files is supported by HPSS, but currently not by HSI. HPSS does not interpret or change the format of any file; files are stored and retrieved as unchanged byte streams. If you are accessing HPSS via FTP, you can use ASCII translation mode when you save and retrieve files.

Files are not archived or backed up offsite. It is your responsibility to back up critical files!

Logically, HPSS consists of the following distinct entities:

• the portion that resides on the local computer system, with which you interface interactively or via shell scripts (the HPSS Interface utility program, HSI), and
• the main HPSS facility that resides on a dedicated computer that executes your instructions and accesses the files. This software is responsible for interfacing with the hardware and for controlling the storage hierarchies.

Throughout this document, the interface portion of HPSS on your local computer is referred to as HSI, and the file storage portion on the main facility is referred to as HPSS.