NCCS | User Info | search  

The HPSS htar Utility

HTAR (HPSS Tape Archiver) is a file bundling and storage utility that was developed at LLNL. It creates and/or manipulates HPSS-resident archive files. The htar utility writes the archive directly to HPSS. The syntax of the htar utility is explained in the htar man page which can be accessed on any HPSS computer by typing man htar or online here.

Why should I use htar?

• htar will bundle many small files together for more efficient handling by HPSS
  
  
• the archive file will be created in memory so that more disk space is not used
  
  
• the archive file created by htar will be automatically transferred to HPSS eliminating an extra step
  
  
• htar creates an index file automatically so that you can easily see a listing of what files are contained in the archive
  
  
• individual files can be retrieved from the archive eliminating the need to retrieve the whole tar archive file
  
  
• htar uses as many parallel interfaces as there are available on the machine to do the transfer using multiple threads thus decreasing transfer time
  
  
• htar can be used to build an index file for tar created archive files
  
  
• htar can be used to rebuild an index file that has been accidentally deleted
  
  
• htar automatically creates a consistency check file to maintain consistency between the archive and its index
  
  

Examples of htar usage

1. Creating an htar archive file in the HPSS home directory:
   
   htar -cvf projfiles.tar projdir
   
   All of the files in the projdir directory will be added to the projfiles.tar archive file
   
   
2. Creating an htar archive file in a subdirectory of the HPSS home directory:
   
   htar -cvf january/readings.tar jan*
   
   All of the files that begin with the letters jan will be added to the readings.tar archive file. This file will be created in the directory, january, which is a subdirectory of your HPSS home directory. Note: the subdirectory will NOT be created by htar, it must already exist.
   
   
3. Looking at the contents of an htar archive:
   
   htar -tvf january/readings.tar
   
   Note: a listing will show a file at the end that looks like:
   
   /tmp/HTAR_CF_CHK_26452_1016481493
   
   This file is used for consistency checking only. It will not be retrieved unless you specify it explicitly. Users would not normally need this file.
   
   
4. Retrieving files from within an htar archive file:
   
   htar -xvf january/readings.tar jan2 jan5
   
   would retrieve the files jan2 and jan5 from the archive file, readings.tar, created in the previous example. This feature eliminates the need to retrieve the entire archive if only a few files are needed. The file names must be passed as arguments to htar. The current version of htar does not handle wildcard characters.
   
   
5. Using htar to rebuild a deleted index file:
   
   When trying to retrieve an archive file, you may get the message
   No such file: january/readings.tar.idx
   ### WARNING htar returned non-zero exit STATUS
   
   The index file for the archive has accidentally been deleted. It can be recreated using htar with the -X (capital letter) option.
   
   htar -Xvf january/readings.tar
   
   This command will recreate the index file and including the "v" option will also give a listing of the files contained in the archive during the build. This same option can also be used to build an index for an archive file that was created with tar rather than with htar. After the index file is built, htar will treat the archive as if it had been created with htar.
   
   
6. Archiving a very large number of files:
   
   The UNIX shells all have a limitation on the length of a command line. To archive a large number of files without exceeding this limit, you can either use a directory or use the find command.
   
   If you store all of the files that you want to archive in a single directory, then you can use htar with the directory name. This solution is the least resource-intensive solution. If you failed to plan ahead by using a directory, then you can use the find command to create a file containing a list of the filenames that you want included in the archive file.
   
   find . -name 'jan*' -print > janfiles
   
   janfiles will contain a list of all files that begin with the letters jan. Then you can use the -L option with htar to build your archive file in HPSS.
   
   htar -cvf janfiles.tar -L janfiles