CCS:
Storage Policy
Hardware and software support for the CCS and ORNL production mass
storage systems is
provided by the Storage
Systems group of the Center for Computational Sciences.
Types of storage
We have three file systems available to CCS users: home directories,
system global temporary directories, and HPSS directories.
- Home directories are provided for each user in a DFS
(Distributed File System) which is mounted on each CCS system.
This is online disk intended for permanent, frequently used
storage and is backed up on a daily basis. This file system
does not generally provide the I/O performance required by
most jobs. The system global area is recommended for all
I/O intensive work.
- Separate system global temporary directories are provided
on each system. The path to these directories is different
on each system, but the environment variable $SYSTEM_USERDIR.
points to this directory. This is a large fast disk area
intended for global access to temporary storage, for example, to hold
output generated by your job. It is not currently backed up
and old files are automatically deleted every night. Any
file older than 7 days is subject to deletion.
Our intention is to maintain these file systems so there is
always a large amount of scratch space available for executing
jobs. Users should move files which are not being actively
used to HPSS.
- HPSS is the archival storage system. It consists of four
terabytes (TB) of disk cache and
some 240 TB of user data stored in robotic tape storage libraries.
HPSS may be accessed by HSI (hsi),
parallel ftp, or ftp.
Projects should not duplicate executables or datasets in each
user's area but should set privileges to share a master copy. Contact the
consultants
for help in setting privileges to facilitate
sharing. Files not in immediate use, such as source code and
output datasets, should be moved to HPSS.
Space allocations
- A research account is entitled to 500 MB (0.5 GB) in
their home directory. With sufficient justification, it is
possible to have 1 GB. For projects with special requirements,
we will consider extensions of this limit on a case-by-case
basis.
- Data in a system global file system should be present if
needed or produced by a running, queued, or recently completed
job. Files should not be retained in this file system
and should be migrated to HPSS as soon as the files are not
being actively used. Because of the working-directory character of
the system global file system, files are automatically purged on a
regular basis. Files older than 7 days are eligible for purging.
- Space on HPSS is for files which are not
needed immediately. We have not yet set a space limit for
HPSS usage. Even though HPSS is a very large storage
system, space is not unlimited. Users must not store files unrelated to
their CCS projects on HPSS. They must also review their
files periodically and remove unneeded files.
Consequences of abuse
Storage usage will be monitored periodically. If time permits, first
offenders will be warned to clean up their space. Ignoring these warnings
will result in loss of access privileges.
If a file system runs out of or gets low on space,
user data on that file system will be subject to involuntary deletion
without prior notice.
storage home
|
