File Systems and Data handling

Contents

• Available File Systems
• Home Directories
• Work Directories
• Transferring Data from/to Njord
• scp
• Basic Tools
• High-Performance Tools
• ftp
• Subversion
• Compressing/Uncompressing files and directories
• How to complile with the NetCDF library
  
• How to compile with HDF5 library
  
• Disk Quota
• List Quota
• Default Quota
• Request Increased Quota
• Backup
• Backup Policy
• How to Restore Files
• Backup
• Backup Policy
• Archiving

 

Available File Systems

njord.hpc.ntnu.no uses the General Parallel File System (GPFS) to provide global access to user-data from any of the nodes. The following user file-systems are available on Njord.

Home Directories (Permanent Data)

The /home file system is currently 48 terabytes, based on large SATA disks. Quota is enabled. All files are backed up daily. /home should be used to store source codes, scripts or any data which need to be kept.

Work Directories (Temporary Data)

The /work file system is currently 30 terabytes, based on faster FC-disks. Quota is enabled. There is NO backup of any file. Files older than 21 days are periodically removed. /work should be used as a work area for running jobs. Any files which needs to be kept, has to be transferred to another systems, or copied to /home.

 

Transferring Data from/to Njord

scp

Basic Tools

Use scp or sftp to transfer files from/to Njord from your workstation/PC.

High-Performance Tools

The ssh client and server on Njord's login nodes is ssh with applied HPN patches. If you also use an hpnssh client when connecting to Njord, then the transfer throughput will be increased.

To use this feature you must have an HPN patched openssh version installed on your PC. You can check if your ssh client has HPN patches by issuing:

ssh -V

if the output contains the word "hpn" followed by version and release then you can make use of the high performance features.

Transfer can then be speeded up by disabling data encryption, AFTER you have been authenticated or logged into the remote host (NONE cipher).

NONE cipher

This cipher has the highest transfer rate. Keep in mind that data after authentication is NOT encrypted, therefore the files can be sniffed and collected unencrypted by any attacker. To use this feature you must add the following to the client command:

-oNoneSwitch=yes -oNoneEnabled=yes

Anytime the None cipher is used a warning will be printed on the screen:

WARNING: NONE CIPHER ENABLED

If you do not see this warning then the NONE cipher is not in use.

ftp

Njord does not allow incoming ftp, however you may initiate outgoing ftp sessions to any file server which provide the ftp service.

Subversion

Subversion can also be used for transferring source tree directories to Njord. Type "module load subversion" before you issue subversion commands.

 

Compressing/Uncompressing files and directories

The available UNIX-commands for compression and uncompression are:

• gzip/gunzip - for description do "info gzip"
  
• compress/uncompress -  for description do "info compress "
  
• pigz  - for description  "man pigz"
• bzip2/bunzip2.

 A discussion on how to use pigz on a mulitcore system is given here

To compress a file, follow the example:

$ ls -l datafile 
-rw-r--r-- 1 mad xno 1536000 2010-07-28 14:17 datafile
$ gzip datafile 
$ ls -l datafile.tar.gz 
-rw-r--r-- 1 mad xno 307447 2010-07-28 14:17 datafile.gz
$

To uncompress a gzipped file, follow the example:

$ ls -l ipm_0.982.tar.gz 
-rw-r--r-- 1 mad xno 307447 2010-07-28 14:17 datafile
$ gunzip datafile.gz 
$ ls -l datafile 
-rw-r--r-- 1 mad xno 1536000 2010-07-28 14:17 datafile
$ 

To compress a directory, the compression tool needs to be used in combination with a archiving tool, like tar:

$ ls -ld project
drwx------ 15 mad xno 32768 2009-12-09 18:25 project
$ tar cf - project/ | gzip --best > project.tar.gz
$ ls -l project.tar.gz 
-rw------- 1 mad xno 304827 2010-08-09 11:07 project.tar.gz
$ 

To unpack a archived directory:

$ gunzip -c project.tar.gz  | tar xf -
$ ls -ld project
drwx------ 15 mad xno 32768 2009-12-09 18:25 project
$

 

How to compile  with the NetCDF library

The NetCDF library is available on njord. NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. Both NetCDF-3 and a parallel version of NetCDF-4 are installed.

NetCDF-3

Using NetCDF-3, compile and link examples (sample files from the Example netCDF programs ):

$ module load netcdf
$ xlc simple_xy_wr.c -o simple_xy_wr -lnetcdf
$ xlf90 simple_xy_wr.f90 -o simple_xy_wr -lnetcdf

NetCDF-4

NetCDF-4 makes available the parallel I/O features of HDF5, and support for an enhanced data model by including the capability to use HDF5 as the data storage layer for the netCDF API. Compile and link examples:

$ module load netcdf/4.1.1
$ mpxlf90 simple_xy_wr.f90 -o simple_xy_wr -lnetcdf -lhdf5 -lhdf5_hl -lz -lgpfs
$ mpcc simple_xy_wr.c -o simple_xy_wr -lnetcdf -lhdf5_hl -lhdf5 -lz -lgpfs -lm

NetCDF resources:

• The NetCDF User Guide
• The NetCDF Tutorial
• The NetCDF C Interface Guide
• The NetCDF Fortran Interface Guide

 

How to compile with HDF5 library

The HDF5 library is available on njord.  How to use HDF5 are described in the Introduction to HDF5 and the HDF5 User Guide.

Here is how you compile a HDF5 C-file. The file are from the example set installed on Njord:

$ module load hdf
$ h5pcc -o h5_attribute h5_attribute.c 

To compile a Fortran file from the example set installed on Njord:

$ module load hdf
$ h5pfc -o attrexample attrexample.f90

Note that the HDF5-compile commands h5pcc and h5pfc are wrappers to the IBM compilers mpcc_r and mpxlf95_r. The IBM compilers include the MPI-library and produce a parallel application. To be tested, the H5DF executable must be run through poe (interactively) or through the batch system.See the description in the Introduction to using njord on  how to run program interactively or through the batch system.

Further details about compiling HDF5 applications is given here:http://www.hdfgroup.org/training/compile5/

 

 

Disk Quota

List Quota

Use "mmlsquota" to list available quota for /home and /work.

Default Quota

Initial default quota for /home is 20 gigabyte, and /work 500 gigabyte.

Request Increased Quota

If you need more disk space, contact support-ntnu@notur.no and ask for more.

 

Backup

Backup Policy

Veritas Netbackup is used to backup user-files on /home to a tape robot. Incremental backup is done daily, full backup each month. Backup expires after 6 months.

How to Restore Files

Contact support-ntnu@notur.no for restoring files or entire directories.

 

Archiving

Archiving is not provided. However you may apply for archive space on Norstore (NTNU).