Run script example

File contents

#!/bin/bash
#
#    If you want to use /bin/csh, you have to change things below!
#
#  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#  May 10nd, 2007
#
#  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#   ____________________________________________________________
#  |                                                            | 
#  | Set initial information for the Queuing system             | 
#  | ==============================================             | 
#  |                                                            | 
#  | All PBS directives below are optional and can be omitted,  | 
#  | resulting in the use of the system defaults.               | 
#  |                                                            | 
#  | Please send comments and questions to support-uit@notur.no | 
#  |                                                            | 
#  |____________________________________________________________|
#
#  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -q itanium
#
#    To run on the Itanium nodes, this isn't really neccessary as the
#    Itanium nodes are the default.  Use -q pentium if you want to run on
#    the pentium nodes.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -lnodes=2:ppn=2
#
#    Number of nodes and cpus per node (ppn) requested, here we ask for a
#    total of 4 cpus, only neccessary for parallel jobs.  This creates a
#    file accessible through the environment variable $PBS_NODEFILE in
#    this script that kan be used by mpirun etc., see below.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -lwalltime=12:00:00
#
#    Expecting to run for 12 hours.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -lpmem=1000MB
#
#    Expecting to use 1000 megabytes of memory per process, this is
#    different from what we use on SMPs where we supply total memory for
#    the whole job.  Remark: The queueing system detects that the nodes
#    have 4000MB of memory which is slightly less than 4GB, so using
#    1000MB will let the queueing system pack 4 processes on one node,
#    while asking for 1GB will get you only 3 procs per node.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -lfile=10gb
#
#    To specify the amount of disk space that needs to available on a node.
#    Only necessary if you are using the local work area on the nodes: /work
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -m abe
#
#    The queuing sytem will send an email on job Abort, Begin, End
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#PBS -A nn9999k
#
#    To specify which account you want to use.
#    Only necessay if you have multiple accounts.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -





#   ______________________________________________________________
#  |                                                              | 
#  |                                                              |
#  |  Seting up and running your job on Snowstorm                 |
#  |  ===========================================                 |
#  |                                                              |
#  |  We are now ready to begin running commands.                 |
#  |                                                              |
#  |  This runscript is run as a regular shell script on the      |
#  |  first node assigned to this job, hereafter called Mother    |
#  |  Superior.                                                   |
#  |______________________________________________________________|
#
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

workdir=/work/$USER/$PBS_JOBID
pbsdsh -u mkdir -p $workdir

#    Creates a unique work area. Never assume that /work/username exist.
#    If a compute node crashes it is likely to reinstall itself and scrap
#    the whole work area.
#
#    The pbsdsh command get the hostlist for this job directly from the 
#    queueing system and using "-u" lets you execute the same command in 
#    parallel on all the nodes belonging to this job. It is much faster 
#    than looping over the host list and ssh-ing to every node.
#
#    If you for some obscure reason needs the hostlist it is in the 
#    file pointed to by $PBS_NODEFILE.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

cd $PBS_O_WORKDIR               

#    cd to the directory where the job was submitted from.  
#
#    Don't confuse this with the above mentioned workdir.
#    This isn't really neccessary here, but can be convenient if you 
#    want to copy many input files to the /work area on the compute nodes.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#

executable=$PBS_O_WORKDIR/mpi_test.x
pbsdsh -u cp $executable $workdir

#    Copy the executable to every node, doing so makes you independent
#    of the nfs server on the frontend that seem to go offline once in
#    a while (This isn't the case anymore, but it is still a good idea).
#    After this every compute node has its own copy of the executable, in
#    this case mpi_test.x
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#    Insert your job commands here, for instance:

cd $workdir 

#    Now we are on local disk on Mother Superior.

scampiexec ./mpi_test.x

#    Running your MPI job by launching it with scampiexec.
#
#    Don't put the job commands in the background, eg. by adding & at
#    the end.  Doing so will make the job escape the queueing system, 
#    create havoc with the scheduling and result in you getting angry
#    email from the sysadmins.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#    After the job has finished it't time to clean up.
#    But be careful, if you are in the wrong catalog you might shoot
#    yourself in the foot. Be sure to copy any important result to
#    your home catalog before doing this. For instance:
#

cp data.d $PBS_O_WORKDIR

#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#    Then remember to cd out of the catalog before you remove it

cd /tmp
pbsdsh -u rm -rf $workdir

#    NEVER use 'rm -rf *' !!!
#
#    If you for some reason is in the wrong place, say, in your home
#    catalog, you will lose data. 90% of all restores we do from backup
#    is because of people having this in their job scripts.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
#
#    The job ends when the script exit.
#
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -