Programs are scheduled to run on Della using the qsub command, a component of Torque. You should never specify a queue unless you are running in . Your job will be put into the appropriate queue, based on the requirements that you describe. For further information, see the Usage Guidelines section and, the qsub man page.
The Gnu and Intel compilers are installed on Della. The standard MPI implementation for Della is OpenMPI, an MPICH-compatible library that supports the Infiniband infrastructure.
To set up your environment correctly on Della, it is highly recommended to use the module command. This is a utility to correctly set your environment without having to know all the paths to the executables. Different environments can be set quickly allowing useful comparisons of code compiled with different executables. In most cases a simple module load openmpi command can be issued setting up your environment to use the latest openmpi as well as the Intel compilers.
Compiling parallel MPI programs
module load openmpi (loads the openmpi environment as well as the Intel compilers)
Submitting a Job
Once the executable is compiled, a job script will need to be created for the scheduler. For this cluster there are a total of 12 processors per node. The 96 GB memory nodes (8 GB per core) can be chosen by adding a special flag to the requirements: e.g., nodes=2:ppn=12:mem96 .
Here is a sample script which uses 48 processors allocated as 12 processors on 4 nodes:
# parallel job using 48 processors. and runs for 4 hours (max)
#PBS -l nodes=4:ppn=12,walltime=4:00:00
# sends mail if process aborts, when it begins, and
# when it ends (abe). Make sure you define your email
#PBS -m abe
#PBS -M yourNetID@princeton.edu
module load openmpi
(Change "yourNetID" to your NetID in the above script.)
To submit the job to the batch queuing system use:
|showbf||Shows how many nodes are available and for how long. The wall clock limit of a job should be less than the duration reported by showbf, otherwise the job will not run before the next scheduled maintenance period.|
|mdiag -p||Shows the priority assigned to queued jobs.|
|showq or qstat||Shows jobs in the queues.|
|xpbs||A graphical display of the queues.|
|pbstop||A text based view of the cluster nodes.|