Tutorials
Running Programs
Programs are scheduled to run on Sesame using the qsub command, a component of Torque. Users belonging to groups with an associated queue, should specify their queue in the job submission script (#PBS -q <queue_name>). Others should NEVER specify a queue. Your job will be put into the appropriate queue, based on the requirements that you describe. See Usage Guidelines and the qsub man page for details.
Programming Sesame
The Gnu, PathScale, and Intel compilers are installed on Sesame. The standard MPI implementation for Sesame is OpenMPI, an MPICH compatible library that supports the Infiniband infrastructure.
To set up your environment correctly on Sesame, it is highly recommended to use the module facility. This is a utility to correctly set your environment without having to know all the paths to the executables. Different environments can be set quickly allowing useful comparisons of code compiled with different executables. In most cases a simple module load openmpi command can be issued setting up your environment to use the latest openmpi as well as the Intel compilers.
Compiling parallel MPI programs
module load openmpi (loads the openmpi environment as well as the Intel compilers)
mpif90 myMPIcode.f
mpicc myMPIcode.c
Submitting a Job
Once the executable is compiled, a job script will need to be created for the scheduler. For this machine there are a total of eight processor cores per node. Here is a sample script which uses 16 processors allocated as 8 processors on 2 nodes:
cd mpi_directory
cat parallel.cmd #!/bin/bash
# parallel job using 16 processors. and runs for 4 hours (max)
#PBS -l nodes=2:ppn=8,walltime=4:00:00
# sends mail if process aborts, when it begins, and
# when it ends (abe). Make sure you define your email
# address.
#PBS -m abe
#PBS -M NetID@princeton.edu
#
module load openmpi
cd /home/yourNetID/mpi_directory
mpiexec ./a.out(Change "NetID" to your NetID in the above script.)
To submit the job to the batch queuing system use:
qsub parallel.cmdUseful Tools
| Command | Synopsis |
| showbf | Shows how many nodes are available and for how long. The wall clock limit of a job should be less than the duration reported by showbf, otherwise the job will not run before the next scheduled maintenance period. |
| mdiag -p | Shows the priority assigned to queued jobs. |
| showq or qstat | Shows jobs in the queues. |
| xpbs | A graphical display of the queues. |
| pbstop | A text based view of the cluster nodes. |
