kb.hurricane-ridge.com » Sun Grid Engine

SGE Queue States

admin — Fri, 30 Sep 2011 16:21:19 +0000

Reference: Monitoring and Controlling Queues

a – Load threshold alarm
o – Orphaned
A – Suspend threshold alarm
C – Suspended by calendar
D – Disabled by calendar
S – Suspended by subordination to another queue
c – Configuration ambiguous
d – Disabled
s – Suspended
u – Unknown
E – Error

List SGE resource availability with qstat -F

admin — Wed, 31 Aug 2011 19:32:54 +0000

For a queue that uses h_vmem as a resource:

> qstat -F h_vmem queuename qtype resv/used/tot. load_avg arch states --------------------------------------------------------------------------------- all.q@compute-1-0.local BIP 0/0/12 0.05 lx26-amd64 hc:h_vmem=48.000G

Disabling and Enabling an SGE queue

admin — Fri, 28 May 2010 21:35:51 +0000

Disable:

# qmod -d all.q@compute-0-0.local workflow.q@compute-0-0.local

Enable:

# qmod -e all.q@compute-0-0.local workflow.q@compute-0-0.local

Anatomy of an SGE Job File

admin — Mon, 01 Jun 2009 22:55:00 +0000

The contents of a basic Sun Grid Engine job file look something like this:

#!/bin/bash # SGE uses shell script comments that start with "#$" to configure how # SGE handles the job. # # Use the bash shell as the shell for the job: #$ -S /bin/bash # # Write output and error stream to the current working directory: #$ -cwd # # The job command or commands are below: /apps/bin/blastall -p blastp \ -d /nethome/aleonard/blast/database.fasta \ -i /nethome/aleonard/blast/input.fasta \ -o /nethome/aleonard/blast/output.txt

To use a job file, create a shell script with contents similar to the above (such as “job.sh”), make the script executable (“chmod 755 job.sh”) and submit it to SGE (“qsub job.sh”).

It is important to note that blastall will only be using one processor (the “-a” flag is not used, so it defaults to one CPU). It is generally the best for throughput – getting the most work done in the least amount of time – if you can split your job into many multiple, single-threaded jobs and run those in parallel, as opposed to running one or a few multiple-threaded jobs.

References: N1 Grid Engine 6 User’s Guide, NCBI BLAST Scaling on Sun Fire T2000

Basic SGE Commands

admin — Mon, 01 Jun 2009 22:50:29 +0000

The “qstat” command provides information on the state of Grid Engine machines and queues. Entering “qstat” alone gives a brief overview of jobs in the queue:

> qstat job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 395 0.55500 run-DSL22- aleonard r 06/13/2007 15:59:18 all.q@compute-0-1.local 1 393 0.55500 run-DSL20- aleonard r 06/13/2007 15:59:18 all.q@compute-0-6.local 1 394 0.55500 run-DSL21- aleonard r 06/13/2007 15:59:18 all.q@compute-0-7.local 1

“qstat -f” gives a “full” display of information, showing the state of all the machines in the cluster:

> qstat -f queuename qtype used/tot. load_avg arch states ---------------------------------------------------------------------------- all.q@compute-0-0.local BIP 0/4 0.23 lx26-amd64 ---------------------------------------------------------------------------- all.q@compute-0-1.local BIP 1/4 1.00 lx26-amd64 395 0.55500 run-DSL22- aleonard r 06/13/2007 15:59:18 1 ---------------------------------------------------------------------------- all.q@compute-0-2.local BIP 0/4 0.05 lx26-amd64 ---------------------------------------------------------------------------- all.q@compute-0-3.local BIP 0/4 0.04 lx26-amd64 ---------------------------------------------------------------------------- all.q@compute-0-4.local BIP 0/4 0.25 lx26-amd64 ---------------------------------------------------------------------------- all.q@compute-0-5.local BIP 0/4 0.86 lx26-amd64 ---------------------------------------------------------------------------- all.q@compute-0-6.local BIP 1/4 1.10 lx26-amd64 393 0.55500 run-DSL20- aleonard r 06/13/2007 15:59:18 1 ---------------------------------------------------------------------------- all.q@compute-0-7.local BIP 1/4 1.89 lx26-amd64 394 0.55500 run-DSL21- aleonard r 06/13/2007 15:59:18 1 ---------------------------------------------------------------------------- all.q@compute-0-8.local BIP 0/4 0.62 lx26-amd64

Jobs are submitted to a queue using the ‘qsub’ command:

> qsub blast.sh

An SGE array job is submitted using the ‘-t’ flag:

> qsub -t 1:3 sge-blast-array.sh