O2 and Orchestra

Workflow
Shell script
Control file, Disease file (T2D), Submit jobs to O2 to execute for different
chromosome disease, chromosome combinations

gwas.R (SNP)
gwas.R (SNP)
(GWAS function)
(GWAS function)
Scale up
Output.csv Output files (disease/chromosome)

Manhattan plots Manhattan plots

Generate your final plots Download generated data files Generate your final plots
Local machine O2
O2
• - Start with an interactive shell!
• srun -p interactive --pty -t 4:00:00 -n 2 bash
• -p : partition (interactive, short, medium, long, priority)
• -t : how long do you want the interactive shell
• -n : number of tasks
• type of shell - bash, tcsh
• edit files, write shell scripts using your favorite editor (emacs, vim, nano)
• Submit jobs
• sbatch -n 4 -p priority -t 1-00:00:00 --wrap="<Command>"
• -n : number of tasks
• -c : number of cores
• -p : partition for the job (probably priority or short) [required]
• -t : time - how long will your job run [required]
• --mem : memory
• -e : errfile
• Best practice: wrap this up in a shell script!
O2 continued
• Check on the progress of jobs
• squeue -u <userid> : list of all jobs for the user
• You can specify if you want to see only running jobs or pending jobs etc.
• squeue -u <userid> -t RUNNING
• sstat: get more detailed stats on the job
• Kill jobs when mistakes are made
• scancel <jobid> (get jobid from squeue)-
• Orchestra has equivalent commands
• Example:
• bsub -q short -W 4:00 -n 4 -e gwas.err Rscript runGwas.R T2D 7
/home/ktc13/BMI704/GWAS-Assignment/WTCCC /home/ktc13/BMI704/GWAS-
Assignment/T2DFILES
Putting it together (Example scripts)
• Orchestra : run.sh
• bsub -q short -W 4:00 -n 4 -e gwas.err Rscript gwas.R T2D 12 <inputfile>
<outputfile>
• Repeat for all combinations
• O2: run.sh
• sbatch –n 4 -p short -t 1-00:00:00 –e gwas.err --wrap="Rscript gwas.R T2D 12
<inputfile> <outputfile>”
 Save some typing
foreach j (BD CAD T1D HT RA)
foreach i (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
set errfile="$j.gwas.err"
set outdir="$j.FILES"
echo "bsub -q short -W 4:00 -n 4 -e $errfile Rscript runGwas.R $j $i inputpath path/$outdir
bsub -q short -W 4:00 -n 4 -e $errfile Rscript runGwas.R $j $i inputpath path/$outdir
end
echo "DONE"
end
exit
Resources
• Basics:
• https://wiki.rc.hms.harvard.edu/display/O2/Using+Slurm+Basic
• CheatSheet:
• https://wiki.rc.hms.harvard.edu/display/O2/O2+Command+CheatSheet#O2C
ommandCheatSheet
• From Orchestra to O2:
• https://wiki.rc.hms.harvard.edu/display/O2/Moving+from+Orchestra+to+O2