Welcome to Wiki Shark cluster
New
Shark will be renewed this year with a new scheduler SLURM in a new network environment.
People who would like to take a look and try the new scheduler please read the new SLURM wiki.
This new SLURM cluster is a test cluster this means we cannot guarantee 100% uptime.
Introduction
In this wiki, we will introduce the Shark cluster. We assume you know your way around in the terminal and that you will use the cluster in a responsible way and respect the available resources. Remember that a cluster runs many different processes submitted by many different users. If you are not sure what you are doing, ask your administrator before submitting very intensive jobs! For more examples and useful commands you can use our gitlab repository.
Shark cluster runs with Open Grid Scheduler/Grid Engine this is a free and open-source batch-queuing system for distributed resource management. OGS/GE is based on Sun Grid Engine, and maintained by the same group of external (i.e. non-Sun) developers who started contributing code to Sun Grid Engine since 2001.
Current release of Open Grid Scheduler : Grid Engine 2011.11p1
Introduction and interesting starting points
The following links contain the minimal information you need to get started on the shark cluster.
When using the cluster you acknowledge that you have read and understand the following articles.
- Guidelines -- what you can and can't do on this cluster
- Function of the Shark cluster -- Primary usage Shark cluster
- Network shares -- where to put your data
- Queues -- which queue is for what
- Batch queuing commands -- explanation of the qsub command
- Examples -- examples how to qsub your jobs
- Makefile -- Howto use qmake with a Makefile
- SGE state symbol codes -- Sun Grid Engine SGE state letter symbol codes meanings
- FAQ -- Frequently asked questions and answers
- Presentations -- Interesting presentations
- Testsets -- Featured test data
For FSW users there is a dedicated Wiki page.
Getting started
Accounting
- How to get a user account on the Shark Cluster
- Short intro course for new Shark users -- 2hr. intro course mandatory for new Users.
- Shark Introduction course schedule 2020
- How to configure public key authentication
- How to change your researchlumc password
Connecting
Usage
- How to use the Evironment Modules
- Forking an input stream to several programs
- How to submit a R-script with qsub
- How to submit a MPICH2 job
- How to start an IPython Notebook session
- How to correct long PacBio reads using pacBioToCA
- Using the fsl analysis tools for FMRI, MRI and DTI brain imaging data
Configuration Shark cluster
- Configuration Type of headnode and blade servers
- Graphical representation of the shark cluster
- Description of the execute nodes
- Hardware ownership
Limitations
Support
- Contact Information
- Ganglia Shark cluster monitor -- view the load, disk space etc. on the Shark cluster
- Grid Engine HOWTOs.
- Gitlab repository (for course material and example scripts).
Usage Statistics
Usefull Links
Documentation
- Setup and install the Shark head node with Ubuntu 10.04.04 LTS and Open Grid Scheduler
- Setup LDAP/Kerberos authentication within a Microsoft Windows 2008 Domain
- Open Grid Scheduler FineTune and solutions
- Shark Ascii logos