index.md 4.69 KB
Newer Older
1
# Welcome to Biopet
sajvanderzeeuw's avatar
sajvanderzeeuw committed
2

3

bow's avatar
bow committed
4
## Introduction
5

bow's avatar
bow committed
6
Biopet (Bio Pipeline Execution Toolkit) is the main pipeline development framework of the LUMC Sequencing Analysis Support Core team. It contains our main pipelines and some of the command line tools we develop in-house. It is meant to be used in the main [SHARK](https://humgenprojects.lumc.nl/trac/shark) computing cluster. While usage outside of SHARK is technically possible, some adjustments may need to be made in order to do so.
sajvanderzeeuw's avatar
sajvanderzeeuw committed
7
8


bow's avatar
bow committed
9
## Quick Start
10

bow's avatar
bow committed
11
### Running Biopet in the SHARK cluster
12

bow's avatar
bow committed
13
Biopet is available as a JAR package in SHARK. The easiest way to start using it is to activate the `biopet` environment module, which sets useful aliases and environment variables:
14

bow's avatar
bow committed
15
16
17
~~~
$ module load biopet/v0.3.0
~~~
18

bow's avatar
bow committed
19
With each Biopet release, an accompanying environment module is also released. The latest release is version 0.3.0, thus `biopet/v0.3.0` is the module you would want to load.
sajvanderzeeuw's avatar
sajvanderzeeuw committed
20

bow's avatar
bow committed
21
After loading the module, you can access the biopet package by simply typing `biopet`:
22

23
~~~
bow's avatar
bow committed
24
$ biopet
25
~~~
bow's avatar
bow committed
26

bow's avatar
bow committed
27
28
29
This will show you a list of tools and pipelines that you can use straight away. You can also execute `biopet pipeline` to show only available pipelines or `biopet tool` to show only the tools. What you should be aware of, is that this is actually a shell function that calls `java` on the system-wide available Biopet JAR file.

~~~
bow's avatar
bow committed
30
$ java -jar <path/to/current/biopet/release.jar>
bow's avatar
bow committed
31
32
33
34
35
~~~

The actual path will vary from version to version, which is controlled by which module you loaded.

Almost all of the pipelines have a common usage pattern with a similar set of flags, for example:
sajvanderzeeuw's avatar
sajvanderzeeuw committed
36

37
~~~
bow's avatar
bow committed
38
$ biopet pipeline <pipeline_name> -config <path/to/config.json> -qsub -jobParaEnv BWA -retry 2
39
~~~
40

bow's avatar
bow committed
41
The command above will do a *dry* run of a pipeline using a config file as if the command would be submitted to the SHARK cluster (the `-qsub` flag) to the `BWA` parallel environment (the `-jobParaEnv BWA` flag). We also set the maximum retry of failing jobs to two times (via the `-retry 2` flag). Doing a good run is a good idea to ensure that your real run proceeds smoothly. It may not catch all the errors, but if the dry run fails you can be sure that the real run will never succeed.
42

bow's avatar
bow committed
43
If the dry run proceeds without problems, you can then do the real run by using the `-run` flag:
44

bow's avatar
bow committed
45
~~~
bow's avatar
bow committed
46
$ biopet pipeline <pipeline_name> -config <path/to/config.json> -qsub -jobParaEnv BWA -retry 2 -run
bow's avatar
bow committed
47
~~~
48

bow's avatar
bow committed
49
It is usually a good idea to do the real run using `screen` or `nohup` to prevent the job from terminating when you log out of SHARK. In practice, using `biopet` as it is is also fine. What you need to keep in mind, is that each pipeline has their own expected config layout. You can check out more about the general structure of our config files [here](general/config.md). For the specific structure that each pipeline accepts, please consult the respective pipeline page.
Sander van der Zeeuw's avatar
Sander van der Zeeuw committed
50

bow's avatar
bow committed
51
### Running Biopet in your own computer
sajvanderzeeuw's avatar
sajvanderzeeuw committed
52

bow's avatar
bow committed
53
At the moment, we do not provide links to download the Biopet package. If you are interested in trying out Biopet locally, please contact us as [sasc@lumc.nl](mailto:sasc@lumc.nl).
sajvanderzeeuw's avatar
sajvanderzeeuw committed
54
55


bow's avatar
bow committed
56
## Contributing to Biopet
sajvanderzeeuw's avatar
sajvanderzeeuw committed
57

bow's avatar
bow committed
58
Biopet is based on the Queue framework developed by the Broad Institute as part of their Genome Analysis Toolkit (GATK) framework. The current Biopet release is based on the GATK 3.3 release.
sajvanderzeeuw's avatar
sajvanderzeeuw committed
59

bow's avatar
bow committed
60
We welcome any kind of contribution, be it merge requests on the code base, documentation updates, or any kinds of other fixes! The main language we use is Scala, though the repository also contains a small bit of Python and R. Our main code repository is located at [https://git.lumc.nl/biopet/biopet](https://git.lumc.nl/biopet/biopet/issues), along with our issue tracker.
61

bow's avatar
bow committed
62
## Local development setup
63

bow's avatar
bow committed
64
To develop Biopet, Java 7, Maven 3.2.2, and GATK Queue 3.3 is required. Please consult the Java homepage and Maven homepage for the respective installation instruction. After you have both Java and Maven installed, you would then need to install GATK Queue. However, as the GATK Queue package is not yet available as an artifact in Maven Central, you will need to download, compile, and install GATK Queue first.
Sander van der Zeeuw's avatar
Sander van der Zeeuw committed
65

bow's avatar
bow committed
66
67
68
69
70
71
~~~
$ git clone https://github.com/broadgsa/gatk
$ cd gatk
$ git checkout 3.3                              # the current release is based on GATK 3.3
$ mvn -U clean install
~~~
Sander van der Zeeuw's avatar
Sander van der Zeeuw committed
72

bow's avatar
bow committed
73
This will install all the required dependencies to your local maven repository. After this is done, you can clone our repository and test if everything builds fine:
74

bow's avatar
bow committed
75
76
77
78
79
~~~
$ git clone git@git.lumc.nl:biopet/biopet.git
$ cd biopet
$ mvn -U clean install
~~~
80

bow's avatar
bow committed
81
If everything builds fine, you're good to go! Otherwise, don't hesitate to contact us or file an issue at our issue tracker.
82
83


Sander van der Zeeuw's avatar
Sander van der Zeeuw committed
84
## About
bow's avatar
bow committed
85

sajvanderzeeuw's avatar
sajvanderzeeuw committed
86
87
88
89
Go to the [about page](about)

## License

90
See: [License](license.md)