Skip to content
Snippets Groups Projects
Commit 52b7ea5d authored by Laros's avatar Laros
Browse files

Incorporated Bowo's suggestions.

parent 59202e38
No related branches found
No related tags found
No related merge requests found
# Project skeleton
A skeleton Git repository for analysis projects.
# Cookiecutter for data analysis projects
Set up a SASC data analysis project.
# Preparations
## Preparations
Install dependencies.
apt-get install git-annex
pip install -r https://git.lumc.nl/sasc/cookiecutter-data-analysis/raw/master/requirements.txt
Configure Cookiecutter by editing `.cookiecutterrc` in your home directory.
......@@ -11,9 +13,9 @@ Configure Cookiecutter by editing `.cookiecutterrc` in your home directory.
```yml
---
default_context:
full_name: Your Name
email: UserName@lumc.nl
cookiecutters_dir: /path/to/cookiecutters/
full_name: John Doe
email: J.Doe@lumc.nl
cookiecutters_dir: /tmp
```
Configure Python GitLab by editing `.python-gitlab.cfg` in your home directory.
......@@ -27,43 +29,10 @@ Configure Python GitLab by editing `.python-gitlab.cfg` in your home directory.
url = https://git.lumc.nl
private_token = your_token_here
# Starting a project
To start a new analysis project based on this skeleton, use the following
command.
cookiecutter https://git.lumc.nl/sasc/cookiecutter-data-analysis.git
## Working together on the same clone
If you need to work with other people on the same repository clone on the Shark cluster, you can use the following command to give group access:
find -type d -exec chmod 775 {} \;
find -type f -exec chmod 664 {} \;
## Cloning to a portable disk (NTFS)
Presuming your disk is attached to your own computer and the repository is on
the Shark cluster, you can use the following commands.
git clone ssh://shark/path/to/project/my-project
cd my-project
For some reason `annex-ignore` is set to `true`, we need to set it to `false`.
The `private_token` is the only variable you need to set. You can find your
private token [here](https://git.lumc.nl/profile/account).
git config remote.origin.annex-ignore false
## Starting a project
To start a new data analysis project, use the following command:
Set the repository to *direct mode* to prevent making symlinks, otherwise it
will be very difficult for windows users to access the files.
git annex direct
SSH caching seems to be broken on NTFS.
git config annex.ssh-options "-S /tmp/ssh_mux_%h_%p_%r"
Or alternatively, disable SSH caching all together (slow when working with a
large number of files).
git config annex.sshcaching false
Get the content.
git annex get .
cookiecutter https://git.lumc.nl/sasc/cookiecutter-data-analysis.git
......@@ -2,11 +2,12 @@
"full_name": "Jeroen F.J. Laros",
"email": "J.F.J.Laros@lumc.nl",
"client_full_name": "John Doe",
"client_email": "J.Doe@lumc.nl",
"client_email": "{{ cookiecutter.client_full_name.split()[0][0] + '.' + ''.join(cookiecutter.client_full_name.split()[1:]) }}@lumc.nl",
"client_username": "{{ cookiecutter.client_email.split('@')[0]|lower }}",
"sequencing_centre": "LGTC",
"sequencing_platform": "Illumina HiSeq 2500",
"molecular_type": "RNA",
"experiment_type": "RNASeq",
"project_name": "jdoe-rnaseq",
"namespace": "sasc"
"project_number": 1,
"project_name": "project-{{ cookiecutter.project_number }}-{{ cookiecutter.experiment_type|lower }}-{{ cookiecutter.client_username.replace('.', '') }}",
"gitlab_repo_namespace": "sasc"
}
......@@ -4,7 +4,7 @@ get_value() {
grep "$1: " | cut -f 2 -d ' '
}
namespace={{ cookiecutter.namespace }}
namespace={{ cookiecutter.gitlab_repo_namespace }}
namespace_id=$(gitlab group search --query $namespace | get_value id)
namespace_opt=""
if [ $namespace_id ]; then
......@@ -16,7 +16,23 @@ fi
git init
git add .
git commit -m "Initial commit."
gitlab project create --name {{ cookiecutter.project_name }} $namespace_opt
project_id=$(
gitlab project create \
--name {{ cookiecutter.project_name }} $namespace_opt | get_value id)
git remote add origin \
git@git.lumc.nl:$namespace/{{ cookiecutter.project_name }}.git
git push --set-upstream origin master
client_id=$(
gitlab user search --query {{ cookiecutter.client_username }} | get_value id)
if [ $client_id ]; then
gitlab project-member create \
--project-id $project_id --user-id $client_id --access-level 30
else
cat << EOF
Could not add client to the project.
https://git.lumc.nl/$namespace/{{ cookiecutter.project_name }}.git
EOF
fi
# General information
{{ cookiecutter.experiment_type }} data analysis for {{ cookiecutter.client_full_name }}.
See the [ticket](https://humgenprojects.lumc.nl/trac/sasc/ticket/{{ cookiecutter.project_number }}) for the original request.
Project started on: {% now 'local', '%a %b %d, %Y' %}.
Project finished on:
## Contact information
Owner: {{ cookiecutter.client_full_name }} <{{ cookiecutter.client_email }}>
Project leader: {{ cookiecutter.full_name }} <{{ cookiecutter.email }}>
......
......@@ -5,7 +5,6 @@ sequencing centre). Also see *Notes* below.
## Details
Sequencing centre: {{ cookiecutter.sequencing_centre }}
Platform: {{ cookiecutter.sequencing_platform }}
Molecular type: {{ cookiecutter.molecular_type }}
Experiment type: {{ cookiecutter.experiment_type }}
Owner: {{ cookiecutter.client_full_name }} <{{ cookiecutter.client_email }}>
Data gathered by: {{ cookiecutter.full_name }} <{{ cookiecutter.email }}>
......@@ -26,3 +25,38 @@ now do `git commit` as normal.
For more information, please see the
[Git-annex walkthrough](http://git-annex.branchable.com/walkthrough/).
### Working together on the same clone
If you need to work with other people on the same repository clone on the Shark cluster, you can use the following command to give group access:
find -type d -exec chmod 775 {} \;
find -type f -exec chmod 664 {} \;
### Cloning to a portable disk (NTFS)
Presuming your disk is attached to your own computer and the repository is on
the Shark cluster, you can use the following commands.
git clone ssh://shark/exports/{{ cookiecutter.gitlab_repo_namespace }}/{{ cookiecutter.project_name }}
cd my-project
For some reason `annex-ignore` is set to `true`, we need to set it to `false`.
git config remote.origin.annex-ignore false
Set the repository to *direct mode* to prevent making symlinks, otherwise it
will be very difficult for windows users to access the files.
git annex direct
SSH caching seems to be broken on NTFS.
git config annex.ssh-options "-S /tmp/ssh_mux_%h_%p_%r"
Or alternatively, disable SSH caching all together (slow when working with a
large number of files).
git config annex.sshcaching false
Get the content.
git annex get .
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment