Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • S Shark CentOS Slurm User guide
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • SharkShark
  • Shark CentOS Slurm User guide
  • Wiki
  • Home

Home · Changes

Page history
fixed some typos in the storage part authored Oct 05, 2021 by van Vliet's avatar van Vliet
Hide whitespace changes
Inline Side-by-side
home.md
View page @ a11357a6
......@@ -1640,20 +1640,20 @@ Data anonymization has been defined as a "process by which personal data is irre
## Data storage / access
This shark cluster has multiple data storage types.
HPC torage does not have any backups!!
HPC storage does not have any backups!!
### Storage solutions
* HPC Isilon storage.
This is fast storage for direct acces to your data on the cluster, which can be purchased from the IT&DI department through [Topdesk](https://topdesk.lumc.nl). Once purchased this storage will be NFS v4 mounted on all the nodes on the cluster. The default mountpoint will be **/exports/<storage-share-name>**.
Access to this mountpoint is handeld by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command `ls -aldh /exports/<storage-share-name> | awk '{print $4}'`
Access to this mountpoint is handled by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command `ls -aldh /exports/<storage-share-name> | awk '{print $4}'`
No backups or snapshots are available for the HPC isilon storage.
* research LTS Isilon storage
This is slow storage for archiving data which can be purchased from the IT&DI department through [Topdesk](https://topdesk.lumc.nl). Once purchased this storage will be NFS v4 mounted on all the execution/gpu/mem nodes on the cluster with read only access, on the login nodes you wil have read and write access. The deafult mountpoint will be **/exports/archive/<storage-share-name>**. Access to this mountpoint is handeld by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command `ls -aldh /exports/archive/<storage-share-name> | awk '{print $4}'`
This is slow storage for archiving data which can be purchased from the IT&DI department through [Topdesk](https://topdesk.lumc.nl). Once purchased this storage will be NFS v4 mounted on all the execution/gpu/mem nodes on the cluster with read only access, on the login nodes you will have read and write access. The default mountpoint will be **/exports/archive/<storage-share-name>**. Access to this mountpoint is handled by an Active Directory group. The default mount access rights are set by an Ansible playbook. To grant users access to this share you need to have them added to the Active Directory group attached to the share. To find out which group is attached to your data storage use the following command `ls -aldh /exports/archive/<storage-share-name> | awk '{print $4}'`
Howto Stage your data:
From within your submit script
Create a directory on the local /tmp `mkdir -p /tmp/$USER/$SLURM_JOB_ID`
Change permissions so only the owner can view the files `chmod 700 /tmp$USER`
Change permissions so only the owner can view the files `chmod 700 /tmp/$USER`
Copy your data `cp <path>/to/Data /tmp/$USER/$SLURM_JOB_ID/`
Execute your binary and save output to /tmp/$USER/$SLURM_JOB_ID/
At the end of your sbatch script move/copy your output to your HPC/LTS storage.
......
Clone repository
  • User Guide
  • Home