Linux shell scripting for high-throughput biological data processing on supercomputers
Organizer: Allegra Via (ELIXIR-IIB Training Coordinator, IBPM-CNR, IT) Loredana Le Pera (ELIXIR-IIB Training Team, IBIOM-CNR, IT) Tiziana Castrignanò (SCAI Department, CINECA, Roma, IT)
Host institution: CINECA
Start: Tuesday, 16 January 2018 @ 09:00
End: Wednesday, 17 January 2018 @ 18:00
Venue: Via dei Tizii, 6B
County: Città Metropolitana di Roma
Scientific topic: Data mining, BioinformaticsTarget audience:
- beginner bioinformaticians
- Biomedical researchers
- PhD Students or young researchers in molecular biology and/or genetics with little or no background in bioinformatics.
An unprecedented amount of biomedical data have been produced and stored in the last years. Managing such biological big data is often not affordable without high-performance computing architectures, needed to analyze and process large-scale datasets. Running high-throughput (HTP) bioinformatics data pipelines on supercomputing machines requires advanced Linux shell command line and scripting skills. Most scientists working with such data often lack such skills or have acquired them by self-learning without becoming fully independent and fluent. This may have repercussions on the quality, reproducibility, and reliability of the analyses. In this two-day course, we will introduce the Linux shell and, on day one, we will show how to navigate and work with files and directories, how to combine commands to do new things, how to perform the same actions on many different files, how to filter and selectively extract data from tables, and how to find objects in files. Moreover, we will show how to connect to a remote supercomputer and how to utilise a supercomputing environment to analyse big amount of biological data, run simple shell scripts and bioinformatics pipelines. Day 2 will be wholly practical. Participants are invited to let us know in advance which are the typical file format(s) they have to deal with (e.g. fastq, table, etc), the typical processes they need to perform on them (e.g. filtering, ordering, etc.) and the typical programs they need to run (e.g. bwa, hisat2, etc.) so that we can prepare tailored practicals. Participants are also welcome to come to the course with one or more files they wish to work with, provided they do not exceed a given size.
- Workshops and courses
- First come first served
- Registration of interest
Keywords: Unix/Linux, HTP data, Shell script, Managing and Processing data, supercomputer