Data Carpentry

Data Carpentry's aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain.

Showing 10 materials.

Note that the figshare download is an archive (.zip) file that rudely explodes all of the files into your current directory. By default SQLite Manager opens in a separate window and it is not possible to zoom in to enlarge the font so that it is more readable, especially for students in the back...

SQL for Ecology: Instructor Notes https://tess.elixir-europe.org/materials/sql-for-ecology-instructor-notes Note that the figshare download is an archive (.zip) file that rudely explodes all of the files into your current directory. By default SQLite Manager opens in a separate window and it is not possible to zoom in to enlarge the font so that it is more readable, especially for students in the back rows. The way to fix this is to: You can then use Ctrl - + to zoom just like any other web page. See this slide deck as a sample intro for the lesson: SQL Intro Deck 2017-05-25

Data Carpentry's aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with ecological data in Python. Data for this lesson is from the...

Data Carpentry Python for Ecologists https://tess.elixir-europe.org/materials/data-carpentry-python-for-ecologists Data Carpentry's aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with ecological data in Python. Data for this lesson is from the Portal Project Teaching Database - available on FigShare. The data files used in this lesson are surveys.csv download link - https://ndownloader.figshare.com/files/2292172 and species.csv download link - https://ndownloader.figshare.com/files/3299483. Requirements: Data Carpentry's teaching is hands-on, so participants are encouraged to bring in and use their own laptops to insure the proper setup of tools for an efficient workflow once you leave the workshop. (We will provide instructions on setting up the required software several days in advance, and the classroom will have computers with the software installed). There are no pre-requisites, and we will assume no prior knowledge about the tools. Participants are required to abide by Software Carpentry's Code of Conduct. Twitter: #datacarpentry 2016-03-07

By default, Data Carpentry does not have people pull the whole repository with all the scripts and addenda. Therefore, you, as the instructor, get to decide how you’d like to provide this script to learners, if at all. To use this, students can navigate into includes/scripts terminal, and execute...

Python for ecologists: Instructor NotesChallenge solutions https://tess.elixir-europe.org/materials/python-for-ecologists-instructor-noteschallenge-solutions By default, Data Carpentry does not have people pull the whole repository with all the scripts and addenda. Therefore, you, as the instructor, get to decide how you’d like to provide this script to learners, if at all. To use this, students can navigate into includes/scripts terminal, and execute the following: If learners receive an AssertionError, it will inform you how to help them correct this installation. Otherwise, it will tell you that the system is good to go and ready for Data Carpentry! What happens when you type a_tuple[2] = 5 vs a_list[1] = 5? As a tuple is immutable, it does not support item assignment. Elements in a list can be altered individually. Type type(a_tuple) into the Python interpreter - what is the object type? 2017-05-25

OpenRefine (formerly Google Refine) is a powerful free and open source tool for working with messy data: cleaning it and transforming it from one format into another. This lesson will teach you to use OpenRefine to effectively clean and format data and automatically track any changes that you...

Open Refine for Ecology https://tess.elixir-europe.org/materials/data-carpentry-openrefine-for-ecology OpenRefine (formerly Google Refine) is a powerful free and open source tool for working with messy data: cleaning it and transforming it from one format into another. This lesson will teach you to use OpenRefine to effectively clean and format data and automatically track any changes that you make. Many people comment that this tool saves them literally months of work trying to make these edits by hand. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to insure the proper setup of tools for an efficient workflow. These lessons assume no prior knowledge of the skills or tools. To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions. This lesson requires a working copy of OpenRefine (also called GoogleRefine). To most effectively use these materials, please make sure to install everything before working through this lesson. 2017-05-25

Note the file types OpenRefine handles: TSV, CSF, *SV, Excel (.xls .xlsx), JSON, XML, RDF as XML, Google Data documents. Support for other formats can be added with OpenRefine extensions. In this first step, we’ll browse our computer to the sample data file for this lesson (If you haven’t...

Open Refine for Ecology: Instructor Notes https://tess.elixir-europe.org/materials/open-refine-for-ecology-instructor-noteslesson Note the file types OpenRefine handles: TSV, CSF, *SV, Excel (.xls .xlsx), JSON, XML, RDF as XML, Google Data documents. Support for other formats can be added with OpenRefine extensions. In this first step, we’ll browse our computer to the sample data file for this lesson (If you haven’t already, download the data from: https://ndownloader.figshare.com/files/2252083). In this case, I’ve modified the Portal_rodents.csv file. I added several columns: scientificName, locality, county, state, country and I generated several more columns in the lesson itself (JSON, decimalLatitude, decimalLongitude). Data in locality, county, country, JSON, decimalLatitude and decimalLongitude are contrived and are in no way related to the original dataset. Once OpenRefine is open, you’ll be asked if you want to Create, Open, or Import a Project. Exploring data by applying multiple filters OpenRefine supports faceted browsing as a mechanism for 2017-05-25

We organize data in spreadsheets in the ways that we as humans want to work with the data, but computers require that data be organized in particular ways. In order to use tools that make computation more efficient, such as programming languages like R or Python, we need to structure our data...

Data Organization in Spreadsheets https://tess.elixir-europe.org/materials/data-carpentry-spreadsheets-for-ecology We organize data in spreadsheets in the ways that we as humans want to work with the data, but computers require that data be organized in particular ways. In order to use tools that make computation more efficient, such as programming languages like R or Python, we need to structure our data the way that computers need the data. Since this is where most research projects start, this is where we want to start too! In this lesson, you will learn: Much of your time as a researcher will be spent in this ‘data wrangling’ stage. It’s not the most fun, but it is necessary. In this lesson you will learn how to think about data organization and some practices for more effective data wrangling. With this approach you can better format current data and plan new data collection so less data wrangling is needed. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to insure the proper setup of tools for an efficient workflow. These lessons assume no prior knowledge of the skills or tools. To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions. 2017-05-25

This lesson is optional The challenge with this lesson is that the instructor’s version of the spreadsheet software is going to look different than about half the room’s. It makes it challenging to show where you can find menu options and navigate through. Instead discuss the concepts of quality...

Data Organization in Spreadsheets: Instructor Notes https://tess.elixir-europe.org/materials/data-organization-in-spreadsheets-instructor-notes This lesson is optional The challenge with this lesson is that the instructor’s version of the spreadsheet software is going to look different than about half the room’s. It makes it challenging to show where you can find menu options and navigate through. Instead discuss the concepts of quality control, and how things like sorting can help you find outliers in your data. Provide information on setting up your environment for learners to view your live coding (increasing text size, changing text color, etc), as well as general recommendations for working with coding tools to best suit the learning environment. The main challenge with this lesson is that Excel looks very different and how you do things is even different between Mac and PC, and between different versions of Excel. So, the presenter’s environment will only be the same as some of the learners. 2017-05-25

Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with ecology data in R. This is an introduction to R designed for...

Data Carpentry: R for data analysis and visualization of Ecological Data https://tess.elixir-europe.org/materials/data-carpentry-r-for-data-analysis-for-ecology Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with ecology data in R. This is an introduction to R designed for participants with no programming experience. These lessons can be taught in a day (~ 6 hours). They start with some basic information about R syntax, the RStudio interface, and move through how to import CSV files, the structure of data frames, how to deal with factors, how to add/remove rows and columns, how to calculate summary statistics from a data frame, and a brief introduction to plotting. The last lesson demonstrates how to work with databases directly from R. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to ensure the proper setup of tools for an efficient workflow. These lessons assume no prior knowledge of the skills or tools, but working through this lesson requires working copies of the software described below. To most effectively use these materials, please make sure to download the data and install everything before working through this lesson. Data files for the lesson are available and can be downloaded manually here: http://dx.doi.org/10.6084/m9.figshare.1314459 2017-05-25

Python is a general purpose programming language that is useful for writing scripts to work effectively and reproducibly with data. This is an introduction to Python designed for participants with no programming experience. These lessons can be taught in a day (~ 6 hours). They start with some...

Python for ecologists https://tess.elixir-europe.org/materials/python-for-ecologists Python is a general purpose programming language that is useful for writing scripts to work effectively and reproducibly with data. This is an introduction to Python designed for participants with no programming experience. These lessons can be taught in a day (~ 6 hours). They start with some basic information about Python syntax, the Jupyter notebook interface, and move through how to import CSV files, using the pandas package to work with data frames, how to calculate summary information from a data frame, and a brief introduction to plotting. The last lesson demonstrates how to work with databases directly from Python. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to insure the proper setup of tools for an efficient workflow. These lessons assume no prior knowledge of the skills or tools. To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions. This lesson requires a working copy of Python. To most effectively use these materials, please make sure to install everything before working through this lesson. 2017-05-25

This lesson will teach you what relational databases are, how you can load data into them and how you can query databases to extract just the information that you need. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to insure the proper setup of...

SQL for Ecology https://tess.elixir-europe.org/materials/data-carpentry-sql-for-ecology This lesson will teach you what relational databases are, how you can load data into them and how you can query databases to extract just the information that you need. Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to insure the proper setup of tools for an efficient workflow. These lessons assume no prior knowledge of the skills or tools. To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions. This lesson requires a working copy of SQLite Manager for SQL. To most effectively use these materials, please make sure to install everything before working through this lesson. If you are teaching this lesson in a workshop, please see the Instructor notes. 2017-05-25
Found 0 upcoming events. Found 0 past events.
scraper updated Data Carpentry at 2016-10-18 08:58:08 UTC.
scraper updated Data Carpentry at 2016-10-14 03:02:04 UTC.
scraper updated Data Carpentry at 2016-10-13 03:02:06 UTC.
scraper updated Data Carpentry at 2016-10-12 03:02:04 UTC.
scraper updated Data Carpentry at 2016-10-11 03:02:14 UTC.
scraper updated Data Carpentry at 2016-10-10 12:21:40 UTC.
scraper updated Data Carpentry at 2016-10-10 09:49:22 UTC.
niall updated Data Carpentry at 2016-10-05 03:02:27 UTC.
niall updated Data Carpentry at 2016-10-04 03:02:27 UTC.
niall updated Data Carpentry at 2016-10-03 03:02:17 UTC.
niall updated Data Carpentry at 2016-10-02 03:02:23 UTC.
niall updated Data Carpentry at 2016-10-01 03:01:56 UTC.
niall updated Data Carpentry at 2016-09-30 03:01:54 UTC.
niall updated Data Carpentry at 2016-09-29 03:02:25 UTC.
niall updated Data Carpentry at 2016-09-28 03:02:24 UTC.
niall updated Data Carpentry at 2016-09-27 03:02:19 UTC.
niall updated Data Carpentry at 2016-09-26 03:02:22 UTC.
niall updated Data Carpentry at 2016-09-25 03:02:17 UTC.
niall updated Data Carpentry at 2016-09-24 03:02:15 UTC.
niall updated Data Carpentry at 2016-09-23 03:02:19 UTC.
niall updated Data Carpentry at 2016-09-22 16:47:22 UTC.
niall updated Data Carpentry at 2016-09-22 03:02:21 UTC.
niall updated Data Carpentry at 2016-09-21 03:02:23 UTC.
niall updated Data Carpentry at 2016-09-20 03:02:29 UTC.
niall updated Data Carpentry at 2016-09-19 03:02:22 UTC.
niall updated Data Carpentry at 2016-09-18 03:02:22 UTC.
niall updated Data Carpentry at 2016-09-17 03:02:17 UTC.
niall updated Data Carpentry at 2016-09-16 03:02:20 UTC.
niall updated Data Carpentry at 2016-09-15 03:02:18 UTC.
niall updated Data Carpentry at 2016-09-14 03:02:14 UTC.
niall updated Data Carpentry at 2016-09-13 14:24:55 UTC.
niall updated Data Carpentry at 2016-09-13 03:02:16 UTC.
niall updated Data Carpentry at 2016-09-12 14:56:47 UTC.
finn updated Data Carpentry at 2016-07-25 14:59:03 UTC.
finn updated Data Carpentry at 2016-07-25 14:53:59 UTC.
niall updated Data Carpentry at 2016-07-25 03:01:49 UTC.
niall updated Data Carpentry at 2016-07-24 03:01:48 UTC.
niall updated Data Carpentry at 2016-07-23 03:01:47 UTC.
niall updated Data Carpentry at 2016-07-22 03:01:48 UTC.
niall updated Data Carpentry at 2016-07-21 16:15:06 UTC.
niall updated Data Carpentry at 2016-07-21 16:00:20 UTC.
scraper updated Data Carpentry at 2016-05-12 03:00:42 UTC.
scraper updated Data Carpentry at 2016-05-11 03:00:42 UTC.
scraper updated Data Carpentry at 2016-05-10 15:17:13 UTC.
scraper updated Data Carpentry at 2016-05-05 10:10:41 UTC.
scraper updated Data Carpentry at 2016-05-05 03:00:40 UTC.
scraper updated Data Carpentry at 2016-05-04 03:00:32 UTC.
scraper updated Data Carpentry at 2016-05-03 03:00:29 UTC.
scraper updated Data Carpentry at 2016-05-02 03:00:29 UTC.
scraper updated Data Carpentry at 2016-05-01 03:00:31 UTC.
scraper updated Data Carpentry at 2016-04-30 03:00:30 UTC.
scraper updated Data Carpentry at 2016-04-29 03:00:31 UTC.
scraper updated Data Carpentry at 2016-04-28 03:00:31 UTC.
scraper updated Data Carpentry at 2016-04-27 03:00:31 UTC.
scraper updated Data Carpentry at 2016-04-26 03:00:32 UTC.
scraper updated Data Carpentry at 2016-04-25 03:00:33 UTC.
scraper updated Data Carpentry at 2016-04-24 03:00:32 UTC.
scraper updated Data Carpentry at 2016-04-23 03:00:30 UTC.
scraper updated Data Carpentry at 2016-04-22 03:00:36 UTC.
scraper updated Data Carpentry at 2016-04-21 03:00:36 UTC.
scraper updated Data Carpentry at 2016-04-20 03:00:39 UTC.
scraper updated Data Carpentry at 2016-04-19 03:00:37 UTC.
scraper updated Data Carpentry at 2016-04-18 03:00:32 UTC.
scraper updated Data Carpentry at 2016-04-17 03:00:33 UTC.
scraper updated Data Carpentry at 2016-04-16 03:00:35 UTC.
scraper updated Data Carpentry at 2016-04-15 03:00:34 UTC.
scraper updated Data Carpentry at 2016-04-14 03:00:34 UTC.
scraper updated Data Carpentry at 2016-04-13 03:03:48 UTC.
scraper updated Data Carpentry at 2016-04-12 03:04:06 UTC.
scraper updated Data Carpentry at 2016-04-08 03:03:56 UTC.
scraper updated Data Carpentry at 2016-04-07 15:08:50 UTC.
scraper created Data Carpentry at 2016-03-07 10:15:06 UTC.