Researchers often spend a significant amount of time on data wrangling tasks, such as reformatting, cleaning, and integrating data from different sources. Despite the availability of software tools, they often end up with difficult-to-reuse workflows that require manual steps. Omnipy is a new Python library that offers a systematic and scalable approach to research data and metadata wrangling. It allows researchers to import data in various formats and continuously reshape it through typed transformations. For large datasets, Omnipy seamlessly scales up local test jobs and provides persistent access to the data state at every step.

This workshop will provide down-to-earth tutorials and examples to help data scientists from any field make use of Omnipy to wrangle real-world datasets into shape.

The workshop is divided into three parts:

1. The first part will introduce the concepts of models, datasets and tasks in Omnipy through small examples. We will also touch upon Python-type hints and Pydantic models as needed, as these are important building blocks for Omnipy.

2. In the second part, the participants will be provided with a rough example dataset that requires cleaning. As a hands-on exercise, the participant will carry out step-wise parsing and shaping of the data to make it comply with a specified metadata schema.

3. In the last part, the participants will be introduced to the metadata mapping functionalities in Omnipy and will be led through another hands-on exercise to set up a transformation that maps the data from one metadata schema to another. This half-day workshop will form the knowledge basis for an intermediate-level workshop after lunch that will focus more on defining and orchestrating data flows, including integrating with data sources and deploying flows onto external compute resources.