At the moment we are facing issues with a certain GitLab CI Runner (hifis-runner-manager-1) that execute your GitLab CI pipelines. We are working on it to bring them back to service again. Please excuse the inconveniences.

README.md 6.18 KB
Newer Older
Timm Schoening's avatar
Timm Schoening committed
1
> **Preamble:** We strive to make marine image data [FAIR](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos/-/blob/master/FAIR-marine-images.md). We maintain [data profiles](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos) to establish a common language of marine imagery, we develop - here in this repository - best-practice [operating procedures](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures) for handling marine images and we develop [software tools](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/software/mar-iqt) to apply the vocabulary and procedures to marine imagery.
Timm Schoening's avatar
Timm Schoening committed
2

Timm Schoening's avatar
Timm Schoening committed
3
## Preface
Timm Schoening's avatar
Timm Schoening committed
4
5
6
Publishing marine image data in a FAIR and open way requires data curation for both image data and image metadata. These quality control steps need to follow common standard operating procedures (SOPs) to facilitate joint data interpretation. This repository mainly collects SOPs for the QA/QC steps between acquisition and publication. It also contains some example SOPs on image acquisition for the interested user. It does not yet provide operational steps for the publication phase (e.g. physical file transfer to Pangaea). The SOPs vary in granularity and not all may be applicable to your usecase.

To make full use of the [MarIQT software tools](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/software/mar-iqt) that implement some of the SOPs functionalities, you need to follow the guidelines on [file formats](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/blob/master/file-formats.md), adhere to the [folder structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/blob/master/folder-structure.md) and the [provenance documentation](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/blob/master/provenance-documentation.md) (which is done for you automatically while using MarIQT).
Timm Schoening's avatar
Timm Schoening committed
7

Timm Schoening's avatar
Timm Schoening committed
8
9
# Standard operating procedures (SOPs)
This section is your starting point for exploring the MareHub AG V/I SOP documents. This readme provides some background and context information on how visualizations of the SOPs are structured and how the data workflow is explained. It is like an SOP for SOPs. You can find the existing marine imaging SOPs [here](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/tree/master/SOPs).
Timm Schoening's avatar
Timm Schoening committed
10

Timm Schoening's avatar
Timm Schoening committed
11
12
13
### General workflows
![MareHub AG Videos/Images SOP documentation: general workflow](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/AGVI_SOP-documentation-1-workflow.jpeg "MareHub AG Videos/Images SOP documentation: general workflow")
SOPs generally describe how processes create or modify entities which are managed in infrastructure.
Timm Schoening's avatar
Timm Schoening committed
14

Timm Schoening's avatar
Timm Schoening committed
15
16
17
### A project's data workflow
![MareHub AG Videos/Images SOP documentation: project workflow](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/AGVI_SOP-documentation-2-project.jpeg "MareHub AG Videos/Images SOP documentation: project workflow")
In terms of research data, this is expressed by a data creation process that produces a data set entity which is managed in a data repository.
Timm Schoening's avatar
Timm Schoening committed
18

Timm Schoening's avatar
Timm Schoening committed
19
20
21
### Actors in a workflow
![MareHub AG Videos/Images SOP documentation: project workflow with actors](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/AGVI_SOP-documentation-3-project-actors.jpeg "MareHub AG Videos/Images SOP documentation: project workflow with actors")
Processes are conducted by an actor - in this case researchers - and similarly infrastructure is operated by actors - in this case a research data management (RDM) team. Infrastructure is further characterized by whether it is publicly accessible and whether it is machine-accessible. Accessibility by humans is always expected.
Timm Schoening's avatar
Timm Schoening committed
22

Timm Schoening's avatar
Timm Schoening committed
23
24
25
### Documentation entities
![MareHub AG Videos/Images SOP documentation: project workflow with documentation](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/AGVI_SOP-documentation-4-project-documentation.jpeg "MareHub AG Videos/Images SOP documentation: project workflow with documentation")
Processes and entities can be accompanied by one or many documentation entities. These can take various forms, depending on use cases, SOPs, software tools used, etc. These documentation entities may be seen by some as just another data entity (_"one wo:mans data is another wo:mans metadata"_), we like to keep it separate. The format of the documentation entities (file format, information content, etc.) is defined by one or several actors. In case the file format is machine-readable, the documentation entity will be marked as such. Like data entities, documentation entities cannot be the end of a workflow. They needs to be further processed or be placed into an infrastructure. This infrastructure might also be access-restricted.
Timm Schoening's avatar
Timm Schoening committed
26

Timm Schoening's avatar
Timm Schoening committed
27
28
29
### Status information
![MareHub AG Videos/Images SOP documentation: project workflow with status information](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/AGVI_SOP-documentation-5-project-status.jpeg "MareHub AG Videos/Images SOP documentation: project workflow with status information")
Of course, all SOPs should represent the best-case scenario and of course all their components should be in place and work as described. But as the marine imaging community is still developing their best-practices, some of the elements of SOPs are still under development. In that case, this is color-coded in the workflow figures: either there is no solution for the entire concept (of a process, entity, infrastructure or documentation) or there is one but it is not commonly supported, operated, executed or maintained.
Timm Schoening's avatar
Timm Schoening committed
30
31
32


# Cheatsheet of the graphical elements
Timm Schoening's avatar
Timm Schoening committed
33
![MareHub AG Videos/Images SOP documentation: cheatsheet of the graphical elements](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures/-/raw/master/SOPs/graphics/MareHubSOP_graphics_cheatsheet.png "MareHub AG Videos/Images SOP documentation: cheatsheet of the graphical elements")