Commit a8314648 authored by Timm Schoening's avatar Timm Schoening
Browse files

Add new file

parent d0048a7a
# File format recommendations for image and video SOPs
We recommend to store all relevant information on the image curation process in a human and machine-readable format. For structured information, we recommend the *.yaml format which is compact (i.e. helps with fast transfer across networks), well-formed (i.e. helps with automating curation tasks), and natively supported by OSs and editors. For tabular data, we recommend to use *.txt files.
## *.txt files
Encoding ASCII information in *.txt files still needs a clear format description. The following is being used throughout the MareHub AGV/I tools:
- The file may contain optional comment lines anywhere. These have to begin with the # symbol. Content of those lines is ignored while reading the files.
- The first non-comment line of a file must contain the column headers.
- Columns are split by the TAB character (\t, chr(9)).
- Rows are split by the line feed character (\n, chr(10)).
- Columns that correspond to the MareHub AG V/I metadata vocabulary have to exactly match the vocabulary terms.
- The file is ASCII/UTF8 encoded plain text.
## *.yaml files
- Need to follow the general *.yaml specification.
- Key or values that correspond to the MareHub AG V/I metadata vocabulary have to exactly match the vocabulary terms.
- The file is ASCII/UTF8 encoded plain text.
# Examples
## Example *.yaml file
```
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg:
image-longitude: -123.854637
image-latitude: 42.133426
image-depth: 4230.3
image-pixel-per-millimeter: 12.1
image-meters-above-ground: 1.3
image-coordinate-uncertainty: 4.2
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg:
image-longitude: -123.854638
image-latitude: 42.133427
image-depth: 4230.4
image-pixel-per-millimeter: 12.1
image-meters-above-ground: 1.4
SO268-1_21-1_GMR_CAM-23_20190513_131417.jpg:
image-longitude: -123.854639
image-latitude: 42.133428
image-depth: 4230.5
image-pixel-per-millimeter: 12.0
image-meters-above-ground: 1.5
```
## Example *.txt file
```
# This is a comment
image-filename image-longitude image-latitude image-depth image-pixel-per-millimeter image-meters-above-ground
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg -123.854637 42.133426 4230.3 12.1 1.3
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg -123.854638 42.133427 4230.4 12.1 1.4
# There may be another comment here
SO268-1_21-1_GMR_CAM-23_20190513_131417.jpg -123.854639 42.133428 4230.5 12.0 1.5
```
# Important files in the image curation process
There are some information in the image curation process that need to be available to facilitate FAIRness of the imagery. In the end, it is the [iFDO](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos/-/blob/master/MareHub_AGVI_iFDO.md) that provides this functionality but on the way to creating the iFDO the curation workflow may create intermediate information files. How those look is very much up to your individual data curation processes and tools. Anyhow we define some essential files here which are used in the software packages provided by the AG V/I.
## Important *.yaml files
- Project curation file (TODO)
- [iFDO](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos/-/blob/master/MareHub_AGVI_iFDO.md)
- [pFDO](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos/-/blob/master/MareHub_AGVI_pFDO.md)
- Processing [provenance](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#provenance-documentation) file
- Image content feature file (TODO)
## Important *.txt files
Note that its not required that you create all these files. In case you can directly create the [iFDO](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/metadata-profiles-fdos/-/blob/master/MareHub_AGVI_iFDO.md) that is also fine. We just found that its helpful to keep these intermediate files available as well.
### Still image navigation file
```
image-filename image-longitude image-latitude {image-depth,image-altitude}
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg -123.854637 42.133426 4230.3
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg -123.854638 42.133427 4230.4
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-navigation.txt`
### Moving image navigation file
```
image-filename image-second image-longitude image-latitude {image-depth,image-altitude}
SO268-1_21-1_GMR_CAM-42_20190513_111213.mp4 0 -123.854637 42.133426 4230.3
SO268-1_21-1_GMR_CAM-42_20190513_111213.mp4 1 -123.854638 42.133427 4230.4
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-navigation.txt`
### Image UUID file
```
image-filename image-uuid
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 9999ba88-1a20-4efe-a0ac-6b4233490ad6
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 1c266c00-33e7-4e69-bc9a-f90fb1bce6d0
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-uuids.txt`
### Image acquisition start time file
```
image-filename image-datetime
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 2019-05-13 13:14:15.0000
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 2019-05-13 13:14:16.0000
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-start-times.txt`
### Image scaling file
```
image-filename image-pixel-per-millimeter
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 12.1
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 12.1
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-scaling.txt`
### Image SHA256 hash file
```
image-filename hash-sha256
SO268-1_21-1_GMR_CAM-23_20190513_131415.jpg 83f30eb35d1325c44c85fba0cf478825c0a629d20177a945069934f6cd07e087
SO268-1_21-1_GMR_CAM-23_20190513_131416.jpg 27c3585560f93a78995a038b5970a002315d2b525b999a24a1d13a5c5e5520f6
...
```
_When working with the AGV/I [data structure](https://gitlab.hzdr.de/datahub/marehub/ag-videosimages/standard-operating-procedures#data-structure) place this file at:_ `/<volume>/<project>/<event>/<sensor>/intermediate/<event>_<sensor>_image-hashes.txt`
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment