README.md 9.85 KB
Newer Older
mdolling-gfz's avatar
add sqa    
mdolling-gfz committed
1
<!--
2
hifis-surveyval
mdolling-gfz's avatar
add sqa    
mdolling-gfz committed
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Framework to help developing analysis scripts for the HIFIS Software survey.

SPDX-FileCopyrightText: 2021 HIFIS Software <support@hifis.net>

SPDX-License-Identifier: GPL-3.0-or-later

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
-->

23
# HIFIS-Surveyval
Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
24
25
26

This project is used to develop analysis scripts for the HIFIS Software survey.

mdolling-gfz's avatar
mdolling-gfz committed
27
28
29
30
31
32
33
## Table of Content

* [Installation](#installation)
* [Getting Started](#getting-started)
* [Start Analysis from Command-Line-Interface](#start-analysis-from-command-line-interface)
* [Contribute with Own Analysis Scripts](#contribute-with-own-analysis-scripts)
* [Resources](#resources)
34
35
* [Author Information](#author-information)
* [Contributors](#contributors)
36
* [License](#license)
mdolling-gfz's avatar
mdolling-gfz committed
37

Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
38
39
## Getting Started

40
41
42
43
44
The project's documentation contains a section to help you as a 
[user of the analysis scripts](#getting-for-users) 
to run the analysis scripts or as a 
[developer of the framework](#getting-started-for-developers)
to set up the development environment.
Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
45

46
### Getting Started for Users
mdolling-gfz's avatar
mdolling-gfz committed
47

48
49
50
51
#### Installation

To install the package locally, you can use 
[Pip](https://pip.pypa.io/en/stable/).
mdolling-gfz's avatar
mdolling-gfz committed
52
53

```shell
54
pip install hifis-surveyval
mdolling-gfz's avatar
mdolling-gfz committed
55
56
```

57
58
After the installation, you can use the tool from the command line with 
`hifis-surveyval --help`.
mdolling-gfz's avatar
mdolling-gfz committed
59

60
### Getting Started for Developers
mdolling-gfz's avatar
mdolling-gfz committed
61

62
#### Installation
mdolling-gfz's avatar
mdolling-gfz committed
63

64
65
66
67
To install the package locally, you can use 
[Poetry](https://python-poetry.org/).

#### Using Poetry
68
69

If you want to actively contribute changes to the project, you are required to
70
also install the development packages alongside the framework.
mdolling-gfz's avatar
mdolling-gfz committed
71
72

```shell
73
74
git clone https://gitlab.hzdr.de/hifis/surveys/hifis-surveyval.git
cd hifis-surveyval
mdolling-gfz's avatar
mdolling-gfz committed
75
76
poetry install
```
77

78
79
After the installation, you can use the tool from the command line with 
`poetry run hifis-surveyval --help`
80

81
Poetry installs some packages that are required for performing quality checks.
82
Usually they are also performed via GitLab CI, but can also be executed locally.
83
84
85
86

It is common practice to run some checks locally before pushing them online.
Therefore, execute below commands:
```console
87
88
$ # Order your imports
$ isort -rc .
89
90
91
$ make lint
```

92
93
94
95
The following documentation references the pip installation.
You can use the same commands with a poetry installation, if you prefix your 
commands with `poetry run COMMAND`.

96
97
98
99
100
101
102
## Start Analysis from Command-Line-Interface

The survey analysis package is a program to be executed on the
Command-Line-Interface (CLI).

### Quick Start Example: Run Analysis

103
104
105
106
107
108
Due to sensible defaults of the project's configurations you need to have the 
analysis scripts as well as metadata and data files in certain locations
in order to run the survey analysis.
Please put your analysis scripts into a sub-folder called _scripts_.
Make sure that the file _meta.yml_ is put into sub-folder _metadata_.
Finally, copy the data-CSV-file for example from the 
109
110
111
[wiki page](https://gitlab.hzdr.de/hifis/survey-about-current-development-practice/-/wikis/home) 
of the associated GitLab project 
[Survey about current Development Practice](https://gitlab.hzdr.de/hifis/survey-about-current-development-practice)
112
113
into a central location like a _data_ sub-folder and tell the program the 
path to that data file.
114

115
116
117
Now you can do the following to start the survey analysis from the CLI:

```shell script
118
hifis-surveyval analyze data/<data_file_name>.csv
119
120
```

121
122
123
The output is then put into a sub-folder within the folder _output_ 
which is named after the stamp of the current date-time if not specified 
differently.
124

125
126
127
128
129
130
131
132
**Caution:** 
Depending on the Operating System used an issue with the file 
encoding might occur.
There might be data-CSV-files around which are encoded with `UTF-8-BOM`
which causes errors when read in on Windows OS.
In this case you need to change the encoding to `UTF-8` before running
the survey analysis.

133
134
### Flags

135
The program accepts two flags:
136
137
138
139
140
141
142
143
144
145
146

1. Help flag
2. Verbosity flag

#### Help flag

Calling the program with the _help_-flag is the first thing to do
when being encountered with this program.
It outputs a so-called _Usage_-message to the CLI:

```shell script
147
$ hifis-surveyval --help
148
149
```

150
151
152
153
Please issue this command on the CLI and read the detailed 
_Usage_-message before continuing with reading the documentation
of the _Usage_-message here.

154
155
156
157
158
159
160
#### Verbosity flag

The _verbosity_-flag can be provided in order to specify the verbosity
of the output to the CLI.
This flag is called `--verbose` or `-v` for short:

```shell script
161
hifis-surveyval --verbose <COMMAND>
162
163
```
```shell script
164
hifis-surveyval -v <COMMAND>
165
166
167
168
169
170
```

The verbosity of the output can be increased even more 
by duplicating the flag `--verbose` or `-v` up to two times:

```shell script
171
hifis-surveyval --verbose --verbose --verbose <COMMAND>
172
173
```
```shell script
174
hifis-surveyval -vvv <COMMAND>
175
176
```

177
### Commands
178

179
There are three different commands implemented which come with its own set of
180
flags and parameters:
181

182
183
184
1. Command _version_
2. Command _init_
3. Command _analyze_
185

186
187
188
#### Command _version_
 
The `version` command outputs the version number of this CLI-program like so:
189
190

```shell script
191
hifis-surveyval version
192
193
```

194
#### Command _init_
195

196
197
198
199
Before you start the analysis you may want to change the defaults of the
configuration variables.
In order to do so, you can create a configuration file that is named 
_hifis-surveyval.yml_ by issuing the _init_ command:
200
201

```shell script
202
hifis-surveyval init
203
204
```

205
This file contains the following information:
206
207

```
208
209
210
211
212
METADATA: metadata/meta.yml
OUTPUT_FOLDER: output
OUTPUT_FORMAT: PNG
SCRIPT_FOLDER: scripts
SCRIPT_NAMES: []
213
214
```

215
216
First of all, each analysis needs metadata about the questions asked in the
survey and answers that participants may give.
217
218
This metadata file is by default located in a folder called _metadata_
and named _meta.yml_.
219

220
Second, you may specify the output folder which is named _output_ by default.
221

222
Third, you may prefer a specific output format like _PDF_, _PNG_, _SVG_ or 
223
224
_SCREEN_.
The default value is _PNG_.
225
226
Note: Be aware that other output formats like text or markdown files may be 
created, which depends largely upon the implementation of the analysis scripts.
227

228
229
Fourth, you may specify the folder which contains the analysis scripts, which
is the _scripts_ folder by default.
230

231
232
Finally, you may select a subset of the analysis scripts available as a list
that ought to be executed.
233
This list is empty by default, which means, all scripts are executed.
234
235
236
237

#### Command _analyze_

The more interesting command is the `analyze` command
238
which comes with a _data_-parameter.
239
240
241
242
243
The _data_-parameter can _not_ be omitted and need to be given explicitly
in order to be able to start the analysis.
This is an example of how to do the analysis:

```shell script
244
hifis-surveyval analyze data/<data_file_name>.csv
245
246
247
248
```

## Contribute with Own Analysis Scripts

249
### Essential Requirements for Developing Own Analysis Scripts
250
251

As you might have read in the previous sections the actual analysis scripts 
252
reside in a specific folder called `scripts`.
253
All scripts in that folder will be automatically discovered by the package 
254
`hifis-surveyval` when running the analysis.
255
In order that the program recognizes the scripts in that folder as
256
analysis scripts they need to fulfill the following two criteria:
257
258
259
260
261
262
263
264

1. The filename need to end on `.py`.
2. The file need to contain a function called `run` without any parameters.

```python
"""
A dummy script for testing the function dispatch

265
.. currentmodule:: hifis_surveyval.scripts.dummy
266
267
268
269
270
271
272
.. moduleauthor:: HIFIS Software <software@hifis.net>
"""

def run():
    print("Example Script")
```

273
If both requirements are satisfied the program will execute the `run`-functions
274
of the analysis scripts in an arbitrary order.
275

276
### File-System Structure of the Core Component
277
278

```shell script
279
280
$ tree hifis_surveyval/
hifis_surveyval/
281
282
├── answer.py
├── cli.py
283
284
285
286
├── core
│   ├── environment.py
│   ├── __init__.py
│   └── settings.py
287
288
289
├── data.py
├── dispatch.py
├── globals.py
290
291
├── __init__.py
├── plot.py
292
├── question.py
293
└── util.py
294
295
```

Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
296
297
## Resources

298
Below are some handy resource links:
Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
299

300
301
* [Project Documentation](TODO)
* [Click](https://click.palletsprojects.com/en/7.x) is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary.
Huste, Tobias (FWCC) - 111645's avatar
Huste, Tobias (FWCC) - 111645 committed
302
303
304
305
* [Sphinx](http://www.sphinx-doc.org/en/master/) is a tool that makes it easy to create intelligent and beautiful documentation, written by Geog Brandl and licnsed under the BSD license.
* [pytest](https://docs.pytest.org/en/latest/) helps you write better programs.
* [GNU Make](https://www.gnu.org/software/make/) is a tool which controls the generation of executables and other non-source files of a program from the program's source files.

306
307
308
309
310
311
312
313
314
315
316
## Author Information

_HIFIS-Surveyval_ was created by 
[HIFIS Software Services](https://software.hifis.net/).

## Contributors

We would like to thank and give credits to the following contributors of this
project:

* Be the first to be named here!
317
318
319
320
321
322
323
324
325
326

## License

Copyright © 2021 HIFIS Software <support@hifis.net>

This work is licensed under the following license(s):
* Everything else is licensed under [GPL-3.0-or-later](LICENSES/GPL-3.0-or-later.txt)

Please see the individual files for more accurate information.

327
> **Hint:** We provided the copyright and license information in accordance to the [REUSE Specification 3.0](https://reuse.software/spec/).