FWCC issueshttps://codebase.helmholtz.cloud/groups/fwcc/-/issues2020-03-20T08:26:25+01:00https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/8Check possibility of autoscaling using SLURM2020-03-20T08:26:25+01:00Huste, TobiasCheck possibility of autoscaling using SLURMThe new cluster will use SLURM as workload manager. It would be ideal to integrate CI jobs with SLURM. Like this, nodes could be used for traditional jobs as well as CI jobs.
This feature is currently not implemented in gitlab-runner. B...The new cluster will use SLURM as workload manager. It would be ideal to integrate CI jobs with SLURM. Like this, nodes could be used for traditional jobs as well as CI jobs.
This feature is currently not implemented in gitlab-runner. But there exist several feature requests and also an open merge request on https://gitlab.com/gitlab-org/gitlab-runner.
Following resources might be helpful:
- https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/667/diffs
- https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/667/diffs
- https://gitlab.com/gitlab-org/gitlab-runner/issues/47
Documentation:
- [GitLab Runner execturs](https://docs.gitlab.com/runner/executors/README.html)
- [Docker Machine Drivers](https://docs.docker.com/machine/drivers/)https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/55MacOS GitLab Runner2019-07-26T10:16:05+02:00Huste, TobiasMacOS GitLab RunnerFor building iOS projects, and also test building software on MacOS, we should provide runners basing on MacOS.
A first test runner is now available using the tags `macos` and `ios`.
Tasks:
- [ ] Test the runner using the iOS project o...For building iOS projects, and also test building software on MacOS, we should provide runners basing on MacOS.
A first test runner is now available using the tags `macos` and `ios`.
Tasks:
- [ ] Test the runner using the iOS project of the Azubis
- [ ] Test the runner with the project of FWKT
- [ ] Decide how to continuehttps://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/issues/1Security concept2020-12-14T13:26:35+01:00David Paped.pape@hzdr.deSecurity conceptThe program needs a security concept.
For now, all jobs are run in sibling directories without any encapsulation/containerization/... This means that a job, running concurrently with other jobs, can easily access their data. This is a p...The program needs a security concept.
For now, all jobs are run in sibling directories without any encapsulation/containerization/... This means that a job, running concurrently with other jobs, can easily access their data. This is a problem when dealing with jobs that contain sensitive information that must not be made public.
A possible solution might be a pool of cluster users that run incoming jobs in a round-robin manner. Their working directories can be secured using UNIX/LDAP permissions.https://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/issues/4Notify SlurmJob::OutputFile of changes in file2019-12-16T11:35:51+01:00David Paped.pape@hzdr.deNotify SlurmJob::OutputFile of changes in fileAt the moment SlurmJob::OutputFile outputs new lines every 100 milliseconds. Can this be improved by using inotify or similar methods?At the moment SlurmJob::OutputFile outputs new lines every 100 milliseconds. Can this be improved by using inotify or similar methods?https://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/issues/6How to access group quotas2020-01-28T10:24:08+01:00Huste, TobiasHow to access group quotasE.g. FWDC has additional quotas. How to make them available via CI/CD on hemera?E.g. FWDC has additional quotas. How to make them available via CI/CD on hemera?https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/78Job's log exceeded limit of 4194304 bytes2020-02-04T15:08:34+01:00Evdokimov, Dr. Ilya (FWDC) - 141789Job's log exceeded limit of 4194304 bytesIs it a default limitation? Can we make it greater?Is it a default limitation? Can we make it greater?Huste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/84In some projects jobs with `performance` tag imediatly get stucked, in some o...2020-05-28T07:41:31+02:00Schlegel, Dr. Fabian (FWDC) - 107795f.schlegel@hzdr.deIn some projects jobs with `performance` tag imediatly get stucked, in some others it worksgets stucked immediatly: https://gitlab.hzdr.de/openfoam/dockerusers/schleg95/-/jobs/96599
works: https://gitlab.hzdr.de/openfoam/Docker/-/jobs/96066
Is there a rule behind?gets stucked immediatly: https://gitlab.hzdr.de/openfoam/dockerusers/schleg95/-/jobs/96599
works: https://gitlab.hzdr.de/openfoam/Docker/-/jobs/96066
Is there a rule behind?Huste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/issues/13Set time limit in Slurm job (automatically)2020-03-17T10:29:31+01:00David Paped.pape@hzdr.deSet time limit in Slurm job (automatically)This would allow for faster access to the cluster if a small time limit is set.
If the user doesn't specify a limit in their `.gitlab.yml` it can be inferred from their projects settings. Unfortunately, there is no variable set by GitLa...This would allow for faster access to the cluster if a small time limit is set.
If the user doesn't specify a limit in their `.gitlab.yml` it can be inferred from their projects settings. Unfortunately, there is no variable set by GitLab that exposes this information.https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/94MR suggestions always get applied to branch tip2020-05-14T23:29:27+02:00Schmerler, Dr. Steve (FWCC) - 131252MR suggestions always get applied to branch tipDear GitLab gurus. We observed the following behavior [just
recently](https://gitlab.hzdr.de/haicu/operations/-/merge_requests/18/diffs). I have
tested this separately in a private repo as well. Can anyone confirm this? Is
this a known i...Dear GitLab gurus. We observed the following behavior [just
recently](https://gitlab.hzdr.de/haicu/operations/-/merge_requests/18/diffs). I have
tested this separately in a private repo as well. Can anyone confirm this? Is
this a known issue? Is there a solution? Thanks!
Say you have created a branch `feature-wip` and a MR into `master`.
When you, being on the GitLab diff discussion page of the MR
* edit a file in that branch interactively
* apply a [code
suggestion](https://gitlab.hzdr.de/help/user/discussions/index.md#suggest-changes)
then a commit will be created in `feature-wip`, with a commit message such as
```
Apply suggestion to <filename>
```
The problem: the commit will *always* be made on top of the branch's tip, i.e.
the most recent commit, and not the commit at which the diff discussion
started. This is no problem, unless someone (or you) pushes new commits to
`feature-wip`, moving tip forward. Now all not yet applied code suggestions refer to
an *older* version of the file, but will be applied onto the new tip, which may
lead to merge conflicts.
In the particular case above, we even saw a GitLab message saying something like "this ... referes to
an older version of this file ...", with links to the versions and everything.
To make things more visual, this is what happens:
```
* aab4893 (HEAD -> feature-wip, origin/feature-wip) Apply suggestion to foo (GitLab GUI)
|
* b02d02f Pushed
|
* 3edc93f Apply suggestion to foo (GitLab GUI)
|
* f9c2a59 Apply suggestion to foo (GitLab GUI)
|
* 034b71f WIP: foo
|
...
|
* 7192849 initial commit
```
In this situation, GitLab could instead create new branch by
applying onto the old tip `3edc93f`
```
* b02d02f (origin/feature-wip, feature-wip) Pushed
|
| * bec979c (HEAD -> gitlab-automatic-branch-xzy2342) Apply suggestion to foo (GitLab GUI)
|/
|
* 3edc93f Apply suggestion to foo (GitLab GUI)
|
* f9c2a59 Apply suggestion to foo (GitLab GUI)
|
* 034b71f WIP: foo
|
...
|
* 7192849 initial commit
```
Then we could use normal git to merge those, or GitLab could automatically
merge and only create a branch when it cannot merge without conflict.
So: we have a file-locking-like situation here. Until all the MR's discussion
items are resolved, nobody can (should not) commit to that branch. Bummer.
Huste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/98Login via Helmholtz AAI not working with GitLab 13.12020-10-15T07:24:15+02:00Huste, TobiasLogin via Helmholtz AAI not working with GitLab 13.1With the upgrade to GitLab 13.1 the login via Helmholt AAI was failing. In order to create a useful issue in the respective projects this issue needs some further investigation.
## Corresponding Sentry Error
https://vlsstack1.fz-rossend...With the upgrade to GitLab 13.1 the login via Helmholt AAI was failing. In order to create a useful issue in the respective projects this issue needs some further investigation.
## Corresponding Sentry Error
https://vlsstack1.fz-rossendorf.de/fwcc/gitlab/issues/2320/ (with login)
https://vlsstack1.fz-rossendorf.de/share/issue/5346cec47ab944818aab786f15204d38/ (HZDR internal)
## Current workaround
Manually downgrading the Ruby Gem `omniauth_openid_connect` from `0.3.5` to `0.3.3` in `/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/omniauth_openid_connect-0.3.5/` fixes the issue temporarily.
## Places to dig deeper
- https://github.com/m0n9oose/omniauth_openid_connect, especially https://github.com/m0n9oose/omniauth_openid_connect/blob/ef2942047c866993d8323115c419371d75f05a60/lib/omniauth/strategies/openid_connect.rb#L229
- https://github.com/nov/openid_connect/blob/a76c9c84458a0d8c3ec3783ec6c815c285c05d91/lib/openid_connect/response_object/id_token.rb#L70
- https://github.com/nov/json-jwtHuste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/issues/17Breaking changes in GitLab 17.02023-03-02T11:12:56+01:00Huste, TobiasBreaking changes in GitLab 17.0GitLab Runner ~~14.0~~ 17.0 will introduce breaking changes that also require this implementation to be updated. In principle, the step called `build_script` so far will be renamed to `step_script`.
This was already unintentionally intr...GitLab Runner ~~14.0~~ 17.0 will introduce breaking changes that also require this implementation to be updated. In principle, the step called `build_script` so far will be renamed to `step_script`.
This was already unintentionally introduced in runner version `13.1.0` and fixed in `13.1.1` with this Merge Request: https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2227
Update: Now this is apparently due in version 17.0. See output of gitlab-runner:
```
WARNING: Starting with version 17.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/groups/gitlab-org/-/epics/6112
```
[GitLab epic here.](https://gitlab.com/groups/gitlab-org/-/epics/6112)2021-04-21https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/99Provide Mattermost push notification service2020-07-13T08:42:22+02:00Huste, TobiasProvide Mattermost push notification serviceProviding a self-hosted push notification service would allow us, to send more useful content as push notifications to mobile devices. This would require us to self-host such a service for iOS and Android (https://developers.mattermost.c...Providing a self-hosted push notification service would allow us, to send more useful content as push notifications to mobile devices. This would require us to self-host such a service for iOS and Android (https://developers.mattermost.com/contribute/mobile/push-notifications/service/).
- [ ] Decision, if that should be done?
- [ ] Does HZDR already have Android and/or iOS developer accounts?
Please also compare to https://www.hzdr.de/db/Support.SupPkg.PData?PMSNR=10832.https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/103Build docker image on self-hosted runner2020-11-20T08:43:13+01:00Ehrig, Simeon (FWKT) - 123249Build docker image on self-hosted runnerIs it possible to build a container on a self-made runner? At the moment we use public runners to build our container for the CI. Unfortunately the wall time of a single job has been reduced to 1h, which is not enough for our jobs (we ne...Is it possible to build a container on a self-made runner? At the moment we use public runners to build our container for the CI. Unfortunately the wall time of a single job has been reduced to 1h, which is not enough for our jobs (we need at least 2h).
So it would be really helpful if we could use our runners to build the container, because they need less time to build and have a higher wall time.
P.S. I have already tried to build a container on our system, without success: https://gitlab.com/hzdr/crp/gitlab-ci-test/-/tree/selfHostedDockerBuilder/self-hosted-Docker-Builderhttps://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/104Publish brutenn2020-12-03T16:32:54+01:00Steinbach, Dr. rer. nat. Peter (FWCC) - 120595Publish brutennI'd like to publish [`brutenn`](https://gitlab.hzdr.de/haicu/internal-projects/brutenn). For this, I'd love to move the repo to `https://github.com/helmholtz-ai` and publish wheels on PyPI.
If support for this is available, that would ...I'd like to publish [`brutenn`](https://gitlab.hzdr.de/haicu/internal-projects/brutenn). For this, I'd love to move the repo to `https://github.com/helmholtz-ai` and publish wheels on PyPI.
If support for this is available, that would be great. Otherwise, please let me know what I have to do to make the publication and ROBIS happy.https://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/111internal Tag not working as expected2021-07-21T12:59:49+02:00Schlegel, Dr. Fabian (FWDC) - 107795f.schlegel@hzdr.deinternal Tag not working as expectedI have a few jobs that update code on hemera2. So far they require the internal tag and this would force gitlab to put it on the `gitlab-runner-manager.hemera` Runners: https://gitlab.hzdr.de/openfoam/fwdc/Developments/-/jobs/330008 Howe...I have a few jobs that update code on hemera2. So far they require the internal tag and this would force gitlab to put it on the `gitlab-runner-manager.hemera` Runners: https://gitlab.hzdr.de/openfoam/fwdc/Developments/-/jobs/330008 However, since a while such jobs with the `internal` tag can be processed by `hifis-runner-manager-1` which results in a connection time out when logging in to hemera (python fabric): https://gitlab.hzdr.de/openfoam/fwdc/Developments/-/jobs/330082 Is the tag still valid?Huste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/114Have "Help" landing page also link to HIFIS Service documentation2021-09-28T09:48:36+02:00Guido JuckelandHave "Help" landing page also link to HIFIS Service documentationThe [help landing page](https://gitlab.hzdr.de/help) currently only has a link to this issue tracker. It should also include a reference to the [service documentation](https://hifis.net/doc/software/gitlab/getting-started/).
Especially ...The [help landing page](https://gitlab.hzdr.de/help) currently only has a link to this issue tracker. It should also include a reference to the [service documentation](https://hifis.net/doc/software/gitlab/getting-started/).
Especially on the login page the "help" link is much more prominent then the documentation link. It was the first link I clicked when checking for information on how to use the AAI and such as I was answering an email from a colleague.Huste, TobiasHuste, Tobiashttps://codebase.helmholtz.cloud/fwcc/HZDR-GitLab/-/issues/116Failure to access GitLab Docker registry on shared-hzdr-*2022-03-22T16:25:21+01:00Paul SkopnikFailure to access GitLab Docker registry on shared-hzdr-*The shared-hzdr-* runners (not: hifis-runner-manager-1, gitlab-runner-manager.hemera) have problems when trying to access the Docker Registry integrated with GitLab.
From a job log:
```
$ docker login "$REGISTRY" --username "$REGISTRY_U...The shared-hzdr-* runners (not: hifis-runner-manager-1, gitlab-runner-manager.hemera) have problems when trying to access the Docker Registry integrated with GitLab.
From a job log:
```
$ docker login "$REGISTRY" --username "$REGISTRY_USER" --password "$REGISTRY_PASSWORD"
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
error during connect: Post "http://docker:2375/v1.24/auth": dial tcp: lookup docker on 149.220.4.2:53: no such host
```
The `$REGISTRY` variable is set to `$CI_REGISTRY` (a [predefined variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) in .gitlab-ci.yml via `variables` stanza.
Its value should be `registry.hzdr.de` but appears to be `docker:2375`.
Examples:
* [Job #556633](https://gitlab.hzdr.de/kit-scc-sdm/onlinestorage/httpd-webdav/docker-basics/-/jobs/556633) fails on shared-hzdr-2
* [Job #556636](https://gitlab.hzdr.de/kit-scc-sdm/onlinestorage/httpd-webdav/docker-basics/-/jobs/556636) succeeds on hifis-runner-manager-1
If there is need, I can build a public project for debugging.https://codebase.helmholtz.cloud/fwcc/statistics-collector/-/issues/24Refactor Graphql Queries2022-06-28T20:54:47+02:00Sultova, Neda Miroslavova (FWCC) - 138965Refactor Graphql QueriesThe following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386194):
> The way you build queries here needs some thorough a...The following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386194):
> The way you build queries here needs some thorough attention.
>
> * Use string interpolation consistently
> * Construct the segments separately and then concatenate them
> * Are there segments that are fixed and can be re-used?
> * Can queries be represented as objects and is there a way to formalize their translation into a string?
> * Could this also be helpful if you would want to build other queries in the future?
> * Revisit the name of the `build_query`-functionSultova, Neda Miroslavova (FWCC) - 138965Sultova, Neda Miroslavova (FWCC) - 138965https://codebase.helmholtz.cloud/fwcc/statistics-collector/-/issues/27Refactor query operations2023-09-05T08:55:35+02:00Sultova, Neda Miroslavova (FWCC) - 138965Refactor query operationsThe following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386215):
> These are the same as in the CI-Jobs class… Why woul...The following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386215):
> These are the same as in the CI-Jobs class… Why would anyone duplicate this code? Suspicious… :thinking:Sultova, Neda Miroslavova (FWCC) - 138965Sultova, Neda Miroslavova (FWCC) - 138965https://codebase.helmholtz.cloud/fwcc/statistics-collector/-/issues/30Add auth into own class?2022-05-31T17:09:26+02:00Sultova, Neda Miroslavova (FWCC) - 138965Add auth into own class?The following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386218):
> Isn't that the same as in the _Active Users_-class? ...The following discussion from !1 should be addressed:
- [ ] @erxleb87 started a [discussion](https://gitlab.hzdr.de/fwcc/statistics-collector/-/merge_requests/1#note_1386218):
> Isn't that the same as in the _Active Users_-class? Should they have a common shared super-class in beyond _Metrics_?Sultova, Neda Miroslavova (FWCC) - 138965Sultova, Neda Miroslavova (FWCC) - 138965