GitLab HPC Driver merge requestshttps://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/merge_requests2021-03-16T16:43:01+01:00https://codebase.helmholtz.cloud/fwcc/gitlab-hpc-driver/-/merge_requests/6Draft: Rewrite for multi-user system2021-03-16T16:43:01+01:00David Paped.pape@hzdr.deDraft: Rewrite for multi-user systemCloses #1
So far, the GitLab HPC Driver uses only one user account. This has many disadvantages like interference of jobs outside of their assigned build directory, e.g. when they install packages to `~/.local`, or "malicious" jobs bei...Closes #1
So far, the GitLab HPC Driver uses only one user account. This has many disadvantages like interference of jobs outside of their assigned build directory, e.g. when they install packages to `~/.local`, or "malicious" jobs being able to read other jobs' data. To resolve this issue, I rewrote the driver to be able to work with multiple user accounts, each running only one CI job at a time.
In `~/.config/gitlab-hpc-driver.cfg`, an array of UIDs is set. The associated users will execute CI jobs on the cluster. The availability of the users and their association with jobs is recorded in an SQLite3 database that will be set up automatically. Each CI job is subdivided into three smaller jobs. Here is what they do:
**Config job**
- get an available user from the database
- associate that user with the given job ID
- return the users home directory path to GitLab
**Run job**
- `chown` the job script generated by GitLab to the user
- `setgid` and `setuid` to the user
- run the job on the cluster using Slurm
- run non-cluster jobs directly using `fork` and `execle`
**Cleanup job**
- `setgid` and `setuid`
- remove all contents of the user's home directory
- remove association of the user and job ID
**Info for deployment:**
It is important, that the users each have their own group, i.e. user1:user1, user2:user2, ... and are also not members of groups like fwc.