README 2.32 KB
Newer Older
1
Prototype of a gitlab-runner using a custom executor to run CI jobs on hemera cluster with Slurm.
Pape, David (FWCC) - 139658's avatar
Pape, David (FWCC) - 139658 committed
2
3


4
5
6
Instructions -- Use in .gitlab-ci.yml
=====================================

Pape, David (FWCC) - 139658's avatar
Pape, David (FWCC) - 139658 committed
7
- CI jobs that are supposed to run on the cluster must be tagged with "hemera"
8

9
10
11
- Pass options to the Slurm job by adding them to your job as variables and prepending "CI_SLURM_"
  to their name. Options must be all upper case, hyphens become underscores. Currently supported
  options are:
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
    - CI_SLURM_NODES
    - CI_SLURM_NTASKS
    - CI_SLURM_NTASKS_PER_NODE
    - CI_SLURM_CPUS_PER_TASK
    - CI_SLURM_PARTITION
    - CI_SLURM_GRES

- Example 1 -- Run a hybrid MPI/OpenMP job

    run_mpi_omp:
        stage: deploy
        dependencies:
            - build_mpi_omp
        tags:
            - hemera
        variables:
            CI_SLURM_NODES: 2
            CI_SLURM_NTASKS_PER_NODE: 4
            CI_SLURM_CPUS_PER_TASK: 8
        script:
            - export OMP_NUM_THREADS=8
            - mpirun ./mpi_omp

- Example 2 -- Run a CUDA job on the GPU partition

    run_cuda:
        stage: deploy
        dependencies:
            - build_cuda
        tags:
            - hemera
        variables:
            CI_SLURM_PARTITION: "gpu"
            CI_SLURM_GRES: "gpu:1"
        script:
            - ./cuda


Instructions -- Runner setup
============================
Pape, David (FWCC) - 139658's avatar
Pape, David (FWCC) - 139658 committed
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

- Run ./setup.sh to symlink the runner configuration file to the apropriate place. You will be
  warned if the gitlab-runner executable (which you will need when working with artifacts) is not in
  your path.

- Start the runner: gitlab-runner run --working-directory

- Register runners to your project: gitlab-runner register
  You'll need one runner with the tag "hemera" and one without.

- Edit the runner-config.toml and place the tokens of the newly generated configuration blocks in
  the two old blocks and delete the rest of the new ones.

This process is necessary when executing the runner as a non-privileged user.


Code overview
=============

setup.sh           -- creates a symlink for the configuration file
runner-config.toml -- configuration of the gitlab-runner
base.sh            -- contains variables commonly used by the other scripts
config.sh          -- is executed in the config stage
run.sh             -- is executed in the run stage
cleanup.sh         -- is executed in the cleanup stage