Skip to content
Snippets Groups Projects
  • Yaroslav Dynnikov's avatar
    5d2dbc21
    ci: refactor pipeline · 5d2dbc21
    Yaroslav Dynnikov authored
    The whole CI is refactored. The following problems are solved:
    
    1. Couldn't test base image locally without overriding the old one.
    
    The testing stage is run in two ways depending on whether some
    of the docker-related files were changed.
    
    If they didn't (most likely), testing stage is run on the `latest`
    docker image as it used to.
    
    Otherwise, the docker image is rebuilt locally and tested. This is done
    by using `image:pull_policy` option, see [1].
    
    It also makes a problem of using protected credentials irrelevant.
    
    2. Gitlab caching used to be slow
    
    Stupid gitlab caching (see [2]) wasted on (un)zipping cache 4 minutes
    (of 6 total). Now it's replaced with a manual one: an archive (tar) is
    placed in a docker volume on a self-hosted gitlab runner with no
    compression.
    
    3. Speed up fetching git.
    
    The tarantool-sys submodule requires a tag to be built correctly, but
    using shallow clones prevented that. We used to work-around it by
    specifying `GIT_DEPTH: 0`, but that's sub-optimal too. This patch
    introduces an elegant yet reliable solution:
    
    ```
    until git describe; do git fetch --deepen 100; done
    ```
    
    4. No fine-grained time tracking was available
    
    Gitlab only shows the total time of `step_script` without splitting it
    into particular commands. This problem is solved by incorporating
    collapsible log sections feature, see [3]. It makes logs fancy too.
    
    Refs:
    
    - [1] https://docs.gitlab.com/ee/ci/yaml/#imagepull_policy
    - [2] https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3523
    - [3] https://docs.gitlab.com/ee/ci/jobs/#custom-collapsible-sections
    5d2dbc21
    History
    ci: refactor pipeline
    Yaroslav Dynnikov authored
    The whole CI is refactored. The following problems are solved:
    
    1. Couldn't test base image locally without overriding the old one.
    
    The testing stage is run in two ways depending on whether some
    of the docker-related files were changed.
    
    If they didn't (most likely), testing stage is run on the `latest`
    docker image as it used to.
    
    Otherwise, the docker image is rebuilt locally and tested. This is done
    by using `image:pull_policy` option, see [1].
    
    It also makes a problem of using protected credentials irrelevant.
    
    2. Gitlab caching used to be slow
    
    Stupid gitlab caching (see [2]) wasted on (un)zipping cache 4 minutes
    (of 6 total). Now it's replaced with a manual one: an archive (tar) is
    placed in a docker volume on a self-hosted gitlab runner with no
    compression.
    
    3. Speed up fetching git.
    
    The tarantool-sys submodule requires a tag to be built correctly, but
    using shallow clones prevented that. We used to work-around it by
    specifying `GIT_DEPTH: 0`, but that's sub-optimal too. This patch
    introduces an elegant yet reliable solution:
    
    ```
    until git describe; do git fetch --deepen 100; done
    ```
    
    4. No fine-grained time tracking was available
    
    Gitlab only shows the total time of `step_script` without splitting it
    into particular commands. This problem is solved by incorporating
    collapsible log sections feature, see [3]. It makes logs fancy too.
    
    Refs:
    
    - [1] https://docs.gitlab.com/ee/ci/yaml/#imagepull_policy
    - [2] https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3523
    - [3] https://docs.gitlab.com/ee/ci/jobs/#custom-collapsible-sections