Closes #240

Дмитрий Кибирев authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

Also sign, deploy and check install.

Resolve issue #219
Check package deployment was added for Ubuntu focal, Ubuntu jammy, Debian
bullseye, Alt p9 and Alt p10 to avoid problems with broken deployments.

Pytest xdist significantly reduce integration tests time
by splitting test cases into parallel child process.
Enable this feature by `-n` or `--numprocesses`,
set to 'auto' for optimal hardware resource usage.

The typo was introduced in dd4f93f8.

1. Rename "check-deployment" stage, make it more concise using
   `parallel:matrix` Gitlab CI feature.
2. Ignore `gitlab-ci.yml` changes in `test-docker` job
3. Remove unused credentials from `sign-rpm-packages` job

Dmitry Travyan authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

This commit provides:

1. Building and pushing a docker image for kubernetes.
2. Helm chart for quick and easy installation in any cluster that supports the Kubernetes API.
3. Building and pushing a docker image with diagnostic tools.
4. Short readme about helm chart features.

Дмитрий Кибирев authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

rename dependency tarantool to tarantool-picodata

Saving cache isn't instantaneous. Usually it takes about a minute.
If another job tries to restore cache at the same time, it fails
with the error "tar: Unexpected EOF in archive".

Now tar writes to a different filename with a random suffix, and then
renames it atomically.

CI on master branch always reuses the same archive. As a result, its
size gradually grows. It already takes more than 6GB, while it should be
about 3GB large. It makes restoring the cache slower and slower.

With this patch the cache is cleared every time in master branch.

IDE plugins (e.g. vscode-yaml) complains about invalid `gitlab-ci.yml`
schema because yaml anchors don't flatten lists.

This patch re-organizes anchors to praise schema validation and to make
it easier to understand (or at least it tries to).

The previous commit did compare changes to the master branch and always
skipped the `push-base-image` stage. In this commit it's refactored and
explained once more.

The whole CI is refactored. The following problems are solved:

1. Couldn't test base image locally without overriding the old one.

The testing stage is run in two ways depending on whether some
of the docker-related files were changed.

If they didn't (most likely), testing stage is run on the `latest`
docker image as it used to.

Otherwise, the docker image is rebuilt locally and tested. This is done
by using `image:pull_policy` option, see [1].

It also makes a problem of using protected credentials irrelevant.

2. Gitlab caching used to be slow

Stupid gitlab caching (see [2]) wasted on (un)zipping cache 4 minutes
(of 6 total). Now it's replaced with a manual one: an archive (tar) is
placed in a docker volume on a self-hosted gitlab runner with no
compression.

3. Speed up fetching git.

The tarantool-sys submodule requires a tag to be built correctly, but
using shallow clones prevented that. We used to work-around it by
specifying `GIT_DEPTH: 0`, but that's sub-optimal too. This patch
introduces an elegant yet reliable solution:

```
until git describe; do git fetch --deepen 100; done
```

4. No fine-grained time tracking was available

Gitlab only shows the total time of `step_script` without splitting it
into particular commands. This problem is solved by incorporating
collapsible log sections feature, see [3]. It makes logs fancy too.

Refs:

- [1] https://docs.gitlab.com/ee/ci/yaml/#imagepull_policy
- [2] https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3523
- [3] https://docs.gitlab.com/ee/ci/jobs/#custom-collapsible-sections

Alexey Protsenko authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

1. Signed rpm packages used to be overridden by unsigned
   ones due to the artifacts clash. This bug is fixed by
   specifying explicit stage dependencies.
2. Add stage for testing installed packages.

Dmitry Travyan authored 2 years ago and Valentin Syrovatskiy committed 2 years ago

run image build only on push to master branch

1. Review `Pipfile`:
  - Remove unused `filelock`;
  - Install `mypy` - static type checker for Python.
2. Add new command `pipenv run lint`.
3. Enable `mypy` in CI. Fix reported errors in `test_basics.py`.
4. Renew readme.