Enabling ubuntu-20.04 hosts for packaging workflows found that DEB
package Github Actions workflows do not need to install createrepo
tool. Also found that createrepo is not ready for ubuntu-20.04 as
described in (till ubuntu-21.04 where it is available as the new
version of this tool named as 'createrepo_c' as DEB package):

  3a7c2102 ('github-ci: ubuntu-20.04 misses createrepo package')

To fix it added 'createrepo_c' build and installation from sources
and changed in update_repo tool 'createrepo' tool to 'createrepo_c'.
This patch is needed to use these workflows on self-hosted runners
which run under ubuntu-20.04 by default for now.

Also checking the patch on ubuntu-20.04 hosts got the following issue:

  Regenerated DEB file: pool/xenial/main/t/tarantool/tarantool-common_2.8.0.185.g4c3e0eb-1_all.deb

  <botocore.awsrequest.AWSRequest object at 0x7f7998a4ca90>

  <botocore.awsrequest.AWSRequest object at 0x7f627d965070>
  make: *** [.gitlab.mk:131: deploy] Error 255
  Error: Process completed with exit code 2.

Found that there is already issue exists in Github Actions on it [1].
Provided solution to setup AWS region in environment helped to
workaround the issue [2].

Closes tarantool/tarantool-qa#110
Closes tarantool/tarantool-qa#111

[1]: https://github.com/aws/aws-cli/issues/5234
[2]: https://github.com/aws/aws-cli/issues/5234#issuecomment-635459464

Created local composite action 'pack_and_deploy' for Tarantool packages
creation and deployment. It was created using local scripts in packages
workflows. It let to change common parts of packages creations and
deployment in each packaging workflow to call for this action. It helped
to consolidate all the instructions on packages creation and deployment
in a single place.

We want to increase testing timeouts for GitLab CI, where we use our own
runners and observe stalls and high disk pressure when several vinyl
tests are run in parallel. The idea is to set variables in GitLab CI web
interface and read them from test-run (see [1]).

First, we need to pass the variables into inner environments. GitLab CI
jobs run the testing using packpack, Docker or VirtualBox.

Packpack already preserves environment variables that are listed in the
PRESERVE_ENVVARS variable (see [2]).

This commit passes the variables that are listed in the PRESERVE_ENVVARS
variable into Docker and VirtualBox environment. So, all jobs will have
given variables in the enviroment. (Also dropped unused EXTRA_ENV
variable.)

The next commit will update the test-run submodule with support of
setting timeouts using environment variables.

[1]: https://github.com/tarantool/test-run/issues/258
[2]: https://github.com/packpack/packpack/pull/135

Due to all the activities moving from Gitlab-CI to Github-CI Actions,
then docker images creation routine updated with the new images naming
and containers registry:

  GITLAB_REGISTRY?=registry.gitlab.com

changed to

  DOCKER_REGISTRY?=docker.io

Part of #5294

Sometimes it is convenient to use default compiler on CentOS 7.
Added test job which uses for compiling default compiler files:

  CC=/usr/bin/gcc
  CXX=/usr/bin/g++

Closes #4941

Update Clang 8 testing with most recent version of
Clang: 11. Clang 8 still in the image.

Closes #5386

Changed S3 location for sources tarballs. Also added ability to
create S3 directory for the tarballs if it was not existed.

Found that tagged commits were not run the deployment gitlab-ci jobs.
To fix it added 'tags' label for deployment and perfomance jobs. Also
found that after the commit tagged it has tag label in format 'x^0'
and all previous commits till the previous tag became to have tags in
format 'x~<commits before>' like 'x~1' or 'x~2' and etc. So the check

  if git name-rev --name-only --tags --no-undefined HEAD ; then

became always pass and previous commits on rerun could began to deploy.
To fix it was used gitlab-ci environment variable 'CI_COMMIT_TAG', it
shows in real if the current commit has tag and has to be deployed.

Part of #3745

Set cleanup based on docker for all of the jobs to avoid of fails
when the job before change could use docker and reassigned the
temporary files in the working directory. If the docker is not
available than no need to use it for cleanup and cleanup runs
using shell. Gitlab-ci clean flags disabled and reorganized localy
as shown at [1].

Disabled gitlab-ci checkout default process as shown at [2] to be
able to fix repository before the checkout. Found that previously
run gitlab-ci jobs could change permissions files at repository
which broke the checkouts at the next jobs. Used for checkout
strategy [3], for submodule update strategy [4]. Submodules local
update routine in .gitlab.mk file became unneeded and removed.

List of steps made localy instead of gitlab-ci preparations:

1. Check/clone the Tarantool repository with submodules.
2. For shell based jobs change ownership of all the sources
   to 'gitlab-runner' user. (NOTE: in Docker based jobs the
   'gitlab-runner' user is not known.)
3. Fetch Tarantool sources with branches and force checkout
   of the testing commit.
4. Update submodules recursively (use force where supports).
5. Cleanup all the sources from all files except from repository.

[1] https://docs.gitlab.com/ee/ci/yaml/README.html#git-clean-flags
[2] https://docs.gitlab.com/ee/ci/yaml/README.html#git-strategy
[3] https://docs.gitlab.com/ee/ci/yaml/README.html#git-checkout
[4] https://docs.gitlab.com/ee/ci/yaml/README.html#git-submodule-strategy

Follows up #5036

Each temporary docker perf image is about 5 GB
and we don’t have space to keep them localy on servers.

- add cleanup after perf docker
  image preparation (perf_bootstrap job).
- cleanup perf docker images on execution hosts
  after perf testing (cleanup stage).

Closes #5003

Moved sources tarball creation from travis-ci to gitlab-ci,
moved its jobs for sources packing and sources deploying.

Close #4895

Fixed static build with '-DBUILD_STATIC=ON' option:

 - installed liblzma-dev library for libunwind static, due to found that
   static libunwind library uses undefined lzma functions:
     nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma
                 U lzma_index_buffer_decode
                 U lzma_index_end
                 U lzma_index_size
                 U lzma_index_uncompressed_size
                 U lzma_stream_buffer_decode
                 U lzma_stream_footer_decode
   while dynamic libunwind correctly sees liblzma installed:
     ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma
       liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000)
   so to fix it the static library of lzma was needed.

 - added lzma library to unwind library for Tarantool build at file:
     cmake/compiler.cmake
   due to fail:
     /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
       In function `xz_uncompressed_size':
     ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode'
     ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode'
     ./src/elfxx.c:205: undefined reference to `lzma_index_size'
     ./src/elfxx.c:210: undefined reference to `lzma_index_end'
     ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size'
     ./src/elfxx.c:210: undefined reference to `lzma_index_end'
     /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
       In function `_Uelf64_extract_minidebuginfo':
     ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode'
     collect2: error: ld returned 1 exit status
     test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134:
       recipe for target 'test/unit/luaL_iterator.test' failed
     make[2]: *** [test/unit/luaL_iterator.test] Error 1

 - added dl library to gomp library for test/unit tests
   binaries builds at file:
     cmake/BuildMisc.cmake
   due to fail:
     /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d):
       more undefined references to `dlsym' follow
     /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function
       `gomp_target_init':
     (.text+0x9cc): undefined reference to `dlerror'
     collect2: error: ld returned 1 exit status

  - added dl library to icu library for test/unit tests
   binaries builds at file:
     cmake/FindICU.cmake
   due to fail:
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dl_open_60':
     (.text+0x1ce2): undefined reference to `dlopen'
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dlsym_func_60':
     (.text+0x1d3d): undefined reference to `dlsym'
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dl_close_60':
     (.text+0x1d21): undefined reference to `dlclose'
     collect2: error: ld returned 1 exit status

Added static build to gitlab-ci in release check criteria named
as static_build job. Previously named static_build job renamed to
static_docker_build, due to it checks the build at Dockerfile.

Also moved static build make targets from .gitlab.mk to .travis.mk
to store it in common place with the other test/build make targets.
Moved environement from .gitlab-ci.yml file into make targets to
make this targets true building in static w/o additional setup.

Close #4551

Switched OSX 10.14 test jobs to MAC mini hosts.
Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
used to pass all appropriate target calls to Travis make rule target.

Enabled Tarantool performance testing on Gitlab-CI for release/master
branches and "*-perf" named branches. For this purpose 'perf' and
'cleanup' stages were added into Gitlab-CI pipeline.

Performance testing support next benchmarks:

- cbench
- linkbench
- nosqlbench (hash and tree Tarantool run modes)
- sysbench
- tpcc
- ycsb (hash and tree Tarantool run modes)

Benchmarks use scripts from repository:
http://github.com/tarantool/bench-run

Performance testing uses docker images, built with docker files from
bench-run repository:

- perf/ubuntu-bionic:perf_master           -- parent image with
                                              benchmarks only
- perf_tmp/ubuntu-bionic:perf_<commit_SHA> -- child images used for
                                              testing Tarantool sources

@Totktonada: Harness and workloads are to be reviewed.

We're going to use S3 compatible storage for Deb and RPM repositories
instead of packagecloud.io service. The main reason is that
packagecloud.io provides a limited amount of storage, which is not
enough for keeping all packages (w/o regular pruning of old versions).

Note: At the moment packages are still pushed to packagecloud.io from
Travis-CI. Disabling this is out of scope of this patch.

This patch implements saving of packages on an S3 compatible storage and
regeneration of a repository metadata.

The layout is a bit different from one we have on packagecloud.io.

packagecloud.io:

 | - 1.10
 | - 2.1
 | - 2.2
 | - ...

S3 compatible storage:

 | - live
 |   - 1.10
 |   - 2.1
 |   - 2.2
 |   - ...
 | - release
 |   - 1.10
 |   - 2.1
 |   - 2.2
 |   - ...

Both 'live' and 'release' repositories track release branches (named as
<major>.<minor>) and master branch. The difference is that 'live' is
updated on every push, but 'release' is only for tagged versions
(<major>.<minor>.<patch>.0).

Packages are also built on '*-full-ci' branches, but only for testing
purposes: they don't pushed anywhere.

The core logic is in the tools/update_repo.sh script, which implements
the following flow:

- create metadata for new packages
- fetch relevant metadata from the S3 storage
- push new packages to the S3 storage
- merge and push the updated metadata to the S3 storage

The script uses 'createrepo' for RPM repositories and 'reprepro' for Deb
repositories.

Closes #3380

Before this patch RUN_TESTS condition in Dockerfile.staticbuild was
ignored and always was true.

However adding of brackets solves only part of problem. If RUN_TESTS is
empty `sh -c` returns 1 and build fails.

However if we run tests we should fail build if tests are not passed.
Ternary logic was rewritten to fair if-else.

This patch fixes it and allows build tarantool statically without
running tests.

@Totktonada: Fixed .gitlab.mk to pass RUN_TESTS environment variable to
docker build arguments.

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>

`apt-get update <...>` fails on Debian Buster on docker_bootstrap goal
(see #4331 for the similar issue).

Added a description how to change dependencies in .travis.mk.

Added static build using Dockerfile on Centos 7 for release
commit criteria only. Added the cleanup for cmake generating
CMakeCache.txt files and CMakeFiles directories to avoid of
cmake localy created setup failing inside the docker after
the whole tarantool path was copied into it. Added testing
into the static build, running only when RUN_TESTS environment
variable set to non empty value, used in gitlab-ci job to run
the testing after the build.

Closes #3668

Implemented GitLab CI testing process additionally to existing Travis
CI. The new testing process is added to run tests faster. It requires to
control a load of machines to avoid flaky fails on timeouts. GitLab CI
allows us to run testing on our machines.

Created 2 stages for testing and deploying packages.

The testing stage contains the following jobs that are run for all
branches:

* Debian 9 (Stretch): release/debug gcc.
* Debian 10 (Buster): release clang8 + lto.
* OSX 14 (Mojave): release.
* FreeBSD 12: release gcc.

And the following jobs that are run of long-term branches (release
branches: for now it is 1.10, 2.1 and master):

* OSX 13 (Sierra): release clang.
* OSX 14 (Mojave): release clang + lto.

The deployment stage contains the same jobs as we have in Travis CI.
They however just build tarballs and packages: don't push them to S3 and
packagecloud.

In order to run full testing on a short-term branch one can name it with
'-full-ci' suffix.

The additional manual work is needed when dependencies are changed in
.travis.mk file ('deps_debian' or 'deps_buster_clang_8' goals):

 | make GITLAB_USER=foo -f .gitlab.mk docker_bootstrap

This command pushes docker images into GitLab Registry and then they are
used in testing. Pre-built images speed up testing.

Fixes #4156