Skip to content
Snippets Groups Projects
  1. Apr 02, 2020
    • Alexander V. Tikhonov's avatar
      static build: create new build w/o dockerfile · fc7b0393
      Alexander V. Tikhonov authored
      Fixed static build with '-DBUILD_STATIC=ON' option:
      
       - installed liblzma-dev library for libunwind static, due to found that
         static libunwind library uses undefined lzma functions:
           nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma
                       U lzma_index_buffer_decode
                       U lzma_index_end
                       U lzma_index_size
                       U lzma_index_uncompressed_size
                       U lzma_stream_buffer_decode
                       U lzma_stream_footer_decode
         while dynamic libunwind correctly sees liblzma installed:
           ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma
             liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000)
         so to fix it the static library of lzma was needed.
      
       - added lzma library to unwind library for Tarantool build at file:
           cmake/compiler.cmake
         due to fail:
           /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
             In function `xz_uncompressed_size':
           ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode'
           ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode'
           ./src/elfxx.c:205: undefined reference to `lzma_index_size'
           ./src/elfxx.c:210: undefined reference to `lzma_index_end'
           ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size'
           ./src/elfxx.c:210: undefined reference to `lzma_index_end'
           /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
             In function `_Uelf64_extract_minidebuginfo':
           ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode'
           collect2: error: ld returned 1 exit status
           test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134:
             recipe for target 'test/unit/luaL_iterator.test' failed
           make[2]: *** [test/unit/luaL_iterator.test] Error 1
      
       - added dl library to gomp library for test/unit tests
         binaries builds at file:
           cmake/BuildMisc.cmake
         due to fail:
           /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d):
             more undefined references to `dlsym' follow
           /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function
             `gomp_target_init':
           (.text+0x9cc): undefined reference to `dlerror'
           collect2: error: ld returned 1 exit status
      
        - added dl library to icu library for test/unit tests
         binaries builds at file:
           cmake/FindICU.cmake
         due to fail:
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dl_open_60':
           (.text+0x1ce2): undefined reference to `dlopen'
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dlsym_func_60':
           (.text+0x1d3d): undefined reference to `dlsym'
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dl_close_60':
           (.text+0x1d21): undefined reference to `dlclose'
           collect2: error: ld returned 1 exit status
      
      Added static build to gitlab-ci in release check criteria named
      as static_build job. Previously named static_build job renamed to
      static_docker_build, due to it checks the build at Dockerfile.
      
      Also moved static build make targets from .gitlab.mk to .travis.mk
      to store it in common place with the other test/build make targets.
      Moved environement from .gitlab-ci.yml file into make targets to
      make this targets true building in static w/o additional setup.
      
      Close #4551
      
      (cherry picked from commit fc55875e)
      fc7b0393
    • Alexander V. Tikhonov's avatar
      gitlab-ci: implement OSX 10.14 testing on mac mini · 49541992
      Alexander V. Tikhonov authored
      Switched OSX 10.14 test jobs to MAC mini hosts.
      Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
      used to pass all appropriate target calls to Travis make rule target.
      
      (cherry picked from commit 7faa1abe)
      49541992
  2. Feb 21, 2020
    • Alexander V. Tikhonov's avatar
      gitlab-ci: enable performance testing · 15f3b822
      Alexander V. Tikhonov authored
      Enabled Tarantool performance testing on Gitlab-CI for release/master
      branches and "*-perf" named branches. For this purpose 'perf' and
      'cleanup' stages were added into Gitlab-CI pipeline.
      
      Performance testing support next benchmarks:
      
      - cbench
      - linkbench
      - nosqlbench (hash and tree Tarantool run modes)
      - sysbench
      - tpcc
      - ycsb (hash and tree Tarantool run modes)
      
      Benchmarks use scripts from repository:
      http://github.com/tarantool/bench-run
      
      Performance testing uses docker images, built with docker files from
      bench-run repository:
      
      - perf/ubuntu-bionic:perf_master           -- parent image with
                                                    benchmarks only
      - perf_tmp/ubuntu-bionic:perf_<commit_SHA> -- child images used for
                                                    testing Tarantool sources
      
      @Totktonada: Harness and workloads are to be reviewed.
      
      (cherry picked from commit 87c68344)
      15f3b822
  3. Feb 20, 2020
  4. Feb 04, 2020
    • Alexander V. Tikhonov's avatar
      gitlab-ci: push Deb/RPM packages to S3 based repos · a97ce0de
      Alexander V. Tikhonov authored
      We're going to use S3 compatible storage for Deb and RPM repositories
      instead of packagecloud.io service. The main reason is that
      packagecloud.io provides a limited amount of storage, which is not
      enough for keeping all packages (w/o regular pruning of old versions).
      
      Note: At the moment packages are still pushed to packagecloud.io from
      Travis-CI. Disabling this is out of scope of this patch.
      
      This patch implements saving of packages on an S3 compatible storage and
      regeneration of a repository metadata.
      
      The layout is a bit different from one we have on packagecloud.io.
      
      packagecloud.io:
      
       | - 1.10
       | - 2.1
       | - 2.2
       | - ...
      
      S3 compatible storage:
      
       | - live
       |   - 1.10
       |   - 2.1
       |   - 2.2
       |   - ...
       | - release
       |   - 1.10
       |   - 2.1
       |   - 2.2
       |   - ...
      
      Both 'live' and 'release' repositories track release branches (named as
      <major>.<minor>) and master branch. The difference is that 'live' is
      updated on every push, but 'release' is only for tagged versions
      (<major>.<minor>.<patch>.0).
      
      Packages are also built on '*-full-ci' branches, but only for testing
      purposes: they don't pushed anywhere.
      
      The core logic is in the tools/update_repo.sh script, which implements
      the following flow:
      
      - create metadata for new packages
      - fetch relevant metadata from the S3 storage
      - push new packages to the S3 storage
      - merge and push the updated metadata to the S3 storage
      
      The script uses 'createrepo' for RPM repositories and 'reprepro' for Deb
      repositories.
      
      Closes #3380
      
      (cherry picked from commit 05d3ed4b)
      a97ce0de
  5. Dec 05, 2019
  6. Nov 08, 2019
    • Alexander V. Tikhonov's avatar
      build: add CentOS 8 into CI / CD · 289bb2cb
      Alexander V. Tikhonov authored
      Added build + test jobs in GitLab-CI and build + test + deploy jobs on
      Travis-CI for CentOS 8.
      
      Updated testing dependencies in the RPM spec to follow the new Python 2
      package naming scheme that was introduced in CentOS 8: it uses
      'python2-' prefix rather then 'python-'.
      
      CentOS 8 does not provide python2-gevent and python2-greenlet packages,
      so they were pushed to https://packagecloud.io/packpack/backports
      repository. This repository is enabled in our build image
      (packpack/packpack:el-8) by default. Those dependencies are build-time,
      so nothing was changed for a user. The source RPM packages were gathered
      from https://cbs.centos.org
      
      .
      
      Disabled app-tap/pwd.test.lua on CentOS 8 due to systemd-nss issue,
      which was not worked around properly. Filed #4592 to resolved it in the
      future.
      
      Eliminated libunwind runtime dependency (and libunwind-devel build
      dependency) on CentOS 8, because the base system does not provide it.
      fiber.info() backtraces and printing of a backtrace after a crash will
      not be available on this system. Hopefully we'll fix it in the future,
      filed #4611 on this.
      
      Closes #4543
      
      Reviewed-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      Reviewed-by: default avatarIgor Munkin <imun@tarantool.org>
      (cherry picked from commit e3d9d8c9)
      289bb2cb
  7. Oct 24, 2019
  8. Aug 27, 2019
    • Alexander V. Tikhonov's avatar
      gitlab-ci: clean up .gitlab-ci.yml · 52a8f5c4
      Alexander V. Tikhonov authored
      Cleaned up the .gitlab-ci.yml file from duplicating
      code - added templates that storres the needed
      configuration values for different jobs.
      Also moved the static_build from 'deploy' tag to
      'deploy_test' tag to be sure that tests will not
      be run under high load.
      52a8f5c4
  9. Aug 21, 2019
    • Alexander V. Tikhonov's avatar
      gitlab-ci: add static build · 100e3028
      Alexander V. Tikhonov authored
      Added static build using Dockerfile on Centos 7 for release
      commit criteria only. Added the cleanup for cmake generating
      CMakeCache.txt files and CMakeFiles directories to avoid of
      cmake localy created setup failing inside the docker after
      the whole tarantool path was copied into it. Added testing
      into the static build, running only when RUN_TESTS environment
      variable set to non empty value, used in gitlab-ci job to run
      the testing after the build.
      
      Closes #3668
      
      (cherry picked from commit f7509186)
      100e3028
  10. Aug 02, 2019
  11. Jul 19, 2019
    • Alexander V. Tikhonov's avatar
      test: need ASAN testing in commit process · 55f7586a
      Alexander V. Tikhonov authored
      Added ASAN tesing in commit process, used clang-8 for
      ASAN build under debian-buster image. Added for testing
      only the passing test suites, the rest of the tests
      not used and will be enabled durring issue #4360. Also
      fixed job for testing LTO with clang on debian-buster
      in travis-ci, changed it to the same as in gitlab-ci,
      changed default clang to clang-8.
      
      Closes #4359
      55f7586a
  12. Jul 06, 2019
  13. Jul 04, 2019
    • Alexander V. Tikhonov's avatar
      Enable GitLab CI testing · ce623a23
      Alexander V. Tikhonov authored
      Implemented GitLab CI testing process additionally to existing Travis
      CI. The new testing process is added to run tests faster. It requires to
      control a load of machines to avoid flaky fails on timeouts. GitLab CI
      allows us to run testing on our machines.
      
      Created 2 stages for testing and deploying packages.
      
      The testing stage contains the following jobs that are run for all
      branches:
      
      * Debian 9 (Stretch): release/debug gcc.
      * Debian 10 (Buster): release clang8 + lto.
      * OSX 14 (Mojave): release.
      * FreeBSD 12: release gcc.
      
      And the following jobs that are run of long-term branches (release
      branches: for now it is 1.10, 2.1 and master):
      
      * OSX 13 (Sierra): release clang.
      * OSX 14 (Mojave): release clang + lto.
      
      The deployment stage contains the same jobs as we have in Travis CI.
      They however just build tarballs and packages: don't push them to S3 and
      packagecloud.
      
      In order to run full testing on a short-term branch one can name it with
      '-full-ci' suffix.
      
      The additional manual work is needed when dependencies are changed in
      .travis.mk file ('deps_debian' or 'deps_buster_clang_8' goals):
      
       | make GITLAB_USER=foo -f .gitlab.mk docker_bootstrap
      
      This command pushes docker images into GitLab Registry and then they are
      used in testing. Pre-built images speed up testing.
      
      Fixes #4156
      ce623a23
Loading