Skip to content
Snippets Groups Projects
  1. Sep 28, 2021
    • VitaliyaIoffe's avatar
      github-ci: use openssl@1.1 · 33830978
      VitaliyaIoffe authored
      OSX workflows use brew for install openssl.
      There was a new release of openssl@3.0 and
      homebrew updated the openssl formula.
      
      Close #6468
      33830978
  2. Aug 19, 2021
  3. Jul 02, 2021
    • Alexander V. Tikhonov's avatar
      github-ci: add GitHub Actions workflow for Odroid · 72c77166
      Alexander V. Tikhonov authored
      Odroid is GNU/Linux ARM64 platform. In scope of this commit new GitHub
      Actions workflows for testing Tarantool on Odroid hosts are added:
      
        Release: .github/workflows/odroid_arm64.yml
        Debug: .github/workflows/odroid_debug_arm64.yml
      
      Introduced new targets in .travis.mk Makefile:
      
        deps_odroid: Installs required dependencies.
      
        build_odroid: Builds Tarantool with the following flags set
          in env of .github/workflows/odroid_debug_arm64.yml file:
            1. to avoid the issue #6142:
               -DENABLE_BACKTRACE=OFF
            2. to avoid the issue #6143:
               -DCMAKE_C_FLAGS="-Wno-type-limits "
               -DCMAKE_BUILD_TYPE=Debug
      
        test_odroid: Builds and tests `LuaJIT-test` suite on Odroid.
      
      Also v1 version of GitHub checkout action is used, because action
      version v2 was introduced in git version 2.18.0 [1]. The latest
      available version on Odroid is the following:
      
        git is already the newest version (1:2.17.1-1ubuntu0.8).
      
      [1]: https://github.com/actions/checkout#readme
      
      Closes tarantool/tarantool-qa#121
      72c77166
  4. Jun 15, 2021
    • Alexander V. Tikhonov's avatar
      github-ci: add GitHub Actions workflow for M1 · 00c6b496
      Alexander V. Tikhonov authored
      In scope of this commit new GitHub Actions workflows for testing
      Tarantool on M1 hosts are added:
      
        Release: .github/workflows/osx_arm64_11_2.yml
        Debug: .github/workflows/osx_debug_arm64_11_2.yml
      
      Since GitHub Actions uses x86_64 environment by default on M1 targets,
      'arch -arm64' prefix is specified in GitHub Actions workflow to make all
      commands in .travis.mk run in ARM64 environment.
      
      Introduced a new temporary target in .travis.mk Makefile to run only
      specific LuaJIT test suites on M1. Now it runs only the following LuaJIT
      test targets:
      * PUC-Rio-Lua-5.1-tests
      * lua-Harness-tests
      * tarantool-tests
      
      Python 3.9 is installed by default on M1 hosts, but gevent is required
      for Tarantool tests, which installation fails with the following error:
      
        Using cached gevent-21.1.2.tar.gz (5.9 MB)
        Installing build dependencies ... done
        Getting requirements to build wheel ... done
          Preparing wheel metadata ... error
          ERROR: Command errored out with exit status 1:
           command: /opt/homebrew/opt/python@3.9/bin/python3.9 /opt/homebrew/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/tmpyy59ae2p
               cwd: /private/var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/pip-install-msbf7_vz/gevent_c2956687bb0d4de9bfb5f0660da759ee
          Complete output (42 lines):
            ...
            File "/private/var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/pip-build-env-1lesbbxi/overlay/lib/python3.9/site-packages/cffi/api.py", line 48, in __init__
              import _cffi_backend as backend
          ImportError: dlopen(/private/var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/pip-build-env-1lesbbxi/overlay/lib/python3.9/site-packages/_cffi_backend.cpython-39-darwin.so, 2): no suitable image found.  Did find:
          	/private/var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/pip-build-env-1lesbbxi/overlay/lib/python3.9/site-packages/_cffi_backend.cpython-39-darwin.so: mach-o, but wrong architecture
          	/private/var/folders/b0/1vlv5rvn77x2rn6zbl2p4tqr0000gp/T/pip-build-env-1lesbbxi/overlay/lib/python3.9/site-packages/_cffi_backend.cpython-39-darwin.so: mach-o, but wrong architecture
      
      This issue is described in gevent/gevent#1721. Fortunately, gevent can
      be successfully installed via Python 3.8, hence to avoid this failure,
      python3 is pinned to the specific version (i.e. python@3.8) until the
      mentioned issue is resolved.
      
      Closes tarantool/tarantool-qa#120
      Relates to #6068
      00c6b496
    • Alexander V. Tikhonov's avatar
      github-ci: refactor OSX jobs parameters · ebd87543
      Alexander V. Tikhonov authored
      To avoid targets duplication in the later use of .travis.mk file, it was
      decided to parameterize OSX jobs and move all fine tuning manipulations
      related to these pipelines from .travis.mk to the corresponding GitHub
      Actions workflows.
      ebd87543
  5. May 19, 2021
    • Alexander V. Tikhonov's avatar
      github-ci: add result message from coveralls in PR · 660e3263
      Alexander V. Tikhonov authored
      Workflow 'debug_coverage' produces and uploads results to 'coveralls.io'
      web site. Message about it can be shown in PR within each run was done.
      This patch adds the ability to send message in available PR otherwise it
      is skipped. Also added 'coverage.info' file to artifacts list.
      
      Found that used 'coverallsapp/github-action' action checks if the
      trigger was 'pull_request' used to run the workflow [1]. And only in
      this way it writes results message to PR. Previously 'pull_request'
      trigger was blocked to avoid duplication with 'push' trigger. To make
      able to run workflow by any single trigger and to send message to PR
      if it exits, it was implemented the following logic:
      
        - run workflows on both triggers 'push' and 'pull_request';
        - for workflow with 'push' trigger check if PR exists then skip all
          jobs otherwise continue running;
        - for workflow with 'pull_request' trigger continue running;
      
      To avoid of issue coverallsapp/github-action#55 [2], sources checkout
      should be based on 2nd version and later.
      
      Closes #5644
      
      [1]: https://github.com/coverallsapp/github-action/blob/master/src/run.ts#L38
      [2]: https://github.com/coverallsapp/github-action/issues/55#issuecomment-644927165
      660e3263
  6. Apr 14, 2021
    • Sergey Kaplun's avatar
      luajit: bump new version · 8a09e18c
      Sergey Kaplun authored
      LuaJIT submodule is bumped to introduce the following changes:
      * test: disable too deep recursive PUC-Rio test
      * test: disable PUC-Rio hanging GC test
      * test: disable PUC-Rio test checking -h option
      * test: disable PUC-Rio test for checking arg layout
      * test: disable PUC-Rio tests for several -l options
      * test: disable PUC-Rio test for syntax level error
      * test: disable PUC-Rio test for non-ascii variable
      * test: disable PUC-Rio test for fast function name
      * test: disable PUC-Rio test for variables in error
      * test: disable PUC-Rio test for getfenv in tailcall
      * test: remove string.gfind assert in PUC-Rio test
      * test: use math.fmod in PUC-Rio tests
      * test: disable locale-dependent PUC-Rio tests
      * test: adapt PUC-Rio test for %q in string.format
      * test: disable PUC-Rio test for per-coroutine hooks
      * test: adapt PUC-Rio test with activeline check
      * test: disable PUC-Rio test for tailcall info
      * test: adapt PUC-Rio test with count hooks
      * test: adapt PUC-Rio test for debug in vararg func
      * test: adapt PUC-Rio tests with vararg functions
      * test: disable PUC-Rio suite tests for line hook
      * test: adapt PUC-Rio tests counting GC steps
      * test: disable PUC-Rio tests for bytecode header
      * test: disable PUC-Rio tests confused by -v output
      * test: adapt PUC-Rio test for arg presence
      * test: remove quotes in progname from PUC-Rio
      * test: adapt PUC-Rio suite for out-of-source build
      * test: build auxiliary C libs from PUC-Rio Lua 5.1
      * test: add PUC-Rio Lua 5.1 test suite
      
      Within this changeset PUC-Rio Lua 5.1 suite[1] is added to Tarantool
      testing. Considering Tarantool specific changes in runtime the suite
      itself is adjusted in LuaJIT submodule.
      
      <test/luajit-test-init.lua> pretest runner is adjusted by setting custom
      `_loadstring()` and `_dofile()` global function to run tests correctly
      for out-of-source build.
      
      Also, this patch excludes PUC-Rio-Lua-5.1 test suite from ASAN checks.
      
      [1]: https://www.lua.org/tests/lua5.1-tests.tar.gz
      
      
      
      Closes #5845
      Closes #5686
      Closes #5694
      Closes #5701
      Closes #5708
      Closes #5710
      Closes #5711
      Closes #5712
      Part of #4473
      
      Reviewed-by: default avatarIgor Munkin <imun@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      Unverified
      8a09e18c
  7. Apr 07, 2021
  8. Apr 05, 2021
    • Alexander V. Tikhonov's avatar
      github-ci: use vardir option in tests runs · 474eda49
      Alexander V. Tikhonov authored
      Got warning message:
      
        [014] WARGING: unix socket's "/home/ubuntu/actions-runner/_work/tarantool/tarantool/test/var/014_box/gh-5422-broken_snapshot.socket-iproto" path has length 108 symbols that is longer than 107. That likely will cause failing of tests.
      
      It caused the following fail:
      
        [038] Starting instance autobootstrap_guest1...
        [038] Start failed: builtin/box/console.lua:865: failed to create server unix/:/home/ubuntu/actions-runner/_work/tarantool/tarantool/test/var/038_replication/autobootstrap_guest1.socket-admin: No buffer space available
      
      To avoid of it use vardir option in tests runs to decrease paths length.
      
      Closes tarantool/tarantool-qa#104
      474eda49
    • Alexander V. Tikhonov's avatar
      github-ci: avoid of use container tags in actions · 58fe0fcb
      Alexander V. Tikhonov authored
      Changed the following workflows:
      
        luacheck
        debug_coverage
        release*
        static_build
        static_build_cmake_linux
      
      It was changed the OS in which the test run from debian to ubuntu.
      Also changed the way how this OS was booted - before the change it
      was booted as docker container using Github Actions tag from inside
      the worklfows. And it caused all the workflow steps to be run inside
      it. After the change no container run anymore on the running host.
      Github Actions host uses for now with its native OS set in 'runs'
      tag. It was decided to use the latest one OS `ubuntu-20.04` which is
      already the default for 'ubuntu-latest' tag.
      
      This change gave us the abilities to:
       - Remove extra container step in workflow.
       - Switch off swap using 'swapoff' command.
       - Use the same OS as Github Actions uses by default.
       - Setup our local hosts using Github Actions image snapshot.
       - Enable use of actions/checkout@v2.3.4 which is better than v1.
       - Light bootstrap of packages in local .*.mk makefile for:
           build: libreadline-dev libunwind-dev
           tests: pip install -r test-run/requirements.txt
      
      Closes tarantool/tarantool-qa#101
      58fe0fcb
  9. Mar 31, 2021
    • Alexander V. Tikhonov's avatar
      github-ci: switch off swap use · fd6ee6d5
      Alexander V. Tikhonov authored
      Github Actions provides hosts for Linux base runners in the following
      configurations:
      
        2 Cores
        7 Gb memory
        4 Gb swap memory
      
      To avoid of issues with hanging/slowing tests on high memory use
      like [1], hosts configurations must avoid of swap memory use. All
      of the tests workflows run inside dockers containers. This patch
      sets in docker run configurations memory limits based on current
      github actions hosts - 7Gb memory w/o swap memory increase.
      
      Checked 10 full runs (29 workflows in each run used the change) and
      got single failed test on gevent() routine in test-run. This result much
      better than w/o this patch when 3-4 of workflows fail on each full run.
      
      It could happen because swappiness set to default value:
      
        cat /sys/fs/cgroup/memory/memory.swappiness
        60
      
      From documentation on swappiness [2]:
      
        This control is used to define the rough relative IO cost of swapping
        and filesystem paging, as a value between 0 and 200. At 100, the VM
        assumes equal IO cost and will thus apply memory pressure to the page
        cache and swap-backed pages equally; lower values signify more
        expensive swap IO, higher values indicates cheaper.
        Keep in mind that filesystem IO patterns under memory pressure tend to
        be more efficient than swap's random IO. An optimal value will require
        experimentation and will also be workload-dependent.
      
      We may try to tune how often anonymous pages are swapped using the
      swappiness parameter, but our goal is to stabilize timings (and make
      them as predictable as possible), so the best option is to disable swap
      at all and work on descreasing memory consumption for huge tests.
      
      For Github Actions host configurations with 7Gb RAM it means that after
      2.8Gb RAM was used swap began to use. But in testing we have some tests
      that use 2.5Gb of RAM like 'box/net_msg_max.test.lua' and memory
      fragmentation could cause after the test run swap use [3].
      
      Also found that disk cache could use some RAM and it also was the cause
      of fast memory use and start swapping. It can be periodically dropped
      from memory [4] using 'drop_cache' system value setup, but it won't fix
      the overall issue with swap use.
      
      After freed cached pages in RAM another system kernel option can be
      tuned [5][6] 'vfs_cache_pressure'. This percentage value controls the
      tendency of the kernel to reclaim the memory which is used for caching
      of directory and inode objects. Increasing it significantly beyond
      default value of 100 may have negative performance impact. Reclaim code
      needs to take various locks to find freeable directory and inode
      objects. With 'vfs_cache_pressure=1000', it will look for ten times more
      freeable objects than there are. This patch won't do this change, but
      it can be done as the next change.
      
      To fix the issue there were made changes:
      
       - For jobs that run tests and use actions/environment and don't use
         Github Actions container tag, it was set 'sudo swapoff -a' command
         in actions/environment action.
      
       - For jobs that run tests and use Github Actions container tag the
         previous solution doesn't work. It was decided to hard-code the
         memory value based on found on Github Actions hosts memory size
         7Gb. It was set for Github container tag as additional options:
           options: '--init --memory=7G --memory-swap=7G'
         This changes were made temporary till these containers tags will
         be removed within resolving tarantool/tarantool-qa#101 issue for
         workflows:
           debug_coverage
           release
           release_asan_clang11
           release_clang
           release_lto
           release_lto_clang11
           static_build
           static_build_cmake_linux
      
       - For VMware VMs like with FreeBSD added 'sudo swapoff -a' command
         before build commands.
      
       - For OSX on Github actions hosts swapping already disabled:
           sysctl vm.swapusage
           vm.swapusage: total = 0.00M  used = 0.00M  free = 0.00M  (encrypted)
         Also manual switching off swap currently not possible due to do
         System Integrity Protection (SIP) must be disabled [7], but we
         don't have such access on Github Actions hosts. For local hosts
         it must be done manually with [8]:
           sudo nvram boot-args="vm_compressor=2"
         Added swap status control to be sure that host correctly configured:
           sysctl vm.swapusage
      
      Closes tarantool/tarantool-qa#99
      
      [1]: https://github.com/tarantool/tarantool-qa/issues/93
      [2]: https://github.com/torvalds/linux/blob/1e43c377a79f9189fea8f2711b399d4e8b4e609b/Documentation/admin-guide/sysctl/vm.rst#swappiness
      [3]: https://unix.stackexchange.com/questions/2658/why-use-swap-when-there-is-more-than-enough-free-space-in-ram
      [4]: https://kubuntu.ru/node/13082
      [5]: https://www.kernel.org/doc/Documentation/sysctl/vm.txt
      [6]: http://devhead.ru/read/uskorenie-raboty-linux
      [7]: https://osxdaily.com/2010/10/08/mac-virtual-memory-swap/
      [8]: https://gist.github.com/dan-palmer/3082266#gistcomment-3667471
      fd6ee6d5
  10. Mar 15, 2021
    • Sergey Bronnikov's avatar
      travis: switch to Python 3 version · 17040677
      Sergey Bronnikov authored
      - install python3 on mac machines
      - run test-run without specifying path to python interpreter
      - get rid our own tarantool brew-tap that depends on Python 2
      
      Part of #5652
      17040677
  11. Feb 28, 2021
  12. Feb 18, 2021
    • Alexander V. Tikhonov's avatar
      jepsen: correct jepsen build preparation target · 5c253e66
      Alexander V. Tikhonov authored
      Found issue:
      
        root@hpalx:/source# make run-jepsen
        [  0%] Performing update step for 'jepsen-tests'
      
        *** Please tell me who you are.
      
        Run
      
          git config --global user.email "you@example.com"
          git config --global user.name "Your Name"
      
        to set your account's default identity.
        Omit --global to set the identity only in this repository.
      
        fatal: unable to auto-detect email address (got 'root@hpalx.(none)')
        Cannot save the current index state
        CMake Error at /source/jepsen-tests-prefix/tmp/jepsen-tests-gitupdate.cmake:83 (message):
          Failed to stash changes
      
        CMakeFiles/jepsen-tests.dir/build.make:95: recipe for target 'jepsen-tests-prefix/src/jepsen-tests-stamp/jepsen-tests-update' failed
      
      added 'Nobody' User setup to git configuration if needed.
      5c253e66
  13. Feb 08, 2021
  14. Jan 25, 2021
    • Alexander V. Tikhonov's avatar
      build: remove autotools from packages spec · 11dcc9cb
      Alexander V. Tikhonov authored
      After cURL was changed to use cmake in commit
      
        2b076019 ('build: enable cmake in curl build')
      
      there is no need to set dependencies to autotools. So removed it from:
      
        - rpm build spec for packages builds
        - deb build control for packages builds
        - OSX build targets
        - static OSX build target
        - freebsd VBOX VM image setup
      11dcc9cb
  15. Dec 27, 2020
    • Alexander Turenko's avatar
      ci: preserve certain environment variables · d2f4bd68
      Alexander Turenko authored
      We want to increase testing timeouts for GitLab CI, where we use our own
      runners and observe stalls and high disk pressure when several vinyl
      tests are run in parallel. The idea is to set variables in GitLab CI web
      interface and read them from test-run (see [1]).
      
      First, we need to pass the variables into inner environments. GitLab CI
      jobs run the testing using packpack, Docker or VirtualBox.
      
      Packpack already preserves environment variables that are listed in the
      PRESERVE_ENVVARS variable (see [2]).
      
      This commit passes the variables that are listed in the PRESERVE_ENVVARS
      variable into Docker and VirtualBox environment. So, all jobs will have
      given variables in the enviroment. (Also dropped unused EXTRA_ENV
      variable.)
      
      The next commit will update the test-run submodule with support of
      setting timeouts using environment variables.
      
      [1]: https://github.com/tarantool/test-run/issues/258
      [2]: https://github.com/packpack/packpack/pull/135
      Unverified
      d2f4bd68
  16. Dec 25, 2020
  17. Dec 22, 2020
    • Alexander V. Tikhonov's avatar
      github-ci: implement coverity check · 9209e8b2
      Alexander V. Tikhonov authored
      Added standalone job with coverity check as described at [1]. This
      job uploads results to coverity.com host to 'tarantool' project when
      COVERITY_TOKEN environment is enabled. Main coverity functionality
      added at .travis.mk make file as standalone targets:
      
        'test_coverity_debian_no_deps' - used in github-ci actions
        'coverity_debian' - additional target with needed tools check
      
      This job configured by cron scheduler on each Saturday 04:00 am.
      
      Closes #5600
      
      [1] - https://scan.coverity.com/download?tab=cxx
      9209e8b2
    • Alexander V. Tikhonov's avatar
      github-ci: switch coverage saving from travis-ci · 7aa0b018
      Alexander V. Tikhonov authored
      Moved coverage saving to coveralls.io repository from travis-ci to
      github-ci. Completely removed travis-ci from commit criteria.
      
      Part of #5294
      7aa0b018
    • Alexander V. Tikhonov's avatar
      github-ci: implement OSX commit testing · 70f2bd5f
      Alexander V. Tikhonov authored
      Implemented github-ci action workflow OSX jobs on commits:
       - OSX 10.15
       - OSX 11.0
      
      Part of #5294
      70f2bd5f
    • Alexander V. Tikhonov's avatar
      test: setup workspace in tmpfs for OOS build · dfcefb63
      Alexander V. Tikhonov authored
      Found that running vinyl test suite in parallel using test-run vardir
      on real hard drive may cause a lot of tests to fail. It happens because
      of bottleneck with hard drive usage up to 100% which can be seen by any
      of the tools like atop during vinyl tests run in parallel. To avoid of
      it all heavy loaded testing processes should use tmpfs for vardir path.
      Found that out-of-source build had to be updated to use tmpfs for it.
      This patch mounts additional tmpfs mount point in OOS build docker run
      process for test-run vardir. This mount point set using '--tmpfs' flag
      because '--mount' does not support 'exec' option which is needed to be
      able to execute commands in it [2][3].
      
      Issues met on OOS before the patch, like described in #5504 and [1]:
      
        Test hung! Result content mismatch:
        --- vinyl/write_iterator.result	Fri Nov 20 14:48:24 2020
        +++ /rw_bins/test/var/081_vinyl/write_iterator.result	Fri Nov 20 15:01:54 2020
        @@ -200,831 +200,3 @@
         ---
         ...
         for i = 1, 100 do space:insert{i, ''..i} if i % 2 == 0 then box.snapshot() end end
        ----
        -...
        -space:delete{1}
        ----
        -...
      
      Closes #5622
      Part of #5504
      
      [1] - https://gitlab.com/tarantool/tarantool/-/jobs/863266476#L5009
      [2] - https://stackoverflow.com/questions/54729130/how-to-mount-docker-tmpfs-with-exec-rw-flags
      [3] - https://github.com/moby/moby/issues/35890
      dfcefb63
  18. Oct 30, 2020
    • Sergey Bronnikov's avatar
      gitlab-ci: enhance jobs with jepsen tests · 4ab0ddcc
      Sergey Bronnikov authored
      To run Jepsen tests in different configurations we need to parametrize run
      script by options, so lein options and number of nodes passed with environment
      variables. By default script runs testing with Tarantool built from latest
      commit.
      
      Added these configurations:
      
      - single instance
      - single instance with enabled TXM
      - cluster with enabled Raft
      - cluster with enabled Raft and TXM
      
      Closes #5437
      4ab0ddcc
  19. Oct 16, 2020
  20. Oct 06, 2020
  21. Sep 25, 2020
    • Alexander V. Tikhonov's avatar
      gitlab-ci: save failed test results artifacts · 814d3e27
      Alexander V. Tikhonov authored
      Added artifacts saver to all gitlab-ci jobs with testing.
      
      Gitlab-ci jobs saves its results files in the following paths:
      
        1. base jobs for testing different features:
          - test/var/artifacts
      
        2. OSX jobs:
          - ${OSX_VARDIR}/artifacts
      
        3. pack/deploy jobs:
          - build/usr/src/*/tarantool-*/test/var/artifacts
      
        4. VBOX jobs (freebsd_12) on virtual host:
          - ~/tarantool/test/var/artifacts
      
      In gitlab-ci configuration added 'after_script' section with script
      which collects from different test places 'artifacts' directories
      created by test-run tool. It saves 'artifacts' directories as root
      path in artifacts packages. User will be able to download these
      packages using gitlab-ci GUI either API.
      
      Additionally added OSX_VARDIR environment variable to be able to
      setup common path for artifacts and OSX shell scripts options.
      
        OSX_VARDIR: /tmp/tnt
      
      Part of #5050
      814d3e27
    • Sergey Bronnikov's avatar
      cmake: move jepsen targets under option WITH_JEPSEN · a36749de
      Sergey Bronnikov authored
      For running Jepsen tests we need to checkout external repository with tests
      source code on a build stage. This behaviour brokes a Tarantool build under
      Gentoo. Option WITH_JEPSEN enables targets only when they needed.
      
      Closes #5325
      a36749de
  22. Sep 18, 2020
    • Sergey Bronnikov's avatar
      ci: integrate Jepsen tests to GitLab CI · a8e89b77
      Sergey Bronnikov authored
      added a new stage with a single job to run Jepsen tests.
      Job is not started automatically by default, one need to
      trigger it manually. Directory with test results
      (logs, graphs, operations history) published to artifacts.
      
      Closes #5277
      a8e89b77
  23. Sep 15, 2020
    • HustonMmmavr's avatar
      build: refactor static build process · 800e5ed6
      HustonMmmavr authored
      
      Refactored static build process to use static-build/CMakeLists.txt
      instead of Dockerfile.staticbuild (this allows to support static
      build on macOS). Following third-party dependencies for static build
      are installed via cmake `ExternalProject_Add`:
        - OpenSSL
        - Zlib
        - Ncurses
        - Readline
        - Unwind
        - ICU
      
      * Added support static build for macOS
      * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
        build at file cmake/BuildLibCURL.cmake:
          - disable building shared libcurl libraries (by setting
            `--disable-shared` option)
          - disable hiding libcurl symbols (by setting
            `--disable-symbol-hiding` option)
          - prevent linking libcurl with system libz (by setting
            `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
      * Removed Dockerfile.staticbuild
      * Added new gitlab.ci jobs to test new style static build:
        - static_build_cmake_linux
        - static_build_cmake_osx_15
      * Removed static_docker_build gitlab.ci job
      
      Closes #5095
      
      Co-authored-by: default avatarYaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
      800e5ed6
  24. Jul 15, 2020
  25. Jun 08, 2020
    • Vladislav Shpilevoy's avatar
      cmake: add option ENABLE_UB_SANITIZER · 366cb668
      Vladislav Shpilevoy authored
      Clang has a built-in sanitizer for undefined behaviour. Such as
      wrong memory alignment, array boundaries violation, 0 division,
      bool values with non standard content, etc.
      
      The sanitizer emits runtime checks which lead to either crash, or
      a trap, or a warning print, depending on what is chosen.
      
      The patch makes it possible to turn the sanitizer on and catch
      UBs. The only supported UB so far is alignment check. Other types
      can be added gradually, along with fixing bugs which they find.
      
      The UB sanitizer is activated for ASAN builds in CI.
      
      Part of #4609
      366cb668
  26. May 15, 2020
  27. Apr 15, 2020
  28. Apr 02, 2020
    • Alexander V. Tikhonov's avatar
      static build: create new build w/o dockerfile · fc55875e
      Alexander V. Tikhonov authored
      Fixed static build with '-DBUILD_STATIC=ON' option:
      
       - installed liblzma-dev library for libunwind static, due to found that
         static libunwind library uses undefined lzma functions:
           nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma
                       U lzma_index_buffer_decode
                       U lzma_index_end
                       U lzma_index_size
                       U lzma_index_uncompressed_size
                       U lzma_stream_buffer_decode
                       U lzma_stream_footer_decode
         while dynamic libunwind correctly sees liblzma installed:
           ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma
             liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000)
         so to fix it the static library of lzma was needed.
      
       - added lzma library to unwind library for Tarantool build at file:
           cmake/compiler.cmake
         due to fail:
           /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
             In function `xz_uncompressed_size':
           ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode'
           ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode'
           ./src/elfxx.c:205: undefined reference to `lzma_index_size'
           ./src/elfxx.c:210: undefined reference to `lzma_index_end'
           ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size'
           ./src/elfxx.c:210: undefined reference to `lzma_index_end'
           /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
             In function `_Uelf64_extract_minidebuginfo':
           ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode'
           collect2: error: ld returned 1 exit status
           test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134:
             recipe for target 'test/unit/luaL_iterator.test' failed
           make[2]: *** [test/unit/luaL_iterator.test] Error 1
      
       - added dl library to gomp library for test/unit tests
         binaries builds at file:
           cmake/BuildMisc.cmake
         due to fail:
           /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d):
             more undefined references to `dlsym' follow
           /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function
             `gomp_target_init':
           (.text+0x9cc): undefined reference to `dlerror'
           collect2: error: ld returned 1 exit status
      
        - added dl library to icu library for test/unit tests
         binaries builds at file:
           cmake/FindICU.cmake
         due to fail:
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dl_open_60':
           (.text+0x1ce2): undefined reference to `dlopen'
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dlsym_func_60':
           (.text+0x1d3d): undefined reference to `dlsym'
           /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
             In function `uprv_dl_close_60':
           (.text+0x1d21): undefined reference to `dlclose'
           collect2: error: ld returned 1 exit status
      
      Added static build to gitlab-ci in release check criteria named
      as static_build job. Previously named static_build job renamed to
      static_docker_build, due to it checks the build at Dockerfile.
      
      Also moved static build make targets from .gitlab.mk to .travis.mk
      to store it in common place with the other test/build make targets.
      Moved environement from .gitlab-ci.yml file into make targets to
      make this targets true building in static w/o additional setup.
      
      Close #4551
      fc55875e
    • Alexander V. Tikhonov's avatar
      gitlab-ci: add memory leaks detection via LSAN · e737400d
      Alexander V. Tikhonov authored
      The change enables memory leaks detection to existing ASAN testing
      routine and introduces suppression files with the corresponding
      exception list:
       * address sanitizer for compile-time: asan/asan.supp
       * memory leak sanitizer for run-time: asan/lsan.supp
      
      Furthermore, added engine and replication suites for ASAN testing
      routine.
      
      Additionally to the tests blacklisted within #4359,
      'box/on_shutdown.test.lua' is also disabled since it fails the
      introduced leak check. All blacklisted tests have to be enabled
      within #4360.
      
      Close #2058
      e737400d
  29. Mar 26, 2020
    • Alexander V. Tikhonov's avatar
      test: fix OSX host setup · a41bef3b
      Alexander V. Tikhonov authored
      Fixed OSX host setup for Tarantool build:
      - set brew installation from Homebrew repository instructions;
      - set in use Python 2 latest commit from tapped local formula,
        since Python 2 is EOL, also removed extra pip installation with
        get-pip script, because tapped formula installs pip itself.
        python@2 was deleted from homebrew/core in commit 028f11f9e:
          python@2: delete (https://github.com/Homebrew/homebrew-core/issues/49796)
          EOL 1 January 2020.
        Tapped formula created from the latest formula before its removal:
          git -C "$(brew --repo homebrew/core)" show 028f11f9e^:Formula/python@2.rb
      - added upgrade packages call to avoid of fails on already
        installed packages, but with previous version;
      - fixed the gitlab-ci configuration for sudo on testing hosts and removed pip
        option '--user' to avoid of use the users paths with special setup for it.
      
      Fixed OSX host setup for Tarantool test:
      - set maximum processes limit value to 2500 for testing process;
      - new Mac machines are going to be added into CI and usernames on them
        are long according to internal policies. It makes a home directory to
        be long too and so a path to a unix socket created during testing can
        be longer then UNIX_PATH_MAX=108 constant which is described as issue
          https://github.com/tarantool/tarantool/issues/4634
        To avoid of it the short working directory for testing set by option:
          --vardir /tmp/tnt
      a41bef3b
Loading